Original: E XPRESSIVE P OWER OF I NVARIANT AND E QUIVARIANT G RAPH N EURAL N ETWORKS
Revised: E XPRESSIVE P OWER OF I NVARIANT AND E QUIVARIANT G RAPH N EURAL N ETWORKS

Original: A BSTRACT
Revised: A BSTRACT

Original: Various classes of Graph Neural Networks (GNN) have been proposed and shown to be successful in a wide range of applications with graph structured data.
Revised: Various classes of Graph Neural Networks (GNN) have been proposed and shown to be successful in a wide range of applications with graph structured data.

Original: In this paper, we propose a theoretical framework able to compare the expressive power of these GNN architectures.
Revised: In this paper, we propose a theoretical framework able to compare the expressive power of these GNN architectures.

Original: The current universality theorems only apply to intractable classes of GNNs.
Revised: The current universality theorems only apply to intractable classes of GNNs.

Original: Here, we prove the ﬁrst approximation guarantees for practical GNNs, paving the way for a better understanding of their generalization.
Revised: Here, we prove the ﬁrst approximation guarantees for practical GNNs, paving the way for a better understanding of their generalization.

Original: Our theoretical results are proved for invariant GNNs computing a graph embedding (permutation of the nodes of the input graph does not affect the output) and equivariant GNNs computing an embedding of the nodes (permutation of the input permutes the output).
Revised: Our theoretical results are proved for invariant GNNs computing a graph embedding (permutation of the nodes of the input graph does not affect the output) and equivariant GNNs computing an embedding of the nodes (permutation of the input permutes the output).

Original: We show that Folklore Graph Neural Networks (FGNN), which are tensor based GNNs augmented with matrix multiplication are the most expressive architectures proposed so far for a given tensor order.
Revised: We show that Folklore Graph Neural Networks (FGNN), which are tensor based GNNs augmented with matrix multiplication are the most expressive architectures proposed so far for a given tensor order.

Original: We illustrate our results on the Quadratic Assignment Problem (a NP-Hard combinatorial problem) by showing that FGNNs are able to learn how to solve the problem, leading to much better average performances than existing algorithms (based on spectral,
Revised: We illustrate our results on the Quadratic Assignment Problem (a NP-Hard combinatorial problem) by showing that FGNNs are able to learn how to solve the problem, leading to much better average performances than existing algorithms (based on spectral,

Original: SDP or other GNNs architectures).
Revised: SDP or other GNNs architectures).

Original: On a practical side, we also implement masked tensors to handle batches of graphs of varying sizes.
Revised: On a practical side, we also implement masked tensors to handle batches of graphs of varying sizes.

Original:  I NTRODUCTION
Revised:  I NTRODUCTION

Original: Graph Neural Networks (GNN) are designed to deal with graph structured data.
Revised: Graph Neural Networks (GNN) are designed to deal with graph structured data.

Original: Since a graph is not changed by permutation of its nodes, GNNs should be either invariant if they return a result that must not depend on the representation of the input (typically when building a graph embedding) or equivariant if the output must be permuted when the input is permuted (typically when building an embedding of the nodes).
Revised: Since a graph is not changed by permutation of its nodes, GNNs should be either invariant if they return a result that must not depend on the representation of the input (typically when building a graph embedding) or equivariant if the output must be permuted when the input is permuted (typically when building an embedding of the nodes).

Original: More fundamentally, incorporating symmetries in machine learning is a fundamental problem as it allows to reduce the number of degree of freedom to be learned.
Revised: More fundamentally, incorporating symmetries in machine learning is a fundamental problem as it allows to reduce the number of degree of freedom to be learned.

Original: Deep learning on graphs.
Revised: Deep learning on graphs.

Original: This paper focuses on learning deep representation of graphs with network architectures, namely GNN, designed to be invariant to permutation or equivariant by permutation.
Revised: This paper focuses on learning deep representation of graphs with network architectures, namely GNN, designed to be invariant to permutation or equivariant by permutation.

Original: From a practical perspective, various message passing GNNs have been proposed, see Dwivedi et al.
Revised: From a practical perspective, various message passing GNNs have been proposed, see Dwivedi et al.

Original: (2020) for a recent survey and benchmarking on learning tasks.
Revised: (2020) for a recent survey and benchmarking on learning tasks.

Original: In this paper, we study 3 architectures:
Revised: In this paper, we study 3 architectures:

Original: Message passing GNN (MGNN) which is probably the most popular architecture used in practice, order-
Revised: Message passing GNN (MGNN) which is probably the most popular architecture used in practice, order-

Original: k Linear GNN ( k -LGNN) proposed in Maron et al.
Revised: k Linear GNN ( k -LGNN) proposed in Maron et al.

Original: and order- k Folklore GNN ( kFGNN) ﬁrst introduced by Maron et al. (2019a).
Revised: and order- k Folklore GNN ( kFGNN) ﬁrst introduced by Maron et al. (2019a).

Original: MGNN layers are local thus highly parallelizable on GPUs which make them scalable for large sparse graphs.
Revised: MGNN layers are local thus highly parallelizable on GPUs which make them scalable for large sparse graphs.

Original: k -LGNN and k -FGNN are dealing with representations of graphs as tensors of order k which make them of little practical use for k ≥ 3 .
Revised: k -LGNN and k -FGNN are dealing with representations of graphs as tensors of order k which make them of little practical use for k ≥ 3 .

Original: In order to compare these architectures, the separating power of these networks has been compared to a hierarchy of graph invariants developed for the graph isomorphism problem.
Revised: In order to compare these architectures, the separating power of these networks has been compared to a hierarchy of graph invariants developed for the graph isomorphism problem.

Original: Namely, for k ≥ 2 , k -WL ( G ) are invariants based on the Weisfeiler-Lehman tests (described in Section 4.1).
Revised: Namely, for k ≥ 2 , k -WL ( G ) are invariants based on the Weisfeiler-Lehman tests (described in Section 4.1).

Original: For each k ≥ 2 , ( k + 1) -WL has strictly more separating power than k -WL
Revised: For each k ≥ 2 , ( k + 1) -WL has strictly more separating power than k -WL

Original: (in the sense that there is a pair of non-isomorphic graphs distinguishable by ( k + 1) -WL and not by k -WL).
Revised: (in the sense that there is a pair of non-isomorphic graphs distinguishable by ( k + 1) -WL and not by k -WL).

Original: GIN (which are invariant
Revised: GIN (which are invariant

Original: MGNN) introduced in Xu et al.
Revised: MGNN) introduced in Xu et al.

Original: (2018) are shown to be as powerful as 2 -WL.
Revised: (2018) are shown to be as powerful as 2 -WL.

Original: In Maron et al.
Revised: In Maron et al.

Original: (2019a),
Revised: (2019a),

Original: Geerts (2020b) and Geerts (2020a), k -LGNN are shown to be as powerful as k -WL and 2 -FGNN is shown to be as powerful as 3 -WL.
Revised: Geerts (2020b) and Geerts (2020a), k -LGNN are shown to be as powerful as k -WL and 2 -FGNN is shown to be as powerful as 3 -WL.

Original: In this paper, we extend this last result about k -FGNN to general values of k .
Revised: In this paper, we extend this last result about k -FGNN to general values of k .

Original: So in term of separating power, when restricted to tensors of order k , k -FGNN is the most powerful architecture among the ones considered in this work.
Revised: So in term of separating power, when restricted to tensors of order k , k -FGNN is the most powerful architecture among the ones considered in this work.

Original: This means that for a given pair of graphs G and G (cid:48) , if ( k + 1) -WL ( G ) (cid:54) =
Revised: This means that for a given pair of graphs G and G (cid:48) , if ( k + 1) -WL ( G ) (cid:54) =

Original: ( k + 1) -WL ( G (cid:48) ) , then there exists a k -FGNN, say
Revised: ( k + 1) -WL ( G (cid:48) ) , then there exists a k -FGNN, say

Original: GNN G,G (cid:48) such that GNN G,G (cid:48) ( G ) (cid:54) =
Revised: GNN G,G (cid:48) such that GNN G,G (cid:48) ( G ) (cid:54) =

Original: GNN G,G (cid:48)
Revised: GNN G,G (cid:48)

Original: ( G (cid:48) ) .
Revised: ( G (cid:48) ) .

Original: Approximation results for GNNs.
Revised: Approximation results for GNNs.

Original: Results on the separating power of GNNs only deal with pairwise comparison of graphs: we need a priori a different GNN for each pair of graphs in order to distinguish them.
Revised: Results on the separating power of GNNs only deal with pairwise comparison of graphs: we need a priori a different GNN for each pair of graphs in order to distinguish them.

Original: Such results are of little help in a practical learning scenario.
Revised: Such results are of little help in a practical learning scenario.

Original: Our main contribution in this paper overcomes this issue and we show that a single GNN can give a meaningful representation for all graphs.
Revised: Our main contribution in this paper overcomes this issue and we show that a single GNN can give a meaningful representation for all graphs.

Original: More precisely, we characterize the set of functions that can be approximated by MGNNs, k -LGNNs and k -FGNNs respectively.
Revised: More precisely, we characterize the set of functions that can be approximated by MGNNs, k -LGNNs and k -FGNNs respectively.

Original: Standard Stone-Weierstrass theorem shows that if an algebra
Revised: Standard Stone-Weierstrass theorem shows that if an algebra

Original: A of real continuous functions separates points, then A is dense in the set of continuous function on a compact set.
Revised: A of real continuous functions separates points, then A is dense in the set of continuous function on a compact set.

Original: Here we extend such a theorem to general functions with symmetries and apply it to invariant and equivaraint functions to get our main result for GNNs.
Revised: Here we extend such a theorem to general functions with symmetries and apply it to invariant and equivaraint functions to get our main result for GNNs.

Original: As a consequence, we show that k -FGNNs have the best approximation power among architectures dealing with tensors of order k .
Revised: As a consequence, we show that k -FGNNs have the best approximation power among architectures dealing with tensors of order k .

Original: Universality results for GNNs.
Revised: Universality results for GNNs.

Original: Universal approximation theorems (similar to Cybenko (1989) for multi-layers perceptron) have been proved for linear GNNs in Maron et al.
Revised: Universal approximation theorems (similar to Cybenko (1989) for multi-layers perceptron) have been proved for linear GNNs in Maron et al.

Original: (2019b)
Revised: (2019b)

Original: ; Keriven & Peyré (2019); Chen et al.
Revised: ; Keriven & Peyré (2019); Chen et al.

Original: They show that some classes of GNNs can approximate any function deﬁned on graphs.
Revised: They show that some classes of GNNs can approximate any function deﬁned on graphs.

Original: To be able to approximate any invariant function, they require the use of very complex networks, namely k -LGNN where k tends to inﬁnity with n the number of nodes.
Revised: To be able to approximate any invariant function, they require the use of very complex networks, namely k -LGNN where k tends to inﬁnity with n the number of nodes.

Original: Since we prove that any invariant function less powerful than ( k + 1) -WL can be approximated by a k -FGNN, letting k tends to inﬁnity directly implies universality.
Revised: Since we prove that any invariant function less powerful than ( k + 1) -WL can be approximated by a k -FGNN, letting k tends to inﬁnity directly implies universality.

Original: Universality results for k -FGNN is another contribution of our work.
Revised: Universality results for k -FGNN is another contribution of our work.

Original: Equivariant GNNs.
Revised: Equivariant GNNs.

Original: Our second set of results extends previous analysis from invariant functions to equivariant functions.
Revised: Our second set of results extends previous analysis from invariant functions to equivariant functions.

Original: There are much less results about equivariant GNNs: Keriven & Peyré (2019) proves the universality of linear equivariant GNNs, and Maehara & Hoang (2019) shows the universality of a new class of networks they introduced.
Revised: There are much less results about equivariant GNNs: Keriven & Peyré (2019) proves the universality of linear equivariant GNNs, and Maehara & Hoang (2019) shows the universality of a new class of networks they introduced.

Original: Here, we consider a natural equivariant extension of k -WL and prove that equivariant ( k + 1)
Revised: Here, we consider a natural equivariant extension of k -WL and prove that equivariant ( k + 1)

Original: -LGNNs and k -FGNN can approximate any equivariant function less powerful than this equivariant ( k + 1) -WL for k ≥ 1 .
Revised: -LGNNs and k -FGNN can approximate any equivariant function less powerful than this equivariant ( k + 1) -WL for k ≥ 1 .

Original: At this stage, we should note that all universality results for GNNs by Maron et al.
Revised: At this stage, we should note that all universality results for GNNs by Maron et al.

Original: (2019b)
Revised: (2019b)

Original: ; Keriven & Peyré (2019);
Revised: ; Keriven & Peyré (2019);

Original: Chen et al.
Revised: Chen et al.

Original: (2019) are easily recovered from our main results.
Revised: (2019) are easily recovered from our main results.

Original: Also our analysis is valid for graphs of varying sizes.
Revised: Also our analysis is valid for graphs of varying sizes.

Original: Empirical results for the Quadratic Assigment Problem (QAP).
Revised: Empirical results for the Quadratic Assigment Problem (QAP).

Original: To validate our theoretical contributions, we empirically show that 2 -FGNN outperforms classical MGNN.
Revised: To validate our theoretical contributions, we empirically show that 2 -FGNN outperforms classical MGNN.

Original: Indeed, Maron et al.
Revised: Indeed, Maron et al.

Original: (2019a) already demonstrate state of the art results for the invariant version of 2 -FGNNs (for graph classiﬁcation or graph regression).
Revised: (2019a) already demonstrate state of the art results for the invariant version of 2 -FGNNs (for graph classiﬁcation or graph regression).

Original: Here we consider the graph alignment problem and show that the equivariant 2 -FGNN is able to learn a node embedding which beats by a large margin other algorithms (based on spectral method, SDP or GNNs).
Revised: Here we consider the graph alignment problem and show that the equivariant 2 -FGNN is able to learn a node embedding which beats by a large margin other algorithms (based on spectral method, SDP or GNNs).

Original: Outline and contribution.
Revised: Outline and contribution.

Original: After reviewing more previous works and notations in the next section, we deﬁne the various classes of GNNs studied in this paper in Section 3 : message passing GNN, linear GNN and folklore GNN.
Revised: After reviewing more previous works and notations in the next section, we deﬁne the various classes of GNNs studied in this paper in Section 3 : message passing GNN, linear GNN and folklore GNN.

Original: Section 4 contains our main theoretical results for GNNs.
Revised: Section 4 contains our main theoretical results for GNNs.

Original: First in
Revised: First in

Original: Section 4.2 we describe the separating power of each GNN architecture with respect to the WeisfeilerLehman test.
Revised: Section 4.2 we describe the separating power of each GNN architecture with respect to the WeisfeilerLehman test.

Original: In Section 4.3, we give approximation guarantees for MGNNs, LGNNs and FGNNs at ﬁxed order of tensor.
Revised: In Section 4.3, we give approximation guarantees for MGNNs, LGNNs and FGNNs at ﬁxed order of tensor.

Original: They cover both the invariant and equivariant cases and are our main theoretical contributions.
Revised: They cover both the invariant and equivariant cases and are our main theoretical contributions.

Original: For these, we develop in Section D a ﬁne-grained Stone-Weierstrass approximation theorem for vector-valued functions with symmetries.
Revised: For these, we develop in Section D a ﬁne-grained Stone-Weierstrass approximation theorem for vector-valued functions with symmetries.

Original: Our theorem handles both invariant and equivariant cases and is inspired by recent works in approximation theory.
Revised: Our theorem handles both invariant and equivariant cases and is inspired by recent works in approximation theory.

Original: In Section 6, we illustrate our theoretical results on a practical application: the graph alignment problem, a well-known NP-hard problem.
Revised: In Section 6, we illustrate our theoretical results on a practical application: the graph alignment problem, a well-known NP-hard problem.

Original: We highlight a previously overlooked implementation question: the handling of batches of graphs of varying sizes.
Revised: We highlight a previously overlooked implementation question: the handling of batches of graphs of varying sizes.

Original: A PyTorch implementation of the code necessary to reproduce the results is available at https://github.com/mlelarge/graph_neural_net
Revised: A PyTorch implementation of the code necessary to reproduce the results is available at https://github.com/mlelarge/graph_neural_net

Original:  R ELATED WORK
Revised:  R ELATED WORK

Original: The pioneering works that applied neural networks to graphs are Gori et al.
Revised: The pioneering works that applied neural networks to graphs are Gori et al.

Original: and Scarselli et al.
Revised: and Scarselli et al.

Original: (2009) that learn node representation with recurrent neural networks.
Revised: (2009) that learn node representation with recurrent neural networks.

Original: More recent message passing architectures make use of non-linear functions of the adjacency matrix (Kipf & Welling, 2016), for example polynomials (Defferrard et al., 2016).
Revised: More recent message passing architectures make use of non-linear functions of the adjacency matrix (Kipf & Welling, 2016), for example polynomials (Defferrard et al., 2016).

Original: For regular-grid graphs, they match classical convolutional networks which by design can only approximate translation-invariant functions and hence have limited expressive power.
Revised: For regular-grid graphs, they match classical convolutional networks which by design can only approximate translation-invariant functions and hence have limited expressive power.

Original: In this paper, we focus instead on more expressive architectures.
Revised: In this paper, we focus instead on more expressive architectures.

Original: Following the recent surge in interest in graph neural networks, some works have tried to extend the pioneering work of Cybenko (1989);
Revised: Following the recent surge in interest in graph neural networks, some works have tried to extend the pioneering work of Cybenko (1989);

Original: Hornik et al.
Revised: Hornik et al.

Original: (1989) for various GNN architectures.
Revised: (1989) for various GNN architectures.

Original: Among the ﬁrst ones is Scarselli et al.
Revised: Among the ﬁrst ones is Scarselli et al.

Original: (2009), which studied invariant message-passing GNNs.
Revised: (2009), which studied invariant message-passing GNNs.

Original: They showed that such networks can approximate, in a weak sense, all functions whose discriminatory power is weaker than 1 -WL.
Revised: They showed that such networks can approximate, in a weak sense, all functions whose discriminatory power is weaker than 1 -WL.

Original: Yarotsky (2018) described universal architectures which are invariant or equivariant to some group action.
Revised: Yarotsky (2018) described universal architectures which are invariant or equivariant to some group action.

Original: These models rely on polynomial intermediate layers of arbitrary degrees, which would be prohibitive in practice.
Revised: These models rely on polynomial intermediate layers of arbitrary degrees, which would be prohibitive in practice.

Original: Maron et al.
Revised: Maron et al.

Original: (2019b) leveraged classical results about the polynomials invariant to a group action to show that k -LGNN are universal as k tends to inﬁnity with the number of nodes.
Revised: (2019b) leveraged classical results about the polynomials invariant to a group action to show that k -LGNN are universal as k tends to inﬁnity with the number of nodes.

Original: Keriven & Peyré (2019) derived a similar result, in the more complicated equivariant case by introducing a new Stone-Weierstrass theorem.
Revised: Keriven & Peyré (2019) derived a similar result, in the more complicated equivariant case by introducing a new Stone-Weierstrass theorem.

Original: Similarly to Maron et al. (2019b), they require the order of tensors to go to inﬁnity.
Revised: Similarly to Maron et al. (2019b), they require the order of tensors to go to inﬁnity.

Original: Another route towards universality is the one of
Revised: Another route towards universality is the one of

Original: Chen et al.
Revised: Chen et al.

Original: In the invariant setting, they show for a class of GNN that universality is equivalent to being able to discriminate between (non-isomorphic) graphs.
Revised: In the invariant setting, they show for a class of GNN that universality is equivalent to being able to discriminate between (non-isomorphic) graphs.

Original: However, the only way to achieve such discriminatory power is to use tensors of arbitrary high order, see also Ravanbakhsh (2020).
Revised: However, the only way to achieve such discriminatory power is to use tensors of arbitrary high order, see also Ravanbakhsh (2020).

Original: Our work encompass and precise these results using high-order tensors as it yields approximation guarantees even at ﬁxed order of tensor.
Revised: Our work encompass and precise these results using high-order tensors as it yields approximation guarantees even at ﬁxed order of tensor.

Original: CPNGNN in Sato et al.
Revised: CPNGNN in Sato et al.

Original: (2019) and DimeNet in Klicpera et al.
Revised: (2019) and DimeNet in Klicpera et al.

Original: (2020) are message passing GNN incorporating more information than those studied here.
Revised: (2020) are message passing GNN incorporating more information than those studied here.

Original: Partial results about their separating power follows from Garg et al.
Revised: Partial results about their separating power follows from Garg et al.

Original: (2020) which provides impossibility results to decide graph properties including girth, circumference, diameter, radius, conjoint cycle, total number of cycles, and k -cliques.
Revised: (2020) which provides impossibility results to decide graph properties including girth, circumference, diameter, radius, conjoint cycle, total number of cycles, and k -cliques.

Original: Chen et al.
Revised: Chen et al.

Original: (2020) studies the ability of GNNs to count graph substructures.
Revised: (2020) studies the ability of GNNs to count graph substructures.

Original: Though our theorems are much more general, note that their results are improved by the present work.
Revised: Though our theorems are much more general, note that their results are improved by the present work.

Original: Note also, that if the nodes are given distinct features, MGNNs become much more expressive Loukas (2019) but looses their invariant or equivariant properties.
Revised: Note also, that if the nodes are given distinct features, MGNNs become much more expressive Loukas (2019) but looses their invariant or equivariant properties.

Original: Averaging i.e. relational pooling (RP) has been proposed to recover these properties Murphy et al. (2019a).
Revised: Averaging i.e. relational pooling (RP) has been proposed to recover these properties Murphy et al. (2019a).

Original: However, the ideal RP, leading to a universal approximation, cannot be used for large graphs due to its complexity of O ( | V | !) .
Revised: However, the ideal RP, leading to a universal approximation, cannot be used for large graphs due to its complexity of O ( | V | !) .

Original: Regarding the other classes of RPGNN i.e. the k -ary pooling (Murphy et al., 2019b), we will show how our general theorems in the invariant case can be applied to characterize their approximation power (see Section 5).
Revised: Regarding the other classes of RPGNN i.e. the k -ary pooling (Murphy et al., 2019b), we will show how our general theorems in the invariant case can be applied to characterize their approximation power (see Section 5).

Original: Note that for neural networks on sets, the situation is a bit simpler.
Revised: Note that for neural networks on sets, the situation is a bit simpler.

Original: Efﬁcient architectures such as
Revised: Efﬁcient architectures such as

Original: DeepSets (Zaheer et al., 2017) or PointNet (Qi et al., 2017) have been shown to be invariant universal.
Revised: DeepSets (Zaheer et al., 2017) or PointNet (Qi et al., 2017) have been shown to be invariant universal.

Original: Similar results exist in the equivariant case (Segol & Lipman, 2020; Maron et al., 2020), whose proofs rely on polynomial arguments.
Revised: Similar results exist in the equivariant case (Segol & Lipman, 2020; Maron et al., 2020), whose proofs rely on polynomial arguments.

Original: Though this is not our main motivation, our approximation theorems could also be applied in this context see Sections D.3 and D.4.
Revised: Though this is not our main motivation, our approximation theorems could also be applied in this context see Sections D.3 and D.4.

Original: 2. N OTATIONS : GRAPHS AS TENSORS
Revised: 2. N OTATIONS : GRAPHS AS TENSORS

Original: We denote by F , F 0 , F 1 / 2 , F 1 , . . .
Revised: We denote by F , F 0 , F 1 / 2 , F 1 , . . .

Original: arbitrary ﬁnite-dimensional spaces of the form R p (for various values of p ) typically representing the space of features.
Revised: arbitrary ﬁnite-dimensional spaces of the form R p (for various values of p ) typically representing the space of features.

Original: Product of vectors in R p always refer to component-wise product.
Revised: Product of vectors in R p always refer to component-wise product.

Original: There are two ways to see graphs with features.
Revised: There are two ways to see graphs with features.

Original: First, graphs can be seen as tensors of order k : G ∈ F n k .
Revised: First, graphs can be seen as tensors of order k : G ∈ F n k .

Original: The classical representation of a graph by its (weighted) adjacency matrix for k = 2 is a tensor of order 2 in R n 2 .
Revised: The classical representation of a graph by its (weighted) adjacency matrix for k = 2 is a tensor of order 2 in R n 2 .

Original: This case allows for features on edges by replacing
Revised: This case allows for features on edges by replacing

Original: R n 2 with F n 2 where F is some R p .
Revised: R n 2 with F n 2 where F is some R p .

Original: Second, graphs can also be represented by their discrete structure with an additional feature vector.
Revised: Second, graphs can also be represented by their discrete structure with an additional feature vector.

Original: More exactly, denote by G n the set of discrete graphs G =
Revised: More exactly, denote by G n the set of discrete graphs G =

Original: ( V, E ) with n nodes V =
Revised: ( V, E ) with n nodes V =

Original: [ n ] and edges E ⊆
Revised: [ n ] and edges E ⊆

Original: V 2 (with no weights on edges).
Revised: V 2 (with no weights on edges).

Original: Such a G ∈ G n with a vector h 0 ∈ F n represents a graphs with features on the vertices.
Revised: Such a G ∈ G n with a vector h 0 ∈ F n represents a graphs with features on the vertices.

Original: 2. D EFINITIONS : I NVARIANT AND EQUIVARIANT OPERATORS
Revised: 2. D EFINITIONS : I NVARIANT AND EQUIVARIANT OPERATORS

Original: Let [ n ] = { 1 , . . . , n } .
Revised: Let [ n ] = { 1 , . . . , n } .

Original: The set of permutations on [ n ] is denoted by S n .
Revised: The set of permutations on [ n ] is denoted by S n .

Original: For G ∈ F n k and σ ∈ S n , we deﬁne: ( σ (cid:63) G ) σ ( i 1 ) ,...,σ ( i k ) =
Revised: For G ∈ F n k and σ ∈ S n , we deﬁne: ( σ (cid:63) G ) σ ( i 1 ) ,...,σ ( i k ) =

Original: G i 1 ,...,i k .
Revised: G i 1 ,...,i k .

Original: Note that the (cid:63) operation is valid between a permutation in S n and a graph G as soon as the number of nodes of G is n , i.e. it is valid for any order k tensor representation of the graph.
Revised: Note that the (cid:63) operation is valid between a permutation in S n and a graph G as soon as the number of nodes of G is n , i.e. it is valid for any order k tensor representation of the graph.

Original: Two graphs G 1 , G 2 are said isomorphic if they have the same number of nodes and there exists a permutation σ such that G 1 = σ (cid:63) G 2 .
Revised: Two graphs G 1 , G 2 are said isomorphic if they have the same number of nodes and there exists a permutation σ such that G 1 = σ (cid:63) G 2 .

Original: Deﬁnition 1.
Revised: Deﬁnition 1.

Original: A function f : F n k→ F 1 is said to be invariant if f ( σ (cid:63) G ) =
Revised: A function f : F n k→ F 1 is said to be invariant if f ( σ (cid:63) G ) =

Original: f ( G ) for every permutation σ ∈ S n and every G ∈ F n k 0 .
Revised: f ( G ) for every permutation σ ∈ S n and every G ∈ F n k 0 .

Original: A function f : F n k 0 → F n (cid:96) 1 is said to be equivariant if f ( σ (cid:63) G )
Revised: A function f : F n k 0 → F n (cid:96) 1 is said to be equivariant if f ( σ (cid:63) G )

Original: =
Revised: =

Original: σ (cid:63) f
Revised: σ (cid:63) f

Original: ( G ) for every permutation σ ∈ S n and every G ∈ F n k 0 .
Revised: ( G ) for every permutation σ ∈ S n and every G ∈ F n k 0 .

Original: Note that composing an equivariant function with an invariant function gives an invariant function.
Revised: Note that composing an equivariant function with an invariant function gives an invariant function.

Original: For k ≥ 1 , we deﬁne the invariant summation layer S k : F n k → F by S k
Revised: For k ≥ 1 , we deﬁne the invariant summation layer S k : F n k → F by S k

Original: ( G ) =
Revised: ( G ) =

Original: (cid:80) i ∈ [ n ] k G i for
Revised: (cid:80) i ∈ [ n ] k G i for

Original: G ∈ F n k .
Revised: G ∈ F n k .

Original: We also deﬁne the equivariant reduction layer S k 1 : F n k → F n as follows:
Revised: We also deﬁne the equivariant reduction layer S k 1 : F n k → F n as follows:

Original: S k 1 ( G ) i = (cid:80) 1 ≤ i 2 ...
Revised: S k 1 ( G ) i = (cid:80) 1 ≤ i 2 ...

Original: i k ≤ n G i,i 2 ,...
Revised: i k ≤ n G i,i 2 ,...

Original: i k .
Revised: i k .

Original: For message passing GNN, we will use the equivariant layer Id + λS 1 : F n → F n deﬁned by, (Id + λS 1 )( G ) i
Revised: For message passing GNN, we will use the equivariant layer Id + λS 1 : F n → F n deﬁned by, (Id + λS 1 )( G ) i

Original: =
Revised: =

Original: G i + λS 1 ( G ) , where λ ∈ R is a learnable parameter.
Revised: G i + λS 1 ( G ) , where λ ∈ R is a learnable parameter.

Original: In the sequel, we will need a mapping I k lifting the input graph to a higher order tensor.
Revised: In the sequel, we will need a mapping I k lifting the input graph to a higher order tensor.

Original: We denote by
Revised: We denote by

Original: I k : F n 2 

Revised: I k : F n 2

Original: → F n k 1 the initialization function mapping for a given graph each k -tuple to its isomorphism type.
Revised: → F n k 1 the initialization function mapping for a given graph each k -tuple to its isomorphism type.

Original: We refer to the appendix Section C.3 for a precise description of this linear equivariant function.
Revised: We refer to the appendix Section C.3 for a precise description of this linear equivariant function.

Original: Note at this stage that I 2 is given by, for G ∈ F n 2 , I ( G ) i,j =
Revised: Note at this stage that I 2 is given by, for G ∈ F n 2 , I ( G ) i,j =

Original: ( G i,j , δ i,j ) where δ i,j is 0 if i (cid:54) =
Revised: ( G i,j , δ i,j ) where δ i,j is 0 if i (cid:54) =

Original: j and 1 otherwise.
Revised: j and 1 otherwise.

Original: Indeed for a pair of nodes i, j in a graph (without features), there are only three isomorphism types:
Revised: Indeed for a pair of nodes i, j in a graph (without features), there are only three isomorphism types:

Original: i =
Revised: i =

Original: j ; i (cid:54) =
Revised: j ; i (cid:54) =

Original: j and ( i, j ) is an edge; i (cid:54) =
Revised: j and ( i, j ) is an edge; i (cid:54) =

Original: j
Revised: j

Original: but ( i, j ) is not an edge.
Revised: but ( i, j ) is not an edge.

Original:  GNN DEFINITIONS
Revised:  GNN DEFINITIONS

Original: In this section, we deﬁne the various GNN architectures studied in this paper.
Revised: In this section, we deﬁne the various GNN architectures studied in this paper.

Original: In all architectures, there is a main building block or layer mapping F n k t to F n k t +1 where F n k t can be seen as the space for the representation of the graph at layer t .
Revised: In all architectures, there is a main building block or layer mapping F n k t to F n k t +1 where F n k t can be seen as the space for the representation of the graph at layer t .

Original: We will deﬁne three different types of layers for message passing GNN, linear GNN and folklore GNN.The case k = 2 is probably the most interesting case from a practical point view and corresponds to a case where a layer takes as input a graph (with features on nodes and edges) and produces as output a graph (with new features on nodes and edges).
Revised: We will deﬁne three different types of layers for message passing GNN, linear GNN and folklore GNN.The case k = 2 is probably the most interesting case from a practical point view and corresponds to a case where a layer takes as input a graph (with features on nodes and edges) and produces as output a graph (with new features on nodes and edges).

Original: For each type of GNNs, there will be an invariant and an equivaraint version.
Revised: For each type of GNNs, there will be an invariant and an equivaraint version.

Original: All architectures will share the last function:
Revised: All architectures will share the last function:

Original: m I : F T +1 → F for the invariant case and m E : F
Revised: m I : F T +1 → F for the invariant case and m E : F

Original: nT +1 → F n for the equivariant case which are continuous functions.
Revised: nT +1 → F n for the equivariant case which are continuous functions.

Original: It is typically modeled by a Multi Layer Perceptron, which is applied on each component for the equivariant case.
Revised: It is typically modeled by a Multi Layer Perceptron, which is applied on each component for the equivariant case.

Original: In words, each network takes as input a graph G ∈ F n 2 0 , produces in the invariant case a graph embedding in F T +1 and in the equivaraint case a node embedding in F nT +1 , then these embeddings are passed through the function m I or m E respectively to get a feature in F or F n for the learning task.
Revised: In words, each network takes as input a graph G ∈ F n 2 0 , produces in the invariant case a graph embedding in F T +1 and in the equivaraint case a node embedding in F nT +1 , then these embeddings are passed through the function m I or m E respectively to get a feature in F or F n for the learning task.

Original: 3. M ESSAGE PASSING GNN
Revised: 3. M ESSAGE PASSING GNN

Original: Message passing GNN (MGNN) are deﬁned for classical graphs G with features on the nodes.
Revised: Message passing GNN (MGNN) are deﬁned for classical graphs G with features on the nodes.

Original: More exactly they take as input a discrete graph G =
Revised: More exactly they take as input a discrete graph G =

Original: ( V, E ) ∈ G n and features on the nodes h 0 ∈ F n .
Revised: ( V, E ) ∈ G n and features on the nodes h 0 ∈ F n .

Original: MGNN are then deﬁned inductively as follows: let h (cid:96)i ∈ F (cid:96) denote the feature at layer (cid:96) associated with node i , the updated features h (cid:96) +

Revised: MGNN are then deﬁned inductively as follows: let h (cid:96)i ∈ F (cid:96) denote the feature at layer (cid:96) associated with node i , the updated features h (cid:96) +

Original: i are obtained as: h (cid:96) +1 i = f (cid:16) h
Revised: i are obtained as: h (cid:96) +1 i = f (cid:16) h

Original: (cid:96)i , (cid:8)(cid:8) h
Revised: (cid:96)i , (cid:8)(cid:8) h

Original: (cid:96)j (cid:9)(cid:9)
Revised: (cid:96)j (cid:9)(cid:9)

Original: j ∼ i (cid:17) , where j ∼ i means that nodes j and i are neighbors in the graph G , i.e. ( i, j ) ∈ E , and the function f is a learnable function taking as input the feature vector of the center vertex h
Revised: j ∼ i (cid:17) , where j ∼ i means that nodes j and i are neighbors in the graph G , i.e. ( i, j ) ∈ E , and the function f is a learnable function taking as input the feature vector of the center vertex h

Original: (cid:96)i and the multiset of features of the neighboring vertices (cid:8)(cid:8) h (cid:96)j (cid:9)(cid:9)
Revised: (cid:96)i and the multiset of features of the neighboring vertices (cid:8)(cid:8) h (cid:96)j (cid:9)(cid:9)

Original: j ∼ i .
Revised: j ∼ i .

Original: Indeed, it follows from Lem.
Revised: Indeed, it follows from Lem.

Original: 33 in Appendix, that any such function f can be approximated by a layer of the form,
Revised: 33 in Appendix, that any such function f can be approximated by a layer of the form,

Original: [Equation]
Revised: [Equation]

Original: where f 0 : F (cid:96) × F (cid:96) +1 / 2 → F (cid:96) +1 and f 1 : F (cid:96) × F (cid:96) → F (cid:96) +1 / 2 , so that F (cid:96) is the ﬁeld for the features at the (cid:96) -th layer.
Revised: where f 0 : F (cid:96) × F (cid:96) +1 / 2 → F (cid:96) +1 and f 1 : F (cid:96) × F (cid:96) → F (cid:96) +1 / 2 , so that F (cid:96) is the ﬁeld for the features at the (cid:96) -th layer.

Original: We call such a function a message passing layer and denote it by F : F n(cid:96)
Revised: We call such a function a message passing layer and denote it by F : F n(cid:96)

Original: → F n(cid:96)
Revised: → F n(cid:96)

Original: + (note that F depends implicitly from the graph).
Revised: + (note that F depends implicitly from the graph).

Original: Then an equivariant message passing GNN is simply obtained by the composition of message passing layers: F T ◦ . . .
Revised: Then an equivariant message passing GNN is simply obtained by the composition of message passing layers: F T ◦ . . .

Original: F 2 ◦ F 1 , where each F i is a message passing layer.
Revised: F 2 ◦ F 1 , where each F i is a message passing layer.

Original: Clearly since each F i is equivariant, this message passing GNN is also equivariant and produces features on each node in the space F T .
Revised: Clearly since each F i is equivariant, this message passing GNN is also equivariant and produces features on each node in the space F T .

Original: In order to obtain an invariant GNN, we apply an invariant function from F nT → F T +1 on the output of an equivariant message passing GNN.
Revised: In order to obtain an invariant GNN, we apply an invariant function from F nT → F T +1 on the output of an equivariant message passing GNN.

Original: In practice, a symmetric function is applied on the vectors of features indexed by the nodes, typically the sum of the features (cid:80) i
Revised: In practice, a symmetric function is applied on the vectors of features indexed by the nodes, typically the sum of the features (cid:80) i

Original: ( F T ◦ . . .
Revised: ( F T ◦ . . .

Original: F 2 ◦ F 1 ( G ))
Revised: F 2 ◦ F 1 ( G ))

Original: i is taken as an invariant feature for the graph G .
Revised: i is taken as an invariant feature for the graph G .

Original: With our notation, S 1 ◦ F T ◦ . . .
Revised: With our notation, S 1 ◦ F T ◦ . . .

Original: F 2 ◦ F 1 (where S 1 was deﬁned in Section2.2) deﬁnes an invariant message passing GNN.
Revised: F 2 ◦ F 1 (where S 1 was deﬁned in Section2.2) deﬁnes an invariant message passing GNN.

Original: Hence, we deﬁne the sets of message passing GNNs as follows:
Revised: Hence, we deﬁne the sets of message passing GNNs as follows:

Original: [Equation]
Revised: [Equation]

Original: where F t : F nt → F nt +1 are message passing layers.
Revised: where F t : F nt → F nt +1 are message passing layers.

Original: 3. L INEAR GNN
Revised: 3. L INEAR GNN

Original: We deﬁne the linear graph layer of order k as F : F n k (cid:96) → F n k (cid:96) +1 , where for all G ∈ F n k (cid:96) , F ( G ) =
Revised: We deﬁne the linear graph layer of order k as F : F n k (cid:96) → F n k (cid:96) +1 , where for all G ∈ F n k (cid:96) , F ( G ) =

Original: f ( L [ G ]) where L : F n k (cid:96) → F n k (cid:96) is a linear equivariant function, and f : F (cid:96) → F (cid:96) +1 is a learnable function applied on each of the n k features and F (cid:96) is the ﬁeld for the features at the (cid:96) -th layer.
Revised: f ( L [ G ]) where L : F n k (cid:96) → F n k (cid:96) is a linear equivariant function, and f : F (cid:96) → F (cid:96) +1 is a learnable function applied on each of the n k features and F (cid:96) is the ﬁeld for the features at the (cid:96) -th layer.

Original: We then deﬁne the sets of linear GNNs as follows:
Revised: We then deﬁne the sets of linear GNNs as follows:

Original: [Equation]
Revised: [Equation]

Original: where I k : F n 2 0 → F n k 1 is deﬁned in §2.2 and for t ≥ 1 , F t : F n k t → F n k t +1 are linear equivariant layers.
Revised: where I k : F n 2 0 → F n k 1 is deﬁned in §2.2 and for t ≥ 1 , F t : F n k t → F n k t +1 are linear equivariant layers.

Original: 3. F OLKLORE GNN
Revised: 3. F OLKLORE GNN

Original: The main building block of Folklore GNN (FGNN) is what we call the folklore graph layer (FGL) of order k deﬁned as follows: for k ≥ 1 , F : F n k (cid:96) → F n k (cid:96) +1 where for all G ∈ F n k (cid:96) and all i ∈ [ n ] k ,
Revised: The main building block of Folklore GNN (FGNN) is what we call the folklore graph layer (FGL) of order k deﬁned as follows: for k ≥ 1 , F : F n k (cid:96) → F n k (cid:96) +1 where for all G ∈ F n k (cid:96) and all i ∈ [ n ] k ,

Original: [Equation]
Revised: [Equation]

Original: [Equation]
Revised: [Equation]

Original: where f 0 : F (cid:96) × F (cid:96) +1 / 2 → F (cid:96) +1 and f k : F (cid:96) → F (cid:96) +1 / 2 are learnable functions.
Revised: where f 0 : F (cid:96) × F (cid:96) +1 / 2 → F (cid:96) +1 and f k : F (cid:96) → F (cid:96) +1 / 2 are learnable functions.

Original: As shown in Lem.in Appendix, FGL is an equivariant function which is indeed very expressive.
Revised: As shown in Lem.in Appendix, FGL is an equivariant function which is indeed very expressive.

Original: For classical graphs G ∈ F n 2 0 , we can now deﬁne 2 -FGNN by composing folklore graph layers
Revised: For classical graphs G ∈ F n 2 0 , we can now deﬁne 2 -FGNN by composing folklore graph layers

Original: F t : F n 2 t → F n 2 t +1 , so that F T ◦ . . .
Revised: F t : F n 2 t → F n 2 t +1 , so that F T ◦ . . .

Original: F 1 ◦ F 0 is an equivariant GNN producing a graph in F n 2 T +1 .
Revised: F 1 ◦ F 0 is an equivariant GNN producing a graph in F n 2 T +1 .

Original: To obtain an invariant feature of the graph, we use the summation layer S 2 deﬁned in Section 2.2 so that
Revised: To obtain an invariant feature of the graph, we use the summation layer S 2 deﬁned in Section 2.2 so that

Original: S 2 ◦ F T ◦ . . .
Revised: S 2 ◦ F T ◦ . . .

Original: F 1 ◦ F 0 is now an invariant 2-FGNN.
Revised: F 1 ◦ F 0 is now an invariant 2-FGNN.

Original: In order to deﬁne general k -FGNN, we ﬁrst need to lift the classical graph to a tensor in F n k , then we apply folklore graph layers of order k and ﬁnally we need to project the tensor in F n k to a tensor in F n for the equivariant version and to a tensor in F for the invariant version.
Revised: In order to deﬁne general k -FGNN, we ﬁrst need to lift the classical graph to a tensor in F n k , then we apply folklore graph layers of order k and ﬁnally we need to project the tensor in F n k to a tensor in F n for the equivariant version and to a tensor in F for the invariant version.

Original: The ﬁrst step is done with the linear equivariant function I k : F n 2 

Revised: The ﬁrst step is done with the linear equivariant function I k : F n 2

Original: → F n kdeﬁned in Section 2.2.
Revised: → F n kdeﬁned in Section 2.2.

Original: The last step is done with the reduction layer S k 1 for the equivariant case and the summation layer S k for the invariant case, both deﬁned in Section 2.2.
Revised: The last step is done with the reduction layer S k 1 for the equivariant case and the summation layer S k for the invariant case, both deﬁned in Section 2.2.

Original: We deﬁne the sets of folklore GNNs as follows:
Revised: We deﬁne the sets of folklore GNNs as follows:

Original: [Equation]
Revised: [Equation]

Original: where F t : F n k t → F n k t +1 are FGLs.
Revised: where F t : F n k t → F n k t +1 are FGLs.

Original:  T HEORETICAL RESULTS FOR GNN S 4. W EISFEILER -L EHMAN INVARIANT AND EQUIVARIANT VERSIONS
Revised:  T HEORETICAL RESULTS FOR GNN S 4. W EISFEILER -L EHMAN INVARIANT AND EQUIVARIANT VERSIONS

Original: We introduce a family of functions on graphs parametrized by integers k ≥ 2 developed for the graph isomorphism problem and working with tuples of k vertices.
Revised: We introduce a family of functions on graphs parametrized by integers k ≥ 2 developed for the graph isomorphism problem and working with tuples of k vertices.

Original: Each k -tuple i ∈
Revised: Each k -tuple i ∈

Original: V k
Revised: V k

Original: = [ n ] k is given a color c 

Revised: = [ n ] k is given a color c

Original: ( i ) corresponding to its isomorphism type (see Section B.2).
Revised: ( i ) corresponding to its isomorphism type (see Section B.2).

Original: The k -WL test relies on the following notion of neighborhood, deﬁned by, for any w ∈ [ k ] , and i =
Revised: The k -WL test relies on the following notion of neighborhood, deﬁned by, for any w ∈ [ k ] , and i =

Original: ( i 1 , . .
Revised: ( i 1 , . .

Original: , i k ) ∈
Revised: , i k ) ∈

Original: V k , N w ( i ) =
Revised: V k , N w ( i ) =

Original: { ( i 1 , . .
Revised: { ( i 1 , . .

Original: , i
Revised: , i

Original: w − 1 , j, i w +1 , . . .
Revised: w − 1 , j, i w +1 , . . .

Original: , i k ) : j ∈ V } .
Revised: , i k ) : j ∈ V } .

Original: Then, the colors of the k -tuples are reﬁned as follows,
Revised: Then, the colors of the k -tuples are reﬁned as follows,

Original: c t +1 ( i ) =
Revised: c t +1 ( i ) =

Original: Lex ( c t ( i ) , ( C t 1 ( i ) , . .
Revised: Lex ( c t ( i ) , ( C t 1 ( i ) , . .

Original: , C tk ( i )))
Revised: , C tk ( i )))

Original: where, for w ∈ [ k ] , C tw ( i ) = (cid:110)(cid:110) c t (˜ i ) :
Revised: where, for w ∈ [ k ] , C tw ( i ) = (cid:110)(cid:110) c t (˜ i ) :

Original: ˜ i ∈ N w ( i ) (cid:111)(cid:111) and the function Lex means that all occuring colors are lexicographically ordered and replaced by an initial segment of the natural numbers.
Revised: ˜ i ∈ N w ( i ) (cid:111)(cid:111) and the function Lex means that all occuring colors are lexicographically ordered and replaced by an initial segment of the natural numbers.

Original: For a graph G , let k -WL TI ( G ) denote the multiset of colors of the k -WL algorithm at the T th iteration.
Revised: For a graph G , let k -WL TI ( G ) denote the multiset of colors of the k -WL algorithm at the T th iteration.

Original: After a ﬁnite number of steps (which depends on the number of vertices in the graph), the algorithm stops because a stable coloring is reached (no color class of k -tuples is further divided).
Revised: After a ﬁnite number of steps (which depends on the number of vertices in the graph), the algorithm stops because a stable coloring is reached (no color class of k -tuples is further divided).

Original: We denote by k
Revised: We denote by k

Original: -WL I ( G ) the multiset of colors in the stable coloring.
Revised: -WL I ( G ) the multiset of colors in the stable coloring.

Original: This is a graph invariant that is usually used to test if graphs are isomorphic.
Revised: This is a graph invariant that is usually used to test if graphs are isomorphic.

Original: The power of this invariant increases with k Cai et al.
Revised: The power of this invariant increases with k Cai et al.

Original: We now deﬁne an equivariant version of k -WL test to express the discriminatory power of equivariant architectures For this, we construct a coloring of the vertices from the coloring of the k -tuples given by the standard k -WL algorithm.
Revised: We now deﬁne an equivariant version of k -WL test to express the discriminatory power of equivariant architectures For this, we construct a coloring of the vertices from the coloring of the k -tuples given by the standard k -WL algorithm.

Original: Formally, deﬁne k -WL TE : F n 2 0 → F n by, for i ∈ V : k -WL TE ( G ) i = (cid:8)(cid:8) c T ( i ) :
Revised: Formally, deﬁne k -WL TE : F n 2 0 → F n by, for i ∈ V : k -WL TE ( G ) i = (cid:8)(cid:8) c T ( i ) :

Original: i ∈ V k , i 1 =
Revised: i ∈ V k , i 1 =

Original: i (cid:9)(cid:9) .
Revised: i (cid:9)(cid:9) .

Original: Similarly, deﬁne k -WL E ( G ) =
Revised: Similarly, deﬁne k -WL E ( G ) =

Original: (cid:8)(cid:8) c ( i ) : i ∈ V k , i 1 =
Revised: (cid:8)(cid:8) c ( i ) : i ∈ V k , i 1 =

Original: i (cid:9)(cid:9) where c ( i ) is the stable coloring obtained by the algorithm.
Revised: i (cid:9)(cid:9) where c ( i ) is the stable coloring obtained by the algorithm.

Original: 4. S EPARATING POWER OF GNN S
Revised: 4. S EPARATING POWER OF GNN S

Original: We formulate our results using the equivalence relation introduced by Timofte (2005), which characterizes the separating power of a set of functions.
Revised: We formulate our results using the equivalence relation introduced by Timofte (2005), which characterizes the separating power of a set of functions.

Original: Deﬁnition 2.
Revised: Deﬁnition 2.

Original: Let F be a set of functions f deﬁned on a set X , where each f takes its values in some
Revised: Let F be a set of functions f deﬁned on a set X , where each f takes its values in some

Original: Y f .
Revised: Y f .

Original: The equivalence relation ρ ( F ) deﬁned by F on X is: for any x, x (cid:48) ∈ X ,
Revised: The equivalence relation ρ ( F ) deﬁned by F on X is: for any x, x (cid:48) ∈ X ,

Original: [Equation]
Revised: [Equation]

Original: Given two sets of functions F and E , we say that F is more separating (resp.
Revised: Given two sets of functions F and E , we say that F is more separating (resp.

Original: strictly more separating) than E if ρ ( F ) ⊆
Revised: strictly more separating) than E if ρ ( F ) ⊆

Original: ρ ( E ) (resp.
Revised: ρ ( E ) (resp.

Original: ρ ( F ) (cid:40) ρ ( E ) ).
Revised: ρ ( F ) (cid:40) ρ ( E ) ).

Original: Note that all the functions in F and E need to be deﬁned on the same set but can take values in different sets.
Revised: Note that all the functions in F and E need to be deﬁned on the same set but can take values in different sets.

Original: For example, we can easily see that for the k -WL algorithm deﬁned above, the equivariant version is more separating than the invariant one.
Revised: For example, we can easily see that for the k -WL algorithm deﬁned above, the equivariant version is more separating than the invariant one.

Original: Some properties of the WL hierarchy of tests can be rephrased with the notion of separating power.
Revised: Some properties of the WL hierarchy of tests can be rephrased with the notion of separating power.

Original: In particular, Cai et al.
Revised: In particular, Cai et al.

Original: (1989) showed that ( k + 1)
Revised: (1989) showed that ( k + 1)

Original: -WL I distinguishes strictly more than k -WL
Revised: -WL I distinguishes strictly more than k -WL

Original: I , which can be rewritten simply as (for a function f , we write ρ ( f ) for ρ ( { f } ) )
Revised: I , which can be rewritten simply as (for a function f , we write ρ ( f ) for ρ ( { f } ) )

Original: [Equation]
Revised: [Equation]

Original: This notion of separating power enables us to concisely summarize the current knowledge about the discriminatory power of classes of GNN.
Revised: This notion of separating power enables us to concisely summarize the current knowledge about the discriminatory power of classes of GNN.

Original: Proposition 3.
Revised: Proposition 3.

Original: We have, for k ≥ 2 ,
Revised: We have, for k ≥ 2 ,

Original: [Equation]
Revised: [Equation]

Original: Only results about the invariant cases were previously known: (4) comes from Xu et al.
Revised: Only results about the invariant cases were previously known: (4) comes from Xu et al.

Original: (2018), (5) from Maron et al.
Revised: (2018), (5) from Maron et al.

Original: Geerts (2020a) and one inclusion of (6) comes from Maron et al.
Revised: Geerts (2020a) and one inclusion of (6) comes from Maron et al.

Original: (2019a).
Revised: (2019a).

Original: The equality in (6) for general k ≥ 2 is proved in Section C.
Revised: The equality in (6) for general k ≥ 2 is proved in Section C.

Original: Note that for k = 2 , all GNNs are dealing with tensors of order 2 i.e. with the adjacency matrix of the graph.
Revised: Note that for k = 2 , all GNNs are dealing with tensors of order 2 i.e. with the adjacency matrix of the graph.

Original: However, the complexities of the various layers are quite different: for the message passing
Revised: However, the complexities of the various layers are quite different: for the message passing

Original: GNN, all computations are local (scaling with the maximum degree in the graph) and can be done in parallel; for the linear layer, there are only 15 linear functions from R n 2 → R n 2 for all values of n (Maron et al., 2018); the folklore layer involves a (dense) matrix multiplication of shape n × n .
Revised: GNN, all computations are local (scaling with the maximum degree in the graph) and can be done in parallel; for the linear layer, there are only 15 linear functions from R n 2 → R n 2 for all values of n (Maron et al., 2018); the folklore layer involves a (dense) matrix multiplication of shape n × n .

Original: If 2 -FGNN is the most complex architecture, we see that it has the best separating power among all architectures proposed so far dealing with tensors of order 2 .
Revised: If 2 -FGNN is the most complex architecture, we see that it has the best separating power among all architectures proposed so far dealing with tensors of order 2 .

Original: 4. A PPROXIMATION RESULTS FOR GNN S
Revised: 4. A PPROXIMATION RESULTS FOR GNN S

Original: For X, Y ﬁnite-dimensional spaces, let us denote by C I ( X, Y ) , C E ( X, Y ) , , the set of invariant, respectively equivariant, continuous functions from X to Y .
Revised: For X, Y ﬁnite-dimensional spaces, let us denote by C I ( X, Y ) , C E ( X, Y ) , , the set of invariant, respectively equivariant, continuous functions from X to Y .

Original: The closure of a class of function F for the uniform norm is denoted by F .
Revised: The closure of a class of function F for the uniform norm is denoted by F .

Original: Our result extend easily to graphs of varying sizes but this is deferred to Section F.2 for clarity.
Revised: Our result extend easily to graphs of varying sizes but this is deferred to Section F.2 for clarity.

Original: The theorem below states in particular that the class k -FGNN can approximate any continuous function that is less separating than ( k + 1) -WL in the invariant and in the equivariant cases.
Revised: The theorem below states in particular that the class k -FGNN can approximate any continuous function that is less separating than ( k + 1) -WL in the invariant and in the equivariant cases.

Original: Theorem 4.
Revised: Theorem 4.

Original: Let K discr ⊆
Revised: Let K discr ⊆

Original: G n × F n 0 , K ⊆ F n 2 0 be compact sets.
Revised: G n × F n 0 , K ⊆ F n 2 0 be compact sets.

Original: For the invariant case, we have:
Revised: For the invariant case, we have:

Original: [Equation]
Revised: [Equation]

Original: For the equivariant case, we have:
Revised: For the equivariant case, we have:

Original: [Equation]
Revised: [Equation]

Original: In the invariant case for k = 2 ,we have MGNN I
Revised: In the invariant case for k = 2 ,we have MGNN I

Original: =
Revised: =

Original: -LGNN
Revised: -LGNN

Original: I (cid:40) 

Revised: I (cid:40)

Original: -FGNN I where the strictness of the last inclusion comes from (3).
Revised: -FGNN I where the strictness of the last inclusion comes from (3).

Original: In other words, 2 -FGNN I has a better power of approximation than the other architectures working with tensors of order 2.
Revised: In other words, 2 -FGNN I has a better power of approximation than the other architectures working with tensors of order 2.

Original: We already knew by Proposition 

Revised: We already knew by Proposition

Original: that 

Revised: that

Original: -FGNN
Revised: -FGNN

Original: I is the best separating architecture among those studied in this paper, dealing with tensor of order 2 and our theorem implies that this is also the case for the approximation power.
Revised: I is the best separating architecture among those studied in this paper, dealing with tensor of order 2 and our theorem implies that this is also the case for the approximation power.

Original: To clarify the meaning of these statements, we explain why the inclusions “ ⊆ ” are actually straightforward.
Revised: To clarify the meaning of these statements, we explain why the inclusions “ ⊆ ” are actually straightforward.

Original: For concreteness, we focus on k
Revised: For concreteness, we focus on k

Original: -FGNN
Revised: -FGNN

Original: I ⊆ { f ∈ C
Revised: I ⊆ { f ∈ C

Original: I ( K, F ) : ρ (( k + 1) -WL I ) ⊆
Revised: I ( K, F ) : ρ (( k + 1) -WL I ) ⊆

Original: ρ ( f ) } .
Revised: ρ ( f ) } .

Original: Take h ∈ k
Revised: Take h ∈ k

Original: -FGNN I , this means that there is a sequence GNN j ∈ k
Revised: -FGNN I , this means that there is a sequence GNN j ∈ k

Original: -FGNN I such that, sup G ∈ K (cid:107)
Revised: -FGNN I such that, sup G ∈ K (cid:107)

Original: h ( G )
Revised: h ( G )

Original: − GNN j ( G ) (cid:107) goes to zero when j goes to inﬁnity.
Revised: − GNN j ( G ) (cid:107) goes to zero when j goes to inﬁnity.

Original: Therefore, h is continuous and constant on each ρ ( k -FGNN I ) -class.
Revised: Therefore, h is continuous and constant on each ρ ( k -FGNN I ) -class.

Original: Indeed, for any ( G, G (cid:48) ) ∈
Revised: Indeed, for any ( G, G (cid:48) ) ∈

Original: ρ ( k -FGNN I ) , GNN j ( G ) =
Revised: ρ ( k -FGNN I ) , GNN j ( G ) =

Original: GNN j ( G (cid:48) ) so that h ( G ) =
Revised: GNN j ( G (cid:48) ) so that h ( G ) =

Original: lim
Revised: lim

Original: i GNN j ( G ) =
Revised: i GNN j ( G ) =

Original: lim j GNN j ( G (cid:48) )
Revised: lim j GNN j ( G (cid:48) )

Original: =
Revised: =

Original: h ( G (cid:48) ) .
Revised: h ( G (cid:48) ) .

Original: Hence we have ρ ( k -FGNN I ) ⊆
Revised: Hence we have ρ ( k -FGNN I ) ⊆

Original: ρ ( h ) and by Prop. 3, ρ ( k -FGNN I ) =
Revised: ρ ( h ) and by Prop. 3, ρ ( k -FGNN I ) =

Original: ρ (( k + 1) -WL I ) , allowing us to get the inclusion above.
Revised: ρ (( k + 1) -WL I ) , allowing us to get the inclusion above.

Original: On the contrary, the reverse inclusions “ ⊃ ” are much more intricate but they are also the most valuable.
Revised: On the contrary, the reverse inclusions “ ⊃ ” are much more intricate but they are also the most valuable.

Original: For instance, consider the inclusion
Revised: For instance, consider the inclusion

Original: k
Revised: k

Original: -FGNN
Revised: -FGNN

Original: I ⊃ { f ∈ C
Revised: I ⊃ { f ∈ C

Original: I ( K, F ) : ρ (( k + 1) -WL I ) ⊆
Revised: I ( K, F ) : ρ (( k + 1) -WL I ) ⊆

Original: ρ ( f ) } .
Revised: ρ ( f ) } .

Original: If one wishes to learn a function h ∈ C
Revised: If one wishes to learn a function h ∈ C

Original: I ( K, F ) with k
Revised: I ( K, F ) with k

Original: -FGNN I , this function must at least be approximable by the class of k
Revised: -FGNN I , this function must at least be approximable by the class of k

Original: -FGNN I .
Revised: -FGNN I .

Original: Our theorem precisely guarantees that if h is less separating that k -WL I , it can be approximated by k
Revised: Our theorem precisely guarantees that if h is less separating that k -WL I , it can be approximated by k

Original: -FGNN
Revised: -FGNN

Original: I :
Revised: I :

Original: [Equation]
Revised: [Equation]

Original: For this, we show a much more general version of the famous Stone-Weierstrass theorem (see Section
Revised: For this, we show a much more general version of the famous Stone-Weierstrass theorem (see Section

Original: D) which relates the separating power with the approximation power.
Revised: D) which relates the separating power with the approximation power.

Original: Following the elegant idea of Maehara & Hoang (2019), we augment the input space to transform vector-valued equivariant functions into scalar invariant maps.
Revised: Following the elegant idea of Maehara & Hoang (2019), we augment the input space to transform vector-valued equivariant functions into scalar invariant maps.

Original: Then, we apply a ﬁne-grained approximation theorem from
Revised: Then, we apply a ﬁne-grained approximation theorem from

Original: Timofte (2005).We also provide specialized versions of our abstract theorem in Section 5, which can be easily used to determine the approximation capabilities of any deep learning architecture.
Revised: Timofte (2005).We also provide specialized versions of our abstract theorem in Section 5, which can be easily used to determine the approximation capabilities of any deep learning architecture.

Original: Our theorem has also implications for universality results like Maron et al.
Revised: Our theorem has also implications for universality results like Maron et al.

Original: (2019b)
Revised: (2019b)

Original: ; Keriven & Peyré (2019).
Revised: ; Keriven & Peyré (2019).

Original: A class of GNN is said to be universal if its closure on a compact set K is the whole C
Revised: A class of GNN is said to be universal if its closure on a compact set K is the whole C

Original: I ( K, F ) (or C E ( K, F n ) ).
Revised: I ( K, F ) (or C E ( K, F n ) ).

Original: In particular, Thm. 4 implies that n -LGNN and n -FGNN are universal as n -WL distinguishes non-isomorphic graphs of size n .
Revised: In particular, Thm. 4 implies that n -LGNN and n -FGNN are universal as n -WL distinguishes non-isomorphic graphs of size n .

Original: This recovers a result of Ravanbakhsh (2020) for
Revised: This recovers a result of Ravanbakhsh (2020) for

Original: LGNN.
Revised: LGNN.

Original: Moreover, we can leverage the extensive literature on the WL tests to give more subtle results.
Revised: Moreover, we can leverage the extensive literature on the WL tests to give more subtle results.

Original: For instance, Cai et al.
Revised: For instance, Cai et al.

Original: (1989, §8.2) show that, for planar graphs, O ( √ n ) -WL can distinguish non-isomoprhic instances.
Revised: (1989, §8.2) show that, for planar graphs, O ( √ n ) -WL can distinguish non-isomoprhic instances.

Original: Therefore, O ( √ n ) -LGNN or O ( √ n ) -FGNN achieve universality in the particular, yet common, case of planar graphs.
Revised: Therefore, O ( √ n ) -LGNN or O ( √ n ) -FGNN achieve universality in the particular, yet common, case of planar graphs.

Original: On a more practical side, Fürer (2010, Thm. 4.5) shows that the spectrum of a graph is less separating than 3 -WL so that functions of the spectrum can actually be well approximated by 2 -FGNN.
Revised: On a more practical side, Fürer (2010, Thm. 4.5) shows that the spectrum of a graph is less separating than 3 -WL so that functions of the spectrum can actually be well approximated by 2 -FGNN.

Original:  E XPRESSIVENESS OF GNN S
Revised:  E XPRESSIVENESS OF GNN S

Original: We now state the general theorems which are our main tools in proving our approximation guarantees for GNNs.
Revised: We now state the general theorems which are our main tools in proving our approximation guarantees for GNNs.

Original: Theirs proofs are deferred to Section D.9 which contains our generalization of the StoneWeierstrass theorem with symmetries.
Revised: Theirs proofs are deferred to Section D.9 which contains our generalization of the StoneWeierstrass theorem with symmetries.

Original: We need to ﬁrst introduce more general deﬁnitions: If G is a ﬁnite group acting on some topological space X , we say that G acts continuously on X if, for all g ∈ G , x (cid:55)→ g · x is continuous.
Revised: We need to ﬁrst introduce more general deﬁnitions: If G is a ﬁnite group acting on some topological space X , we say that G acts continuously on X if, for all g ∈ G , x (cid:55)→ g · x is continuous.

Original: If G is a ﬁnite group acting on some compact set X and some topological space Y , we deﬁne the sets of equivariant and invariant continuous functions by,
Revised: If G is a ﬁnite group acting on some compact set X and some topological space Y , we deﬁne the sets of equivariant and invariant continuous functions by,

Original: [Equation]
Revised: [Equation]

Original: Note that these deﬁnitions extend Deﬁnition 1 to a general group.
Revised: Note that these deﬁnitions extend Deﬁnition 1 to a general group.

Original: Theorem 5.
Revised: Theorem 5.

Original: Let X be a compact space, Y = R p be some ﬁnite-dimensional vector space, G be a ﬁnite group acting (continuously) on X .
Revised: Let X be a compact space, F = R p be some ﬁnite-dimensional vector space, G be a ﬁnite group acting (continuously) on X .

Original: Let F 0 ⊆
Revised: Let F 0 ⊆

Original: (cid:83) ∞
Revised: (cid:83) ∞

Original: h =1 C I ( X, R h ) be a non-empty set of invariant functions, stable by concatenation, and consider,
Revised: h =1 C I ( X, R h ) be a non-empty set of invariant functions, stable by concatenation, and consider,

Original: [Equation]
Revised: [Equation]

Original: Then the closure of F is,
Revised: Then the closure of F is,

Original: [Equation]
Revised: [Equation]

Original: We can apply Theorem 6 to the class of k -ary relational pooling GNN introduced in Murphy et al.
Revised: We can apply Theorem 5 to the class of k -ary relational pooling GNN introduced in Murphy et al.

Original: (2019a).
Revised: (2019a).

Original: As a result, we get that this class of invariant k -RP GNN can approximate any continuous function f with ρ ( k − RPGNN ) ⊆
Revised: As a result, we get that this class of invariant k -RP GNN can approximate any continuous function f with ρ ( k − RPGNN ) ⊆

Original: ρ ( f ) but to the best of our knowledge, ρ ( k − RPGNN ) is not known and only ρ ( k − RPGNN ) ⊂
Revised: ρ ( f ) but to the best of our knowledge, ρ ( k − RPGNN ) is not known and only ρ ( k − RPGNN ) ⊂

Original: ρ (2 − WL I ) is proved in Murphy et al. (2019a).
Revised: ρ (2 − WL I ) is proved in Murphy et al. (2019a).

Original: We now state our general theorem for the equivariant case:
Revised: We now state our general theorem for the equivariant case:

Original: Theorem 6.
Revised: Theorem 6.

Original: Let X be a compact space,Y = F n , with F = R p and G = S n the permutation group, acting (continuously) on X and acting on F n by, for σ ∈ S n , x ∈ F n , 
Revised: Let X be a compact space, F = R p and G = S n the permutation group, acting (continuously) on X and acting on F n by, for σ ∈ S n , x ∈ F n , [Equation] i [Equation]

Original: [Equation]
Revised: [Equation]

Revised: σ

Revised: [Equation]

Revised: [Equation]

Original: Let F 0 ⊆
Revised: Let F 0 ⊆

Original: (cid:83) ∞
Revised: (cid:83) ∞

Original: h =1 C E ( X,  R h )  be a non-empty set of equivariant functions, stable by concatenation, and consider,
Revised: h =1 C E (cid:0) X, ( R h ) n (cid:1) be a non-empty set of equivariant functions, stable by concatenation, and consider,

Original: [Equation]
Revised: [Equation]

Original: Assume, that, if f ∈ F 0 , then,
Revised: Assume, that, if f ∈ F 0 , then,

Original: [Equation]
Revised: [Equation]

Original: Then the closure of F is,
Revised: Then the closure of F is,

Original: [Equation]
Revised: [Equation]

Original: Applications of these theorems fo the case of Pointnet Qi et al.
Revised: Applications of these theorems fo the case of Pointnet Qi et al.

Original: (2017) are provided in Section D.
Revised: (2017) are provided in Section D.

Original:  Q UADRATIC A SSIGNMENT P ROBLEM
Revised:  Q UADRATIC A SSIGNMENT P ROBLEM

Original: To empirically evaluate our results, we study the Quadratic Assignment Problem (QAP), a classical problem in combinatorial optimization.
Revised: To empirically evaluate our results, we study the Quadratic Assignment Problem (QAP), a classical problem in combinatorial optimization.

Original: For A, B n × n symmetric matrices, it consists in solving maximize trace ( AXBX (cid:62) ) , subject to X ∈ Π , where Π is the set of n × n permutation matrices.
Revised: For A, B n × n symmetric matrices, it consists in solving maximize trace ( AXBX (cid:62) [Equation] , subject to X ∈ Π [Equation]

Revised: where Π is the set of n × n permutation matrices.

Original: Many optimization problems can be formulated as
Revised: Many optimization problems can be formulated as

Original: QAP.
Revised: QAP.

Original: An example is the network alignment problem, which consists in ﬁnding the best matching between two graphs, represented by their adjacency matrices A and B .
Revised: An example is the network alignment problem, which consists in ﬁnding the best matching between two graphs, represented by their adjacency matrices A and B .

Original: Though QAP is known to be NP-hard, recent works such as Nowak et al.
Revised: Though QAP is known to be NP-hard, recent works such as Nowak et al.

Original: (2018) have investigated whether it can be solved efﬁciently w.r.t.
Revised: (2018) have investigated whether it can be solved efﬁciently w.r.t.

Original: a ﬁxed input distribution.
Revised: a ﬁxed input distribution.

Original: More precisely, Nowak et al.
Revised: More precisely, Nowak et al.

Original: (2018) studied whether one can learn to solve this problem using a MGNN trained on a dataset of already solved instances.
Revised: (2018) studied whether one can learn to solve this problem using a MGNN trained on a dataset of already solved instances.

Original: However, as shown below, both the baselines and their approach fail on regular graphs, a class of graph considered as particularly hard for isomorphism testing.
Revised: However, as shown below, both the baselines and their approach fail on regular graphs, a class of graph considered as particularly hard for isomorphism testing.

Original: To remedy this weakness, we consider 2 -FGNN E .
Revised: To remedy this weakness, we consider 2 -FGNN E .

Original: We then follow the siamese method of (Nowak et al., 2018): given two graphs, our system produces an embedding in F n for each graph, where n is the number of nodes, which are then multiplied together to obtain a n × n similarity matrix on nodes.
Revised: We then follow the siamese method of (Nowak et al., 2018): given two graphs, our system produces an embedding in F n for each graph, where n is the number of nodes, which are then multiplied together to obtain a n × n similarity matrix on nodes.

Original: A permutation is ﬁnally computed by solving a Linear Assignment Problem (LAP) with this resulting n × n as cost matrix.
Revised: A permutation is ﬁnally computed by solving a Linear Assignment Problem (LAP) with this resulting n × n as cost matrix.

Original: We tested our architecture on two distribution: the Erd˝os–Rényi model and random regular graphs.
Revised: We tested our architecture on two distribution: the Erd˝os–Rényi model and random regular graphs.

Original: The accuracy in matching the graphs is much improved compare to previous works.
Revised: The accuracy in matching the graphs is much improved compare to previous works.

Original: The experimental setup is described more precisely in Section A.1.
Revised: The experimental setup is described more precisely in Section A.1.

Original: [Figure]
Revised: [Figure]

Original: Noise level
Revised: Noise level

Original: [Figure]
Revised: [Figure]

Original: Erd˝os
Revised: Erd˝os

Original: [Figure]
Revised: [Figure]

Original: Rényi graph model 0 .0 .0 .0 .0 .
Revised: Rényi graph model 0 .0 .0 .0 .0 .

Original: Noise level
Revised: Noise level

Original: [Figure]
Revised: [Figure]

Original: ac
Revised: ac

Original: [Figure]
Revised: [Figure]

Original: Regular graph model
Revised: Regular graph model

Original: This work
Revised: This work

Original: SDP (Peng et al., 2010)
Revised: SDP (Peng et al., 2010)

Original: LowRankAlign (Feizi et al., 2016)
Revised: LowRankAlign (Feizi et al., 2016)

Original: GNN (Nowak et al., 2018)
Revised: GNN (Nowak et al., 2018)

Original:  C ONCLUSION
Revised:  C ONCLUSION

Original: We derived the expressive power of various practical GNN architectures: message passing GNN, linear GNN and folklore GNN; both for their invariant and equivariant counterparts.
Revised: We derived the expressive power of various practical GNN architectures: message passing GNN, linear GNN and folklore GNN; both for their invariant and equivariant counterparts.

Original: Our results unify and extend the recent works in this direction.
Revised: Our results unify and extend the recent works in this direction.

Original: In particular, we are able to recover all the universality results proved for GNNs so far.
Revised: In particular, we are able to recover all the universality results proved for GNNs so far.

Original: Similarly to existing results in the literature, we do not deal here with the sizes of the embeddings constructed at different layers, i.e. the sizes of the spaces F (cid:96) , and these sizes are supposed to grow to inﬁnity with the number of nodes n in the graph.
Revised: Similarly to existing results in the literature, we do not deal here with the sizes of the embeddings constructed at different layers, i.e. the sizes of the spaces F (cid:96) , and these sizes are supposed to grow to inﬁnity with the number of nodes n in the graph.

Original: Obtaining bounds on the scaling of the sizes of the features to ensure that the results presented here are still valid is an interesting open question.
Revised: Obtaining bounds on the scaling of the sizes of the features to ensure that the results presented here are still valid is an interesting open question.

Original: We show that folklore GNNs have the best power of approximation among all GNNs studied here dealing with tensors of order 2 .
Revised: We show that folklore GNNs have the best power of approximation among all GNNs studied here dealing with tensors of order 2 .

Original: From a practical perspective, we demonstrate their improved performance on the QAP with a signiﬁcant gap in performances compared to other approaches.
Revised: From a practical perspective, we demonstrate their improved performance on the QAP with a signiﬁcant gap in performances compared to other approaches.

Original: A CKNOWLEDGMENTS
Revised: A CKNOWLEDGMENTS

Original: This work was supported in part by the French government under management of Agence Nationale de la Recherche as part of the “Investissements d’avenir” program, reference ANR19-P3IA- (PRAIRIE 3IA Institute).
Revised: This work was supported in part by the French government under management of Agence Nationale de la Recherche as part of the “Investissements d’avenir” program, reference ANR19-P3IA- (PRAIRIE 3IA Institute).

Original: M.L. thanks Google for Google Cloud Platform research credits and
Revised: M.L. thanks Google for Google Cloud Platform research credits and

Original: NVIDIA for a NVIDIA GPU Grant.
Revised: NVIDIA for a NVIDIA GPU Grant.

Original: A Experimental results
Revised: A Experimental results

Original: A.1 Details on the experimental setup . . . . . . . . . . . . . . . . . . . . . . . . . . .
Revised: A.1 Details on the experimental setup . . . . . . . . . . . . . . . . . . . . . . . . . . .

Original: A.2 Experimental results on graphs of varying size . . . . . . . . . . . . . . . . . . . .
Revised: A.2 Experimental results on graphs of varying size . . . . . . . . . . . . . . . . . . . .

Original: A
Revised: A

Original: [Table]
Revised: [Table]

Original: 3 Generalization for
Revised: 3 Generalization for

Original: [Table]
Revised: [Table]

Original: graphs . . . . . . . . . . . . . . . . . . . . . . . . . . .
Revised: graphs . . . . . . . . . . . . . . . . . . . . . . . . . . .

Original: B tests
Revised: B tests

Original: B.1 Weisfeiler-Lehman test on vertices . . . . . . . . . . . . . . . . . . . . . . . . . .
Revised: B.1 Weisfeiler-Lehman test on vertices . . . . . . . . . . . . . . . . . . . . . . . . . .

Original: B.2 Isomorphism type . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Revised: B.2 Isomorphism type . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Original: B.
Revised: B.

Original: [Table]
Revised: [Table]

Original: and tests k
Revised: and tests k

Original: [Table]
Revised: [Table]

Original: C Separating power of GNN
Revised: C Separating power of GNN

Original: C.1 Mutli-linear perceptrons . .
Revised: C.1 Mutli-linear perceptrons . .

Original: C.2 Augmented separating power . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Revised: C.2 Augmented separating power . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Original: C.3 Initialization layer .
Revised: C.3 Initialization layer .

Original: C.4 Known results about the separating power of some GNN classes . . . . . . . . . .
Revised: C.4 Known results about the separating power of some GNN classes . . . . . . . . . .

Original: C.5 Bounding the separating power of k -FGNN . . . . . . . . . . . . . . . . . . . . .
Revised: C.5 Bounding the separating power of k -FGNN . . . . . . . . . . . . . . . . . . . . .

Original: C.6 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Revised: C.6 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Original: D Stone-Weierstrass theorem with symmetries
Revised: D Stone-Weierstrass theorem with symmetries

Original: D.1 General notations . . .
Revised: D.1 General notations . . .

Original: D.

Revised: D.

Original: Separating power . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Revised: Separating power . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Original: D.3 Approximation theorems for real-valued functions . . . . . . . . . . . . . . . . . .
Revised: D.3 Approximation theorems for real-valued functions . . . . . . . . . . . . . . . . . .

Original: D.4 The equivariant approximation theorem . . . . . . . . . . . . . . . . . . . . . . .
Revised: D.4 The equivariant approximation theorem . . . . . . . . . . . . . . . . . . . . . . .

Original: D.

Revised: D.

Original: A preliminary version of the equivariant approximation theorem . . . . . . . . . .
Revised: A preliminary version of the equivariant approximation theorem . . . . . . . . . .

Original: D.6 Characterizing the subalgebras of R p . . . . . . . . . . . . . . . . . . . . . . . . .
Revised: D.6 Characterizing the subalgebras of R p . . . . . . . . . . . . . . . . . . . . . . . . .

Original: D.7 Proof of the main equivariant approximation theorem . . . . . . . . . . . . . . . .
Revised: D.7 Proof of the main equivariant approximation theorem . . . . . . . . . . . . . . . .

Original: D.8 Practical reductions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Revised: D.8 Practical reductions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Original: D.9 Reductions for GNNs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Revised: D.9 Reductions for GNNs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Original: E Proofs for expressiveness of GNNs
Revised: E Proofs for expressiveness of GNNs

Original: E.1 Expressivity of GNN layers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Revised: E.1 Expressivity of GNN layers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Original: E.2 Approximation theorems for GNNs . . . . . . . . . . . . . . . . . . . . . . . . .
Revised: E.2 Approximation theorems for GNNs . . . . . . . . . . . . . . . . . . . . . . . . .

Original: F Extension to graphs of varying sizes
Revised: F Extension to graphs of varying sizes

Original: F.
Revised: F.

Original: Extension to disconnected input spaces . . . . . . . . . . . . . . . . . . . . . . . .
Revised: Extension to disconnected input spaces . . . . . . . . . . . . . . . . . . . . . . . .

Original: F.
Revised: F.

Original: Approximation theorem with varying graph size . . . . . . . . . . . . . . . . . . .
Revised: Approximation theorem with varying graph size . . . . . . . . . . . . . . . . . . .

Original: A E XPERIMENTAL RESULTS A. D ETAILS ON THE EXPERIMENTAL SETUP
Revised: A E XPERIMENTAL RESULTS A. D ETAILS ON THE EXPERIMENTAL SETUP

Original: We consider a 2 -FGNN E and train it to solve random planted problem instances of the QAP.
Revised: We consider a 2 -FGNN E and train it to solve random planted problem instances of the QAP.

Original: Given a pair of graphs G 1 , G 2 with n nodes each, we consider the siamese 2 -FGNN E encoder producing embeddings E 1 , E 2 ∈ R n × k .
Revised: Given a pair of graphs G 1 , G 2 with n nodes each, we consider the siamese 2 -FGNN E encoder producing embeddings E 1 , E 2 ∈ R n × k .

Original: Those embeddings are used to predict a matching as follows: we ﬁrst compute the outer product E 1 E T 2 , then we take a softmax along each row and use standard cross-entropy loss to predict the corresponding permutation index.
Revised: Those embeddings are used to predict a matching as follows: we ﬁrst compute the outer product E 1 E T 2 , then we take a softmax along each row and use standard cross-entropy loss to predict the corresponding permutation index.

Original: We used 2 -FGNN E with 2 layers, each MLP having depth 3 and hidden states of size 64.
Revised: We used 2 -FGNN E with 2 layers, each MLP having depth 3 and hidden states of size 64.

Original: We trained for 25 epochs with batches of size 32, a learning rate of 1e-4 and Adam optimizer.
Revised: We trained for 25 epochs with batches of size 32, a learning rate of 1e-4 and Adam optimizer.

Original: The PyTorch code is available in the supplementary material.
Revised: The PyTorch code is available in the supplementary material.

Original: For each experiment, the dataset was made of 20000 graphs for the train set, 1000 for the validation set and 1000 for the test set.
Revised: For each experiment, the dataset was made of 20000 graphs for the train set, 1000 for the validation set and 1000 for the test set.

Original: For the experiment with Erd˝os–Rényi random graphs, we consider G 1 to be a random Erd˝os–Rényi graph with edge density
Revised: For the experiment with Erd˝os–Rényi random graphs, we consider G 1 to be a random Erd˝os–Rényi graph with edge density

Original: p e
Revised: p e

Original: = 0 .
Revised: = 0 .

Original: 2 and n = 50 vertices.
Revised: 2 and n = 50 vertices.

Original: The graph G 2 is a small perturbation of G 1 according to the following error model considered in Feizi et al.
Revised: The graph G 2 is a small perturbation of G 1 according to the following error model considered in Feizi et al.

Original: [Equation]
Revised: [Equation]

Original: where Q and Q (cid:48) are Erd˝os–Rényi random graphs with edge density p 1 and p 2 =
Revised: where Q and Q (cid:48) are Erd˝os–Rényi random graphs with edge density p 1 and p 2 =

Original: p 1 p e / (1 − p e ) respectively, so that G 2 has the same expected degree as G 1 .
Revised: p 1 p e / (1 − p e ) respectively, so that G 2 has the same expected degree as G 1 .

Original: The noise level is the parameter p 1 .
Revised: The noise level is the parameter p 1 .

Original: For regular graphs, we followed the same experimental setup but now G 1 is a random regular graph with degree d = 10 .
Revised: For regular graphs, we followed the same experimental setup but now G 1 is a random regular graph with degree d = 10 .

Original: Regular graphs are interesting example as they tend to be considered harder to align due to their more symmetric structure.
Revised: Regular graphs are interesting example as they tend to be considered harder to align due to their more symmetric structure.

Original: A. E XPERIMENTAL RESULTS ON GRAPHS OF VARYING SIZE
Revised: A. E XPERIMENTAL RESULTS ON GRAPHS OF VARYING SIZE

Original: [Equation]
Revised: [Equation]

Original: [Table]
Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original: level
Revised: level

Original: [Figure]
Revised: [Figure]

Original: Regular graph model
Revised: Regular graph model

Original: This work
Revised: This work

Original: We tested our models on dataset of graphs of varying size, as this setting is also encompassed by our theory.
Revised: We tested our models on dataset of graphs of varying size, as this setting is also encompassed by our theory.

Original: However, contrary to message-passing GNN, GNN based on tensors do not work well with batches of graphs of varying size.
Revised: However, contrary to message-passing GNN, GNN based on tensors do not work well with batches of graphs of varying size.

Original: Previous implementations, such as the one of Maron et al. (2019a), group the graphs in the dataset by size, enabling the GNN to only deal with batches of graphs on the same size.
Revised: Previous implementations, such as the one of Maron et al. (2019a), group the graphs in the dataset by size, enabling the GNN to only deal with batches of graphs on the same size.

Original: Instead, we use masking, which is a standard practice in recurrent neural networks.
Revised: Instead, we use masking, which is a standard practice in recurrent neural networks.

Original: A batch of b tensors of sizes n 1 × n 1 , n 2 × n 2 , . .
Revised: A batch of b tensors of sizes n 1 × n 1 , n 2 × n 2 , . .

Original: n b × n b is represented as a tensor b × n max × n max where n max = max i =1 ,...,b n i .
Revised: n b × n b is represented as a tensor b × n max × n max where n max = max i =1 ,...,b n i .

Original: A mask is created at initialization and is used to ensure that the operations on the full tensor translates to valid operations on each of the individual tensor.
Revised: A mask is created at initialization and is used to ensure that the operations on the full tensor translates to valid operations on each of the individual tensor.

Original: We implemented this functionnality as a class MaskedTensors .
Revised: We implemented this functionnality as a class MaskedTensors .

Original: Thanks to the newest improvements of PyTorch (Paszke et al., 2019), MaskedTensors act as a subclass of fundamental Tensor class.
Revised: Thanks to the newest improvements of PyTorch (Paszke et al., 2019), MaskedTensors act as a subclass of fundamental Tensor class.

Original: Thus they almost seamlessly integrate into standard PyTorch code.
Revised: Thus they almost seamlessly integrate into standard PyTorch code.

Original: We refer the reader to the code for more details: https://github.com/mlelarge/graph_neural_net
Revised: We refer the reader to the code for more details: https://github.com/mlelarge/graph_neural_net

Original: Results of our architecture and implementation with graphs of varying size are shown below on Figure 2.
Revised: Results of our architecture and implementation with graphs of varying size are shown below on Figure 2.

Original: The only difference with the setting described above is that the number of nodes is now random.
Revised: The only difference with the setting described above is that the number of nodes is now random.

Original: The number of vertices of a graph is indeed chosen randomly according to binomial distribution of parameters n = 50 and p n = 0 .
Revised: The number of vertices of a graph is indeed chosen randomly according to binomial distribution of parameters n = 50 and p n = 0 .

Original: A. G ENERALIZATION FOR REGULAR GRAPHS
Revised: A. G ENERALIZATION FOR REGULAR GRAPHS

Original: We made the following experiment with the same general setting as in Section A.1 with regular graphs.
Revised: We made the following experiment with the same general setting as in Section A.1 with regular graphs.

Original: We trained different models for all noise levels between 0 and 0 .
Revised: We trained different models for all noise levels between 0 and 0 .

Original: but in Figure 3, we plot the accuracy of each model across all noise levels.
Revised: but in Figure 3, we plot the accuracy of each model across all noise levels.

Original: We observe that a majority of the models actually generalize to settings with noise level on which they were not trained.
Revised: We observe that a majority of the models actually generalize to settings with noise level on which they were not trained.

Original: Indeed, the model trained with noise level ≈ 0 .
Revised: Indeed, the model trained with noise level ≈ 0 .

Original: 1 is performing best among all models across all noise levels!
Revised: 1 is performing best among all models across all noise levels!

Original: B W EISFEILER -L EHMAN TESTS
Revised: B W EISFEILER -L EHMAN TESTS

Original: Here we describe more precisely this hierarchy of tests, which will be used extensively to characterize the discirminatory power of classes of GNN.
Revised: Here we describe more precisely this hierarchy of tests, which will be used extensively to characterize the discirminatory power of classes of GNN.

Original: See Douglas (2011); Grohe (2017); Fürer (2017) for graph-theoretic introductions to these algorithms.
Revised: See Douglas (2011); Grohe (2017); Fürer (2017) for graph-theoretic introductions to these algorithms.

Original: B. W EISFEILER -L EHMAN TEST ON VERTICES
Revised: B. W EISFEILER -L EHMAN TEST ON VERTICES

Original: We now present the initial vertex coloring algorithm.
Revised: We now present the initial vertex coloring algorithm.

Original: Input.
Revised: Input.

Original: This algorithm takes as input a discrete graph structure G =
Revised: This algorithm takes as input a discrete graph structure G =

Original: ( V, E ) ∈ G n with V =
Revised: ( V, E ) ∈ G n with V =

Original: [ n ] ,
Revised: [ n ] ,

Original: E ⊆ V 2 and h ∈ F n 0 features on the vertices.
Revised: E ⊆ V 2 and h ∈ F n 0 features on the vertices.

Original: Initialization.
Revised: Initialization.

Original: Each vertex s ∈ V is given a color c 0 WL ( G, h ) s
Revised: Each vertex s ∈ V is given a color c 0 WL ( G, h ) s

Original: =
Revised: =

Original: h s corresponding to its features vector.
Revised: h s corresponding to its features vector.

Original: Reﬁning the coloring.
Revised: Reﬁning the coloring.

Original: The colors of the vertices are updated as follows,
Revised: The colors of the vertices are updated as follows,

Original: [Equation]
Revised: [Equation]

Original: and the function Lex means that all occuring colors are lexicographically ordered.
Revised: and the function Lex means that all occuring colors are lexicographically ordered.

Original: For each graph G ∈ G n and each vector of features h ∈ F n 0 , there exists a time T ( G, h ) from which the sequence of colorings ( c t WL ( G, h ))
Revised: For each graph G ∈ G n and each vector of features h ∈ F n 0 , there exists a time T ( G, h ) from which the sequence of colorings ( c t WL ( G, h ))

Original: t ≥ 0 is stationary.
Revised: t ≥ 0 is stationary.

Original: More exactly, the colorings are not reﬁned anymore: for any t ≥ T ( G, h ) , s, s (cid:48) ∈ V ,
Revised: More exactly, the colorings are not reﬁned anymore: for any t ≥ T ( G, h ) , s, s (cid:48) ∈ V ,

Original: [Equation]
Revised: [Equation]

Original: Denote the resulting coloring c T ( G,h ) WL ( G, h ) by simply c WL ( G, h ) .
Revised: Denote the resulting coloring c T ( G,h ) WL ( G, h ) by simply c WL ( G, h ) .

Original: c WL is now a mapping from
Revised: c WL is now a mapping from

Original: G n × F n 0 → Z n for some space of colors Z .
Revised: G n × F n 0 → Z n for some space of colors Z .

Original: Invariant tests The proper Weisfeiler-Lehman test is invariant and is deﬁned by, for t ≥ 0 and ( G, h ) ∈ G n × F n 0 a graph,
Revised: Invariant tests The proper Weisfeiler-Lehman test is invariant and is deﬁned by, for t ≥ 0 and ( G, h ) ∈ G n × F n 0 a graph,

Original: [Equation]
Revised: [Equation]

Original: Equivariant tests For the vertex coloring algorithm,
Revised: Equivariant tests For the vertex coloring algorithm,

Original: c WL is already an equiavriant mapping so we deﬁne, for t ≥ 0 and ( G, h ) ∈ G n × F n 0 a graph,
Revised: c WL is already an equiavriant mapping so we deﬁne, for t ≥ 0 and ( G, h ) ∈ G n × F n 0 a graph,

Original: [Equation]
Revised: [Equation]

Original: B. I SOMORPHISM TYPE
Revised: B. I SOMORPHISM TYPE

Original: The initialization of the higher-order variants of the Weisfeiler-Lehman test is slightly more intricate.
Revised: The initialization of the higher-order variants of the Weisfeiler-Lehman test is slightly more intricate.

Original: For this we need to deﬁne the isomorphism type of a k -tuple w.r.t.
Revised: For this we need to deﬁne the isomorphism type of a k -tuple w.r.t.

Original: a graph described by a tensor F n 2 0 .
Revised: a graph described by a tensor F n 2 0 .

Original: A k -tuple ( i 1 , . . .
Revised: A k -tuple ( i 1 , . . .

Original: , i k ) ∈ [ n ] k in a graph G ∈ F n 2 0 and a k -tuple ( j 1 , . . .
Revised: , i k ) ∈ [ n ] k in a graph G ∈ F n 2 0 and a k -tuple ( j 1 , . . .

Original: , j k ) ∈ [ n ] k in a graph
Revised: , j k ) ∈ [ n ] k in a graph

Original: H ∈ F n 2 0 are said to have the same isomoprhism type if the mapping i w (cid:55)→ j w is a well-deﬁned partial isormophism.
Revised: H ∈ F n 2 0 are said to have the same isomoprhism type if the mapping i w (cid:55)→ j w is a well-deﬁned partial isormophism.

Original: Explicitly, this means that,
Revised: Explicitly, this means that,

Original: • ∀ w, w (cid:48) ∈
Revised: • ∀ w, w (cid:48) ∈

Original: [ k ] , i w = i w (cid:48)
Revised: [ k ] , i w = i w (cid:48)

Original: ⇐⇒ j w = j w (cid:48) .
Revised: ⇐⇒ j w = j w (cid:48) .

Original: • ∀ w, w (cid:48) ∈
Revised: • ∀ w, w (cid:48) ∈

Original: [ k ] , G i w ,i w (cid:48)
Revised: [ k ] , G i w ,i w (cid:48)

Original: =
Revised: =

Original: H j w ,j w (cid:48) .
Revised: H j w ,j w (cid:48) .

Original: Denote by iso( G ) i 1 ,...,i k the isomorphism type of the k -tuple ( i 1 , . . .
Revised: Denote by iso( G ) i 1 ,...,i k the isomorphism type of the k -tuple ( i 1 , . . .

Original: , i k ) ∈ [ n ] k in a graph G ∈ F n 2 0 .
Revised: , i k ) ∈ [ n ] k in a graph G ∈ F n 2 0 .

Original: B. W EISFEILER -L EHMAN AND F OLKLORE W EISFEILER -L EHMAN TESTS OF ORDER k ≥
Revised: B. W EISFEILER -L EHMAN AND F OLKLORE W EISFEILER -L EHMAN TESTS OF ORDER k ≥

Original: We now present the folklore version of the Weisfeiler-Lehman test of order k ( k -FWL), for k ≥ 2 , along with k -WL for clarity.
Revised: We now present the folklore version of the Weisfeiler-Lehman test of order k ( k -FWL), for k ≥ 2 , along with k -WL for clarity.

Original: For both, we follow the presentation of Maron et al. (2019a) (except for the equivariant tests).
Revised: For both, we follow the presentation of Maron et al. (2019a) (except for the equivariant tests).

Original: Input.
Revised: Input.

Original: These algorithms take as input a graph G ∈ F n 2 0 which can be seen as a coloring on the pair of nodes.
Revised: These algorithms take as input a graph G ∈ F n 2 0 which can be seen as a coloring on the pair of nodes.

Original: Initialization.
Revised: Initialization.

Original: Each k -tuple s ∈ V k is given a color c 0 k -WL ( G )
Revised: Each k -tuple s ∈ V k is given a color c 0 k -WL ( G )

Original: s
Revised: s

Original: = c
Revised: = c

Original: 0 k -FWL ( G ) s corresponding to its isomorphism type.
Revised: 0 k -FWL ( G ) s corresponding to its isomorphism type.

Original: k -WL.
Revised: k -WL.

Original: The k -WL test relies on the following notion of neighborhood, deﬁned by, for any w ∈ [ k ] , and s =
Revised: The k -WL test relies on the following notion of neighborhood, deﬁned by, for any w ∈ [ k ] , and s =

Original: ( i 1 , . .
Revised: ( i 1 , . .

Original: , i k ) ∈
Revised: , i k ) ∈

Original: V k ,
Revised: V k ,

Original: [Equation]
Revised: [Equation]

Original: Then, the colors of the k -tuples s ∈ V k are reﬁned as follows,
Revised: Then, the colors of the k -tuples s ∈ V k are reﬁned as follows,

Original: [Equation]
Revised: [Equation]

Original: where, for w ∈ [ k ] ,
Revised: where, for w ∈ [ k ] ,

Original: [Equation]
Revised: [Equation]

Original: For each graph G ∈ F n 0 , there exists a time T ( G ) from which the sequence of colorings ( c tk -WL ( G ))
Revised: For each graph G ∈ F n 0 , there exists a time T ( G ) from which the sequence of colorings ( c tk -WL ( G ))

Original: t ≥ 0 is stationary.
Revised: t ≥ 0 is stationary.

Original: More exactly, the colorings are not reﬁned anymore: for any t ≥ T ( G ) , s, s (cid:48)
Revised: More exactly, the colorings are not reﬁned anymore: for any t ≥ T ( G ) , s, s (cid:48)

Original: ∈ V k , c tk
Revised: ∈ V k , c tk

Original: [Equation]
Revised: [Equation]

Original: WL
Revised: WL

Original: [Equation]
Revised: [Equation]

Original: Denote the resulting coloring c T ( G ) k
Revised: Denote the resulting coloring c T ( G ) k

Original: -WL ( G ) by simply
Revised: -WL ( G ) by simply

Original: c k -WL ( G ) .
Revised: c k -WL ( G ) .

Original: k -FWL.
Revised: k -FWL.

Original: For k -FWL, the corresponding notion of neighborhood is deﬁned by, for any j ∈ V , and
Revised: For k -FWL, the corresponding notion of neighborhood is deﬁned by, for any j ∈ V , and

Original: [Equation]
Revised: [Equation]

Original: =
Revised: =

Original: [Equation]
Revised: [Equation]

Original: i 1 , . .
Revised: i 1 , . .

Original: , i k ) ∈
Revised: , i k ) ∈

Original: V k
Revised: V k

Original: [Equation]
Revised: [Equation]

Original: Then, the colors of the k -tuples s ∈ V k are reﬁned as follows,
Revised: Then, the colors of the k -tuples s ∈ V k are reﬁned as follows,

Original: [Equation]
Revised: [Equation]

Original: where, for j ∈ V ,
Revised: where, for j ∈ V ,

Original: [Equation]
Revised: [Equation]

Original: Like k -WL, for each graph G ∈ F n 0 , there exists a time T ( G ) from which the sequence of colorings ( c tk -WL ( G ))
Revised: Like k -WL, for each graph G ∈ F n 0 , there exists a time T ( G ) from which the sequence of colorings ( c tk -WL ( G ))

Original: t ≥ 0 is stationary.
Revised: t ≥ 0 is stationary.

Original: Similarly, denote the resulting coloring c T ( G )
Revised: Similarly, denote the resulting coloring c T ( G )

Original: k -WL ( G ) by simply
Revised: k -WL ( G ) by simply

Original: c k -WL ( G ) .
Revised: c k -WL ( G ) .

Original: The colors c tk -FWL and c tk -FWL at iteration t deﬁne a mapping from F n 2 0 to space of colorings of k -tuples, Z n k for some space Z .
Revised: The colors c tk -FWL and c tk -FWL at iteration t deﬁne a mapping from F n 2 0 to space of colorings of k -tuples, Z n k for some space Z .

Original: Invariant tests
Revised: Invariant tests

Original: The standard versions of the Weisfeiler-Lehman tests are invariant and can be deﬁned by, for t ≥ 0 and G ∈ F n 2 0 a graph,
Revised: The standard versions of the Weisfeiler-Lehman tests are invariant and can be deﬁned by, for t ≥ 0 and G ∈ F n 2 0 a graph,

Original: [Equation]
Revised: [Equation]

Original: [Table]
Revised: [Table]

Original: Equivariant tests We now introduce the equivariant version of these tests.
Revised: Equivariant tests We now introduce the equivariant version of these tests.

Original: Many extensions are possible, we chose this one for its simplicity.
Revised: Many extensions are possible, we chose this one for its simplicity.

Original: For t ≥ 0 , G ∈ F n 2 0 a graph, i ∈ V ,
Revised: For t ≥ 0 , G ∈ F n 2 0 a graph, i ∈ V ,

Original: [Equation]
Revised: [Equation]

Original: C S EPARATING POWER OF GNN
Revised: C S EPARATING POWER OF GNN

Original: The goal of this section is to prove,
Revised: The goal of this section is to prove,

Original: Proposition 3.
Revised: Proposition 3.

Original: We have, for k ≥ 2 ,
Revised: We have, for k ≥ 2 ,

Original: [Equation]
Revised: [Equation]

Original: C. M UTLI - LINEAR PERCEPTRONS
Revised: C. M UTLI - LINEAR PERCEPTRONS

Original: In the following we will use extensively multi-linear perceptrons (MLP) and their universality properties.
Revised: In the following we will use extensively multi-linear perceptrons (MLP) and their universality properties.

Original: Yet, for the sake of simplicity, we do not deﬁne precisely what we mean by MLP.
Revised: Yet, for the sake of simplicity, we do not deﬁne precisely what we mean by MLP.

Original: Given two ﬁnite-dimensional feature spaces F 0 and F 1 , we only assume we are given a class of (continuous) MLP from F 0 to F 1 which is large enough to be dense in C ( F 0 , F 1 ) .
Revised: Given two ﬁnite-dimensional feature spaces F 0 and F 1 , we only assume we are given a class of (continuous) MLP from F 0 to F 1 which is large enough to be dense in C ( F 0 , F 1 ) .

Original: See for instance
Revised: See for instance

Original: Hornik et al.
Revised: Hornik et al.

Original: ; Cybenko (1989) for precise conditions for MLP to be universal.
Revised: ; Cybenko (1989) for precise conditions for MLP to be universal.

Original: C. A UGMENTED SEPARATING POWER
Revised: C. A UGMENTED SEPARATING POWER

Original: To factor the proof, we introduce another notion of separating power.
Revised: To factor the proof, we introduce another notion of separating power.

Original: Deﬁnition 7.
Revised: Deﬁnition 7.

Original: For n, m ≥ 1 ﬁxed, let X be a set, F be a some space and F be a set of functions from
Revised: For n, m ≥ 1 ﬁxed, let X be a set, F be a some space and F be a set of functions from

Original: X to Y = F n × m .
Revised: X to Y = F n × m .

Original: Then, the augmented separating power of F is,
Revised: Then, the augmented separating power of F is,

Original: [Equation]
Revised: [Equation]

Original: Explicitly, for x, y ∈ X , i, j ∈ { 1 , . . . , n } ,
Revised: Explicitly, for x, y ∈ X , i, j ∈ { 1 , . . . , n } ,

Original: [Equation]
Revised: [Equation]

Original: Note that when n = m = 1 , the augmented separating power is exactly the same as the original separating power, so we identify ρ ( . ) with ρ augm 1 ,( . ) .
Revised: Note that when n = m = 1 , the augmented separating power is exactly the same as the original separating power, so we identify ρ ( . ) with ρ augm 1 ,( . ) .

Original: We also identify ρ augm n,( . ) with ρ augm 1 ,n ( . ) , that we denote by ρ augmn ( . ) .
Revised: We also identify ρ augm n,( . ) with ρ augm 1 ,n ( . ) , that we denote by ρ augmn ( . ) .

Original: First, it is easy to see that this notion is more precise than the separating power.
Revised: First, it is easy to see that this notion is more precise than the separating power.

Original: Lemma 8.
Revised: Lemma 8.

Original: If F and G are set of functions from X to F n × m ,
Revised: If F and G are set of functions from X to F n × m ,

Original: [Equation]
Revised: [Equation]

Original: and, in particular,
Revised: and, in particular,

Original: [Equation]
Revised: [Equation]

Original: The interest in this notion is justiﬁed by the following lemma, which shows that this notion behaves well under composition with “reduction layers”.
Revised: The interest in this notion is justiﬁed by the following lemma, which shows that this notion behaves well under composition with “reduction layers”.

Original: Lemma 9.
Revised: Lemma 9.

Original: For n, m ≥ 1 ﬁxed, let X be a compact topological space, Y = F n × m , F some ﬁnitedimensional space,
Revised: For n, m ≥ 1 ﬁxed, let X be a compact topological space, Y = F n × m , F some ﬁnitedimensional space,

Original: F ⊆ C ( X, Y ) , and τ : X → Z n × m a function for some space Z .
Revised: F ⊆ C ( X, Y ) , and τ : X → Z n × m a function for some space Z .

Original: Deﬁne (cid:101) F ⊆ C ( X, F n ) by,
Revised: Deﬁne (cid:101) F ⊆ C ( X, F n ) by,

Original: [Equation]
Revised: [Equation]

Original: and (cid:101)
Revised: and (cid:101)

Original: τ : X → (cid:101) Z n by, for x ∈ X ,
Revised: τ : X → (cid:101) Z n by, for x ∈ X ,

Original: [Equation]
Revised: [Equation]

Original: Then,
Revised: Then,

Original: [Equation]
Revised: [Equation]

Original: Note that, in the statement, we implicitly see (cid:101) F as a set of functions from X to F n × 1 to ﬁt the deﬁnition of augmented separating power.
Revised: Note that, in the statement, we implicitly see (cid:101) F as a set of functions from X to F n × 1 to ﬁt the deﬁnition of augmented separating power.

Original: Proof.
Revised: Proof.

Original: We show the two inclusions independently.
Revised: We show the two inclusions independently.

Original: ( ⊆ )
Revised: ( ⊆ )

Original: We ﬁrst show that,
Revised: We ﬁrst show that,

Original: [Equation]
Revised: [Equation]

Original: Take ( x, i, y, k ) ∈
Revised: Take ( x, i, y, k ) ∈

Original: ρ augmn,m (cid:16) (cid:101) F (cid:17) .
Revised: ρ augmn,m (cid:16) (cid:101) F (cid:17) .

Original: This means that, for any h : F → F , f ∈ F , m (cid:88)
Revised: This means that, for any h : F → F , f ∈ F , m (cid:88)

Original: j = h
Revised: j = h

Original: [Equation]
Revised: [Equation]

Original: f
Revised: f

Original: [Equation]
Revised: [Equation]

Original: x
Revised: x

Original: [Equation]
Revised: [Equation]

Original: h ( f ( y ) k,j ) .
Revised: h ( f ( y ) k,j ) .

Original: By Lem.
Revised: By Lem.

Original: 31 and the universality of MLP, there exists a permutation σ ∈ S m such that ( f ( x ) i,σ (1) , . . .
Revised: 31 and the universality of MLP, there exists a permutation σ ∈ S m such that ( f ( x ) i,σ (1) , . . .

Original: , f ( x ) i,σ ( m ) )
Revised: , f ( x ) i,σ ( m ) )

Original: =
Revised: =

Original: ( f ( y ) k, 1 , . . .
Revised: ( f ( y ) k, 1 , . . .

Original: , f ( y ) k,m ) .
Revised: , f ( y ) k,m ) .

Original: By deﬁnition of the augmented separating power, this means that, for any j ∈ { 1 , . . .
Revised: By deﬁnition of the augmented separating power, this means that, for any j ∈ { 1 , . . .

Original: , m } , ( x, i, σ ( j ) , y, k, j ) ∈ ρ augmn,m ( F ) .
Revised: , m } , ( x, i, σ ( j ) , y, k, j ) ∈ ρ augmn,m ( F ) .

Original: Hence, by assumption, for any j ∈ { 1 , . . . , m } , ( x, i, σ ( j ) , y, k, j ) ∈ ρ augmn,m ( τ ) , i.e. τ ( x ) i,σ ( j ) =
Revised: Hence, by assumption, for any j ∈ { 1 , . . . , m } , ( x, i, σ ( j ) , y, k, j ) ∈ ρ augmn,m ( τ ) , i.e. τ ( x ) i,σ ( j ) =

Original: τ ( y ) k,j .
Revised: τ ( y ) k,j .

Original: But this exactly means that (cid:101) τ
Revised: But this exactly means that (cid:101) τ

Original: ( x ) i = (cid:101) τ ( y )
Revised: ( x ) i = (cid:101) τ ( y )

Original: k
Revised: k

Original: so that ( x, i, y, k ) ∈
Revised: so that ( x, i, y, k ) ∈

Original: ρ augmn,m (˜ τ ) as required.
Revised: ρ augmn,m (˜ τ ) as required.

Original: ( ⊃ ) We now show the other inclusion,
Revised: ( ⊃ ) We now show the other inclusion,

Original: [Equation]
Revised: [Equation]

Original: Take ( x, i, y, k ) ∈
Revised: Take ( x, i, y, k ) ∈

Original: ρ augmn,m ( (cid:101) τ ) .
Revised: ρ augmn,m ( (cid:101) τ ) .

Original: By deﬁnition of (cid:101) τ , this means that there exists σ ∈ S m such that τ ( x ) i,σ ( j ) =
Revised: By deﬁnition of (cid:101) τ , this means that there exists σ ∈ S m such that τ ( x ) i,σ ( j ) =

Original: τ ( y ) k,j so that ( x, i, σ ( j ) , y, k, j ) ∈ ρ augmn,m ( τ ) ⊆
Revised: τ ( y ) k,j so that ( x, i, σ ( j ) , y, k, j ) ∈ ρ augmn,m ( τ ) ⊆

Original: ρ augmn,m ( F ) .
Revised: ρ augmn,m ( F ) .

Original: Hence, for any f ∈ F , ( f ( x ) i,σ (1) , . . .
Revised: Hence, for any f ∈ F , ( f ( x ) i,σ (1) , . . .

Original: , f ( x ) i,σ ( m ) )
Revised: , f ( x ) i,σ ( m ) )

Original: =
Revised: =

Original: ( f ( y ) k, 1 , . . .
Revised: ( f ( y ) k, 1 , . . .

Original: , f ( y ) k,m ) , and so, for any h : F → F , m (cid:88)
Revised: , f ( y ) k,m ) , and so, for any h : F → F , m (cid:88)

Original: j = h ( f ( x ) i,j ) = m (cid:88)
Revised: j = h ( f ( x ) i,j ) = m (cid:88)

Original: j = h ( f ( y ) k,j ) .
Revised: j = h ( f ( y ) k,j ) .

Original: Therefore, ( x, i, y, k ) ∈ ρ augmn,m (cid:16) (cid:101) F (cid:17) , which concludes the proof.
Revised: Therefore, ( x, i, y, k ) ∈ ρ augmn,m (cid:16) (cid:101) F (cid:17) , which concludes the proof.

Original: I NITIALIZATION LAYER
Revised: I NITIALIZATION LAYER

Original: We use the same initialization layer as Maron et al. (2019a); Chen et al.
Revised: We use the same initialization layer as Maron et al. (2019a); Chen et al.

Original: and recall it below.
Revised: and recall it below.

Original: The initial graph is a tensor of the form G ∈ F n 2 0 with F 0 = R e +2 ; the last channel of G : , : ,e +1 encodes the adjacency matrix of the graph and the ﬁrst e channels G : , : , 1: e are zero outside the diagonal and
Revised: The initial graph is a tensor of the form G ∈ F n 2 0 with F 0 = R e +2 ; the last channel of G : , : ,e +1 encodes the adjacency matrix of the graph and the ﬁrst e channels G : , : , 1: e are zero outside the diagonal and

Original: G i,i, 1: e ∈ R e is the color of vertex v i ∈ V .
Revised: G i,i, 1: e ∈ R e is the color of vertex v i ∈ V .

Original: We then deﬁne I k : F n 2 

Revised: We then deﬁne I k : F n 2

Original: → F n k 1 with F 1 = R k 2 × ( e +2) as follows:
Revised: → F n k 1 with F 1 = R k 2 × ( e +2) as follows:

Original: [Equation]
Revised: [Equation]

Original: for i ∈ [ n ] k and r, s ∈ [ k ] .
Revised: for i ∈ [ n ] k and r, s ∈ [ k ] .

Original: This linear equivariant layer has the same separating power as the isormorphism type, which is deﬁned in Section B.2.
Revised: This linear equivariant layer has the same separating power as the isormorphism type, which is deﬁned in Section B.2.

Original: Lemma 10 ((Maron et al., 2019a, C.1)) .
Revised: Lemma 10 ((Maron et al., 2019a, C.1)) .

Original: For k ≥ 2 , p 0 ≥ 1 , F 0 =
Revised: For k ≥ 2 , p 0 ≥ 1 , F 0 =

Original: R p 0 , F 1 =
Revised: R p 0 , F 1 =

Original: R k 2 × ( p 0 +1) , there exists I k : F n 2 

Revised: R k 2 × ( p 0 +1) , there exists I k : F n 2

Original: → F n k 1 such that,
Revised: → F n k 1 such that,

Original: [Equation]
Revised: [Equation]

Original: C
Revised: C

Original: [Equation]
Revised: [Equation]

Original: K NOWN RESULTS ABOUT THE SEPARATING POWER OF SOME GNN CLASSES
Revised: K NOWN RESULTS ABOUT THE SEPARATING POWER OF SOME GNN CLASSES

Original: First, we need to deﬁne some classes of GNN from which both the invariant and equivaraint GNN we considered are built.
Revised: First, we need to deﬁne some classes of GNN from which both the invariant and equivaraint GNN we considered are built.

Original: See Section 3 for details about the different layers.
Revised: See Section 3 for details about the different layers.

Original: [Equation]
Revised: [Equation]

Original: message passing layer
Revised: message passing layer

Original: [Equation]
Revised: [Equation]

Original: linear equivariant layer
Revised: linear equivariant layer

Original: [Equation]
Revised: [Equation]

Original: [Equation]
Revised: [Equation]

Original: Then, the precise results from the literature can be rephrased as,
Revised: Then, the precise results from the literature can be rephrased as,

Original: Lemma 11 (Xu et al.
Revised: Lemma 11 (Xu et al.

Original: (2018),Maron et al. (2018),Maron et al.
Revised: (2018),Maron et al. (2018),Maron et al.

Original: (2019a)) .
Revised: (2019a)) .

Original: For k ≥ 2 ,
Revised: For k ≥ 2 ,

Original: [Equation] [Equation]
Revised: [Equation] [Equation]

Original: (8) comes from Xu et al.
Revised: (8) comes from Xu et al.

Original: (2018, §A, §B), (9) and (10) from Maron et al. (2019a, §C, §D).
Revised: (2018, §A, §B), (9) and (10) from Maron et al. (2019a, §C, §D).

Original: C. B OUNDING THE SEPARATING POWER OF k -FGNN
Revised: C. B OUNDING THE SEPARATING POWER OF k -FGNN

Original: We complete the results of the literature with a bound on the separating power of k -FGNN.
Revised: We complete the results of the literature with a bound on the separating power of k -FGNN.

Original: Note that the particular case of k = 2 is already proven in Geerts (2020a).
Revised: Note that the particular case of k = 2 is already proven in Geerts (2020a).

Original: Lemma 12.
Revised: Lemma 12.

Original: For any k ≥ 2 ,
Revised: For any k ≥ 2 ,

Original: [Equation]
Revised: [Equation]

Original: so that
Revised: so that

Original: [Equation]
Revised: [Equation]

Original: Proof.
Revised: Proof.

Original: Deﬁne,
Revised: Deﬁne,

Original: [Equation]
Revised: [Equation]

Original: the set of functions deﬁned by exactly T FGL layers.
Revised: the set of functions deﬁned by exactly T FGL layers.

Original: We show by induction that, for any T ≥ 0 ,
Revised: We show by induction that, for any T ≥ 0 ,

Original: [Equation]
Revised: [Equation]

Original: For T = 0 , this is immediate by the deﬁnition of I k in Section C.3.
Revised: For T = 0 , this is immediate by the deﬁnition of I k in Section C.3.

Original: Assume now that this inclusion holds at T − 1 ≥ 0 .
Revised: Assume now that this inclusion holds at T − 1 ≥ 0 .

Original: We show that it also holds at T .
Revised: We show that it also holds at T .

Original: Take G, G (
Revised: Take G, G (

Original: cid:48) ∈ F n kand s, s (cid:48) ∈
Revised: cid:48) ∈ F n kand s, s (cid:48) ∈

Original: [ n ] k such that,
Revised: [ n ] k such that,

Original: [Equation]
Revised: [Equation]

Original: We need to show that, for any f ∈ k -FGNN Temb ,
Revised: We need to show that, for any f ∈ k -FGNN Temb ,

Original: [Equation]
Revised: [Equation]

Original: But, by deﬁnition of the update rule k -FWL, the equality of the colors of s and s (cid:48) above implies that,
Revised: But, by deﬁnition of the update rule k -FWL, the equality of the colors of s and s (cid:48) above implies that,

Original: [Equation]
Revised: [Equation]

Original: and that there exists σ ∈ S n such that, for any j ∈ [ n ] ,
Revised: and that there exists σ ∈ S n such that, for any j ∈ [ n ] ,

Original: [Equation]
Revised: [Equation]

Original: Let s =
Revised: Let s =

Original: ( i 1 , . .
Revised: ( i 1 , . .

Original: , i k ) and s (cid:48) =
Revised: , i k ) and s (cid:48) =

Original: ( j 1 , . . .
Revised: ( j 1 , . . .

Original: , j k ) .
Revised: , j k ) .

Original: Then this implies that, for any w ∈ [ k ] , j ∈ [ n ] ,
Revised: Then this implies that, for any w ∈ [ k ] , j ∈ [ n ] ,

Original: [Equation]
Revised: [Equation]

Original: We now use the induction hypothesis, i.e. that,
Revised: We now use the induction hypothesis, i.e. that,

Original: [Equation]
Revised: [Equation]

Original: Take any f T − 1 ∈
Revised: Take any f T − 1 ∈

Original: k
Revised: k

Original: -FGNN
Revised: -FGNN

Original: T − 1 emb .
Revised: T − 1 emb .

Original: By (11),
Revised: By (11),

Original: [Equation]
Revised: [Equation]

Original: By (12), for any w ∈ [ k ] , j ∈ [ n ] ,
Revised: By (12), for any w ∈ [ k ] , j ∈ [ n ] ,

Original: [Equation]
Revised: [Equation]

Original: By the deﬁnition of FGL Section 3.3, for any F T : F n k T → F n k T +1 FGL, F T ◦ f T − 1 ( G ) s =
Revised: By the deﬁnition of FGL Section 3.3, for any F T : F n k T → F n k T +1 FGL, F T ◦ f T − 1 ( G ) s =

Original: F T ◦ f T − 1 ( G (cid:48) )
Revised: F T ◦ f T − 1 ( G (cid:48) )

Original: s (cid:48) .
Revised: s (cid:48) .

Original: Therefore, for any f ∈ k -FGNN Temb ,
Revised: Therefore, for any f ∈ k -FGNN Temb ,

Original: [Equation]
Revised: [Equation]

Original: which concludes the proof.
Revised: which concludes the proof.

Original: C. C ONCLUSION
Revised: C. C ONCLUSION

Original: Proposition 13.
Revised: Proposition 13.

Original: We have, for k ≥ 2 ,
Revised: We have, for k ≥ 2 ,

Original: [Equation]
Revised: [Equation]

Original: Proof.
Revised: Proof.

Original: Most of the statements come from the literature or are direct consequences of the lemmas above.
Revised: Most of the statements come from the literature or are direct consequences of the lemmas above.

Original: • Proof of (13).
Revised: • Proof of (13).

Original: The invariant case is proven in Xu et al.
Revised: The invariant case is proven in Xu et al.

Original: (2018, Lem. 2, Thm. 3).
Revised: (2018, Lem. 2, Thm. 3).

Original: The equivariant case comes from Lem.
Revised: The equivariant case comes from Lem.

Original: 11, Lem.
Revised: 11, Lem.

Original: 8, the fact that the layers m E ◦ (Id + λS 1 ) does not change the separating power and recalling that simply WL E = c WL .
Revised: 8, the fact that the layers m E ◦ (Id + λS 1 ) does not change the separating power and recalling that simply WL E = c WL .

Original: • (14) is the exact result Chen et al.
Revised: • (14) is the exact result Chen et al.

Original: (2020, Thm. 6).
Revised: (2020, Thm. 6).

Original: • Proof of (15).
Revised: • Proof of (15).

Original: The invariant case is exactly Maron et al.
Revised: The invariant case is exactly Maron et al.

Original: (2019a, Thm. 1).
Revised: (2019a, Thm. 1).

Original: The equivariant case comes from Lem.
Revised: The equivariant case comes from Lem.

Original: 11, Lem.
Revised: 11, Lem.

Original: 9, Lem.
Revised: 9, Lem.

Original: 8 and the fact that the layer m E does not change the separating power.
Revised: 8 and the fact that the layer m E does not change the separating power.

Original: • Proof of (16).
Revised: • Proof of (16).

Original: The direct inclusion of the invariant case corresponds to Maron et al.
Revised: The direct inclusion of the invariant case corresponds to Maron et al.

Original: (2019a, Thm. 2).
Revised: (2019a, Thm. 2).

Original: The other cases are a consequence of Lem.
Revised: The other cases are a consequence of Lem.

Original: 11, Lem.
Revised: 11, Lem.

Original: 9, Lem.
Revised: 9, Lem.

Original: 8 and the fact that the layers m I and m E does not change the separating power.
Revised: 8 and the fact that the layers m I and m E does not change the separating power.

Original: D S TONE -W EIERSTRASS THEOREM WITH SYMMETRIES
Revised: D S TONE -W EIERSTRASS THEOREM WITH SYMMETRIES

Original: This section presents our extension of the Stone-Weierstrass theorem dealing with functions with symmetries.
Revised: This section presents our extension of the Stone-Weierstrass theorem dealing with functions with symmetries.

Original: The scope of this section is not restricted to graphs or even tensors and we will deal with general spaces and general symmetries.
Revised: The scope of this section is not restricted to graphs or even tensors and we will deal with general spaces and general symmetries.

Original: To illustrate it, we will present applications for the PointNet architecture Qi et al.
Revised: To illustrate it, we will present applications for the PointNet architecture Qi et al.

Original: Our approximation results for GNNs (Theorems Thm. 5 and Thm. 6) are then obtained from these theoretical results applied to tensors and the symmetric group in Section
Revised: Our approximation results for GNNs (Theorems Thm. 5 and Thm. 6) are then obtained from these theoretical results applied to tensors and the symmetric group in Section

Original: D.
Revised: D.

Original: D. G ENERAL NOTATIONS
Revised: D. G ENERAL NOTATIONS

Original: As explained above, we are dealing in this section with a much larger scope than graphs and permutations.
Revised: As explained above, we are dealing in this section with a much larger scope than graphs and permutations.

Original: We ﬁrst need to extend the notations introduced above.
Revised: We ﬁrst need to extend the notations introduced above.

Original: The notations introduced below will make this section self-contained.
Revised: The notations introduced below will make this section self-contained.

Original: If X is some topological space, and F ⊆ X , denote by F its closure.
Revised: If X is some topological space, and F ⊆ X , denote by F its closure.

Original: If X is a topological space and Y
Revised: If X is a topological space and Y

Original: =
Revised: =

Original: R p
Revised: R p

Original: some ﬁnite-dimensional space, denote by C ( X, Y ) the set of continuous functions from X to Y .
Revised: some ﬁnite-dimensional space, denote by C ( X, Y ) the set of continuous functions from X to Y .

Original: Moreover, if X is compact, we endow C
Revised: Moreover, if X is compact, we endow C

Original: ( X, Y ) with the topology of uniform convergence, which is deﬁned by the norm, f (cid:55)→ sup x ∈ X (cid:107) f
Revised: ( X, Y ) with the topology of uniform convergence, which is deﬁned by the norm, f (cid:55)→ sup x ∈ X (cid:107) f

Original: ( x ) (cid:107) for some norm (cid:107) .
Revised: ( x ) (cid:107) for some norm (cid:107) .

Original: (cid:107) on Y .
Revised: (cid:107) on Y .

Original: If G is a ﬁnite group acting on some topological space X , we say that G acts continuously on X
Revised: If G is a ﬁnite group acting on some topological space X , we say that G acts continuously on X

Original: if, for all g ∈ G , x (cid:55)→ g · x is continuous.
Revised: if, for all g ∈ G , x (cid:55)→ g · x is continuous.

Original: If G is a ﬁnite group acting on some compact set X and some topological space Y , we deﬁne the sets of equivariant and invariant continuous functions by,
Revised: If G is a ﬁnite group acting on some compact set X and some topological space Y , we deﬁne the sets of equivariant and invariant continuous functions by,

Original: [Equation]
Revised: [Equation]

Original: Note that these deﬁnitions extend Deﬁnition 1 to a general group.
Revised: Note that these deﬁnitions extend Deﬁnition 1 to a general group.

Original: If Y = R p , we denote the coordinate-wise multiplication, or Hadamard product of y, y (cid:48)
Revised: If Y = R p , we denote the coordinate-wise multiplication, or Hadamard product of y, y (cid:48)

Original: ∈ Y simply by yy (cid:48)
Revised: ∈ Y simply by yy (cid:48)

Original: =
Revised: =

Original: ( y 1 y (cid:48) 1 , . . .
Revised: ( y 1 y (cid:48) 1 , . . .

Original: , y p y (cid:48) p ) ∈ Y .
Revised: , y p y (cid:48) p ) ∈ Y .

Original: We say that a subset A ⊆ R p is a subalgebra of R p if it is both a linear space and stable by multiplication.
Revised: We say that a subset A ⊆ R p is a subalgebra of R p if it is both a linear space and stable by multiplication.

Original: This product in turn deﬁnes a product on C ( X, Y ) with Y =
Revised: This product in turn deﬁnes a product on C ( X, Y ) with Y =

Original: R p by, for f, g ∈ C
Revised: R p by, for f, g ∈ C

Original: ( X, Y ) , fg : x (cid:55)→ f ( x ) g ( x ) .
Revised: ( X, Y ) , fg : x (cid:55)→ f ( x ) g ( x ) .

Original: In addition, we also extend the scalar-vector product of Y =
Revised: In addition, we also extend the scalar-vector product of Y =

Original: R p to functions: if g ∈ C ( X, R ) and f ∈ C ( X, Y ) , their product gf is the function gf : x (cid:55)→ g ( x ) f ( x ) .
Revised: R p to functions: if g ∈ C ( X, R ) and f ∈ C ( X, Y ) , their product gf is the function gf : x (cid:55)→ g ( x ) f ( x ) .

Original: Given a set of scalar functions
Revised: Given a set of scalar functions

Original: S ⊆ C ( X, R ) and a set of vector-valued functions F ⊆ C ( X, Y ) , the set of products of functions of these two sets will be denoted by,
Revised: S ⊆ C ( X, R ) and a set of vector-valued functions F ⊆ C ( X, Y ) , the set of products of functions of these two sets will be denoted by,

Original: [Equation]
Revised: [Equation]

Original: Moreover, we denote by 1 the continuous function from some X to R p deﬁned by x (cid:55)→ (1 , . . .
Revised: Moreover, we denote by 1 the continuous function from some X to R p deﬁned by x (cid:55)→ (1 , . . .

Original: In particular, if f is a function from X to R , f 1 denotes the function x
Revised: In particular, if f is a function from X to R , f 1 denotes the function x

Original: (cid:55)→ ( f ( x ) , . . . , f ( x )) which goes from X to Y
Revised: (cid:55)→ ( f ( x ) , . . . , f ( x )) which goes from X to Y

Original: = R p .
Revised: = R p .

Original: Finally, we say that F ⊆ C ( X, Y ) with Y being some R p is a subalgebra if it is a linear space which is also stable by multiplication.
Revised: Finally, we say that F ⊆ C ( X, Y ) with Y being some R p is a subalgebra if it is a linear space which is also stable by multiplication.

Original: D. S EPARATING POWER
Revised: D. S EPARATING POWER

Original: We recall the deﬁnition of separating power that we introduced above:
Revised: We recall the deﬁnition of separating power that we introduced above:

Original: Deﬁnition 14.
Revised: Deﬁnition 14.

Original: Let F be a set of functions f deﬁned on a set X , where each f takes its values in some Y f .
Revised: Let F be a set of functions f deﬁned on a set X , where each f takes its values in some Y f .

Original: The equivalence relation ρ ( F ) deﬁned by F on X is: for any x, x
Revised: The equivalence relation ρ ( F ) deﬁned by F on X is: for any x, x

Original: (cid:48) ∈ X , ( x, x (cid:48) )
Revised: (cid:48) ∈ X , ( x, x (cid:48) )

Original: ∈
Revised: ∈

Original: ρ ( F ) ⇐⇒ ∀ f ∈ F , f ( x ) =
Revised: ρ ( F ) ⇐⇒ ∀ f ∈ F , f ( x ) =

Original: f
Revised: f

Original: ( x (cid:48) ) .
Revised: ( x (cid:48) ) .

Original: For a function f , we write ρ ( f ) for ρ ( { f } ) .
Revised: For a function f , we write ρ ( f ) for ρ ( { f } ) .

Original: Separating power is stable by closure:
Revised: Separating power is stable by closure:

Original: Lemma 15.
Revised: Lemma 15.

Original: Let X be a compact topological space, Y be some ﬁnite-dimensional and F ⊆ C ( X, Y ) .
Revised: Let X be a compact topological space, Y be some ﬁnite-dimensional and F ⊆ C ( X, Y ) .

Original: Then,
Revised: Then,

Original: [Equation]
Revised: [Equation]

Original: =
Revised: =

Original: [Equation]
Revised: [Equation]

Original: Proof
Revised: Proof

Original: F ⊆ F
Revised: F ⊆ F

Original: F
Revised: F

Original: more separating
Revised: more separating

Original: e. ρ
Revised: e. ρ

Original: Conversely, take ( x, y ) / ∈ ρ (cid:0) F (cid:1) .
Revised: Conversely, take ( x, y ) / ∈ ρ (cid:0) F (cid:1) .

Original: By deﬁnition, there exists h ∈ F such that h ( x ) (cid:54) =
Revised: By deﬁnition, there exists h ∈ F such that h ( x ) (cid:54) =

Original: h ( y )
Revised: h ( y )

Original: so that if (cid:15) = (cid:107) h
Revised: so that if (cid:15) = (cid:107) h

Original: ( x ) − h ( y ) (cid:107) , (cid:15) > 0 (for some norm (cid:107) .
Revised: ( x ) − h ( y ) (cid:107) , (cid:15) > 0 (for some norm (cid:107) .

Original: (cid:107) on Y ).
Revised: (cid:107) on Y ).

Original: As h ∈ F , there is some f ∈ F such that sup X (cid:107) h − f (cid:107) ≤ (cid:15) 3 .
Revised: As h ∈ F , there is some f ∈ F such that sup X (cid:107) h − f (cid:107) ≤ (cid:15) 3 .

Original: Therefore, by the triangular inequality,
Revised: Therefore, by the triangular inequality,

Original: [Equation] [Equation]
Revised: [Equation] [Equation]

Original: ) (cid:107) .
Revised: ) (cid:107) .

Original: It follows that (cid:107) f
Revised: It follows that (cid:107) f

Original: ( x ) − f ( y ) (cid:107) ≥ (cid:15) 3 > 0 so that f ( x ) (cid:54) =
Revised: ( x ) − f ( y ) (cid:107) ≥ (cid:15) 3 > 0 so that f ( x ) (cid:54) =

Original: f ( y ) and ( x, y ) / ∈
Revised: f ( y ) and ( x, y ) / ∈

Original: ρ ( F ) .
Revised: ρ ( F ) .

Original: D. A PPROXIMATION THEOREMS FOR REAL - VALUED FUNCTIONS
Revised: D. A PPROXIMATION THEOREMS FOR REAL - VALUED FUNCTIONS

Original: We start by recalling Stone-Weierstrass theorem, see Rudin (1991, Thm. 5.7).
Revised: We start by recalling Stone-Weierstrass theorem, see Rudin (1991, Thm. 5.7).

Original: Theorem 16 (Stone-Weierstrass) .
Revised: Theorem 16 (Stone-Weierstrass) .

Original: Let X be a compact space, and F be a subalgebra of C ( X, R ) the space of real-valued continuous functions of X , which contains the constant function 1 .
Revised: Let X be a compact space, and F be a subalgebra of C ( X, R ) the space of real-valued continuous functions of X , which contains the constant function 1 .

Original: If F separates points, i.e. ρ ( F ) = { ( x, x ) : x ∈ X } , then F is dense in C ( X, R ) .
Revised: If F separates points, i.e. ρ ( F ) = { ( x, x ) : x ∈ X } , then F is dense in C ( X, R ) .

Original: We now prove an extension of this classical result due to Timofte (2005) allowing us to deal with much smaller F by dropping the requirement that F separates points.
Revised: We now prove an extension of this classical result due to Timofte (2005) allowing us to deal with much smaller F by dropping the requirement that F separates points.

Original: Corollary 17.
Revised: Corollary 17.

Original: Let X be a compact space, and F be a subalgebra of C ( X, R ) the space of real-valued continuous functions of X , which contains the constant function 1 .
Revised: Let X be a compact space, and F be a subalgebra of C ( X, R ) the space of real-valued continuous functions of X , which contains the constant function 1 .

Original: Then,
Revised: Then,

Original: [Equation]
Revised: [Equation]

Original: Note that if F separates points, we get back the classical result as every function satisﬁes { ( x, x ) : x ∈ X } ⊆
Revised: Note that if F separates points, we get back the classical result as every function satisﬁes { ( x, x ) : x ∈ X } ⊆

Original: ρ ( f ) .
Revised: ρ ( f ) .

Original: Example 18.
Revised: Example 18.

Original: The invariant version of PointNet is able to learn functions of the form (cid:80) i f
Revised: The invariant version of PointNet is able to learn functions of the form (cid:80) i f

Original: ( x i ) for f ∈ C ( R p , R ) .
Revised: ( x i ) for f ∈ C ( R p , R ) .

Original: We can apply Corollary 17 to this setting.
Revised: We can apply Corollary 17 to this setting.

Original: Consider the case where X is a compact subset of R p and F = { x (cid:55)→ g ( (cid:80) ni
Revised: Consider the case where X is a compact subset of R p and F = { x (cid:55)→ g ( (cid:80) ni

Original: =1 f ( x i )) , f ∈ C ( X, R h ) , g ∈ C
Revised: =1 f ( x i )) , f ∈ C ( X, R h ) , g ∈ C

Original: ( R h , R ) } .
Revised: ( R h , R ) } .

Original: F is a subalgebra of C ( X, R )
Revised: F is a subalgebra of C ( X, R )

Original: (indeed of C I ( X, R ) ) which contains the constant function 1 .
Revised: (indeed of C I ( X, R ) ) which contains the constant function 1 .

Original: Then, it easy to see that ρ ( F ) =
Revised: Then, it easy to see that ρ ( F ) =

Original: { ( x, σ (cid:63) x ) , σ ∈ S n } , where σ (cid:63) x is deﬁned by ( σ (cid:63) x ) σ ( i ) =
Revised: { ( x, σ (cid:63) x ) , σ ∈ S n } , where σ (cid:63) x is deﬁned by ( σ (cid:63) x ) σ ( i ) =

Original: x i for all i (see Lem.
Revised: x i for all i (see Lem.

Original: 31 for a formal statement).
Revised: 31 for a formal statement).

Original: Now note that for a function f ∈ C ( X, R ) , the condition { ( x, σ (cid:63) x ) , σ ∈ S n } ⊆
Revised: Now note that for a function f ∈ C ( X, R ) , the condition { ( x, σ (cid:63) x ) , σ ∈ S n } ⊆

Original: ρ ( f ) is equivalent to f ∈ C
Revised: ρ ( f ) is equivalent to f ∈ C

Original: I ( X, R ) .
Revised: I ( X, R ) .

Original: So that Corollary 17 implies that F = C I ( X, R ) which means that PointNet is universal for approximating invariant functions.
Revised: So that Corollary 17 implies that F = C I ( X, R ) which means that PointNet is universal for approximating invariant functions.

Original: This was already proved in Qi et al.
Revised: This was already proved in Qi et al.

Original: We now provide a proof of Corollary 17 for completeness.
Revised: We now provide a proof of Corollary 17 for completeness.

Original: Proof.
Revised: Proof.

Original: The ﬁrst inclusion F ⊆ { f ∈ C ( X, R ) : ρ ( F ) ⊆
Revised: The ﬁrst inclusion F ⊆ { f ∈ C ( X, R ) : ρ ( F ) ⊆

Original: ρ ( f ) } follows from the same argument as the one given below Theorem 4 so we focus on the other one.
Revised: ρ ( f ) } follows from the same argument as the one given below Theorem 4 so we focus on the other one.

Original: For every x ∈ X , let x F denote its ρ ( F ) -class.
Revised: For every x ∈ X , let x F denote its ρ ( F ) -class.

Original: The quotient set and the canonical surjection are:
Revised: The quotient set and the canonical surjection are:

Original: X F = X/ρ ( F ) = { x F , x ∈ X } and π F : X → X F , π F ( x ) = x F .
Revised: X F = X/ρ ( F ) = { x F , x ∈ X } and π F : X → X F , π F ( x ) = x F .

Original: A function g : X → R factorizes as g = ˆ g ◦ π F for some ˆ g : X F → R
Revised: A function g : X → R factorizes as g = ˆ g ◦ π F for some ˆ g : X F → R

Original: if and only if ρ ( F ) ⊆
Revised: if and only if ρ ( F ) ⊆

Original: ρ ( g ) .
Revised: ρ ( g ) .

Original: In this case ˆ g is unique, since π F is a surjection.
Revised: In this case ˆ g is unique, since π F is a surjection.

Original: In particular.
Revised: In particular.

Original: every f ∈ F factorizes uniquely as f = ˆ f ◦ π F , ˆ f : X F → R , and ˆ F = { ˆ f, f ∈ F} clearly separates points on X F .
Revised: every f ∈ F factorizes uniquely as f = ˆ f ◦ π F , ˆ f : X F → R , and ˆ F = { ˆ f, f ∈ F} clearly separates points on X F .

Original: We refer to
Revised: We refer to

Original: Munkres (2000, §22) for the properties of the quotient topology.
Revised: Munkres (2000, §22) for the properties of the quotient topology.

Original: In particular, by the properties of the quotient topology on X F , ˆ F is a subalgebra of C ( X F , R ) and X F is compact.
Revised: In particular, by the properties of the quotient topology on X F , ˆ F is a subalgebra of C ( X F , R ) and X F is compact.

Original: Hence, we can apply Theorem 16 to ˆ F and ˆ F is dense in C ( X F , R ) .
Revised: Hence, we can apply Theorem 16 to ˆ F and ˆ F is dense in C ( X F , R ) .

Original: Now take f ∈ C ( X, R ) with ρ ( F ) ⊆
Revised: Now take f ∈ C ( X, R ) with ρ ( F ) ⊆

Original: ρ ( f ) and we show that f ∈ F .
Revised: ρ ( f ) and we show that f ∈ F .

Original: Again, ρ ( F ) ⊆
Revised: Again, ρ ( F ) ⊆

Original: ρ ( f ) implies that f
Revised: ρ ( f ) implies that f

Original: = ˆ f ◦ π F .
Revised: = ˆ f ◦ π F .

Original: Let (cid:15) > 0 .
Revised: Let (cid:15) > 0 .

Original: By density of ˆ F , there is some ˆ h ∈ ˆ F such that sup x F | ˆ h ( x F ))
Revised: By density of ˆ F , there is some ˆ h ∈ ˆ F such that sup x F | ˆ h ( x F ))

Original: − ˆ f
Revised: − ˆ f

Original: ( x F ) | ≤ (cid:15) .
Revised: ( x F ) | ≤ (cid:15) .

Original: But, by construction of ˆ F , there exits h ∈ F such that ˆ h ◦ π F = h .
Revised: But, by construction of ˆ F , there exits h ∈ F such that ˆ h ◦ π F = h .

Original: Thus,
Revised: Thus,

Original: [Equation]
Revised: [Equation]

Original: As this holds for any (cid:15) > 0 , we have proven that f ∈ F .
Revised: As this holds for any (cid:15) > 0 , we have proven that f ∈ F .

Original: D. T HE EQUIVARIANT APPROXIMATION THEOREM
Revised: D. T HE EQUIVARIANT APPROXIMATION THEOREM

Original: We ﬁrst need to extend Corollary 17 to vector-valued functions.
Revised: We ﬁrst need to extend Corollary 17 to vector-valued functions.

Original: For this, we need to have a vectorvalued version of the Stone-Weierstrass theorem and as shown by the example below additional assumptions have to be made.
Revised: For this, we need to have a vectorvalued version of the Stone-Weierstrass theorem and as shown by the example below additional assumptions have to be made.

Original: Example 19.
Revised: Example 19.

Original: We consider now the equivariant version of PointNet corresponding to the particular case where X is a compact subset of ( R p ) n , Y = R n and F = { x (cid:55)→ ( f ( x 1 ) , . . .
Revised: We consider now the equivariant version of PointNet corresponding to the particular case where X is a compact subset of ( R p ) n , Y = R n and F = { x (cid:55)→ ( f ( x 1 ) , . . .

Original: , f ( x n )) , f ∈
Revised: , f ( x n )) , f ∈

Original: C ( R , R ) } .
Revised: C ( R , R ) } .

Original: Then clearly F is a subalgebra of C E ( X, Y ) containing the constant function 1 and ρ ( F )
Revised: Then clearly F is a subalgebra of C E ( X, Y ) containing the constant function 1 and ρ ( F )

Original: = { ( x, x ) , x ∈ X } .
Revised: = { ( x, x ) , x ∈ X } .

Original: Hence if Corollary 17 would be true with vector-valued functions instead of real-valued functions, we would have that F is dense in C ( X, Y ) .
Revised: Hence if Corollary 17 would be true with vector-valued functions instead of real-valued functions, we would have that F is dense in C ( X, Y ) .

Original: But this can clearly not be true as F ⊆ C E ( X, Y ) which is clearly not dense in C ( X, Y ) .
Revised: But this can clearly not be true as F ⊆ C E ( X, Y ) which is clearly not dense in C ( X, Y ) .

Original: We now present an extension of Corollary 17 also due to Timofte (2005):
Revised: We now present an extension of Corollary 17 also due to Timofte (2005):

Original: Proposition 20.
Revised: Proposition 20.

Original: Let X be a compact space, Y = R p for some p ≥ 1 .
Revised: Let X be a compact space, Y = R p for some p ≥ 1 .

Original: Let F ⊆ C
Revised: Let F ⊆ C

Original: ( X, Y ) .
Revised: ( X, Y ) .

Original: If there exists a nonempty subset S ⊆ C ( X, R ) such that:
Revised: If there exists a nonempty subset S ⊆ C ( X, R ) such that:

Original: [Equation]
Revised: [Equation]

Original: Then we have
Revised: Then we have

Original: [Equation]
Revised: [Equation]

Original: 18) where F ( x ) = { f ( x ) , f ∈ F} .
Revised: 18) where F ( x ) = { f ( x ) , f ∈ F} .

Original: Moreover in (18), we can replace ρ ( F ) by ρ ( S ) .
Revised: Moreover in (18), we can replace ρ ( F ) by ρ ( S ) .

Original: Note that in the particular case
Revised: Note that in the particular case

Original: Y = R and if F is a subalgebra of C
Revised: Y = R and if F is a subalgebra of C

Original: ( X, R ) , then we can take S = F in (17) and if the constant function 1 is in F , then F ( x )
Revised: ( X, R ) , then we can take S = F in (17) and if the constant function 1 is in F , then F ( x )

Original: =
Revised: =

Original: R , so that we recover Corollary 17.
Revised: R , so that we recover Corollary 17.

Original: Now consider the case where F is a subalgebra of C ( X, R p ) .
Revised: Now consider the case where F is a subalgebra of C ( X, R p ) .

Original: We need to ﬁnd a set S ∈ C
Revised: We need to ﬁnd a set S ∈ C

Original: ( X, R ) satisfying (17) i.e. with a better separating power than F but containing real-valued functions such that sf ∈ F for all s ∈ S and f ∈ F .
Revised: ( X, R ) satisfying (17) i.e. with a better separating power than F but containing real-valued functions such that sf ∈ F for all s ∈ S and f ∈ F .

Original: In the sequel, we will consider the set F scal =
Revised: In the sequel, we will consider the set F scal =

Original: { f ∈ C
Revised: { f ∈ C

Original: ( X, R ) : f 1 ∈ F} .
Revised: ( X, R ) : f 1 ∈ F} .

Original: We clearly have F scal · F ⊆ F since F is a subalgebra.
Revised: We clearly have F scal · F ⊆ F since F is a subalgebra.

Original: Hence, in this setting, Prop. 20 can be rewritten as follows:
Revised: Hence, in this setting, Prop. 20 can be rewritten as follows:

Original: Corollary 21.
Revised: Corollary 21.

Original: Let X be a compact space, Y
Revised: Let X be a compact space, Y

Original: = R p for some p , G be a ﬁnite group acting (continuously) on X and F ⊆ C
Revised: = R p for some p , G be a ﬁnite group acting (continuously) on X and F ⊆ C

Original: I ( X, Y ) a (non-empty) set of invariant functions associated to G .
Revised: I ( X, Y ) a (non-empty) set of invariant functions associated to G .

Original: Consider the following assumptions,
Revised: Consider the following assumptions,

Original: F is a sub-algebra of C ( X, Y ) and the constant function 1 is in F . 2.
Revised: F is a sub-algebra of C ( X, Y ) and the constant function 1 is in F . 2.

Original: The set of functions F scal ⊆
Revised: The set of functions F scal ⊆

Original: C ( X, R ) deﬁned by, F scal = { f ∈ C ( X, R ) : f 1 ∈ F} satisfy, ρ ( F scal ) ⊆
Revised: C ( X, R ) deﬁned by, F scal = { f ∈ C ( X, R ) : f 1 ∈ F} satisfy, ρ ( F scal ) ⊆

Original: ρ ( F ) .
Revised: ρ ( F ) .

Original: For any x ∈ X , there exists f ∈ F such that f ( x ) has pairwise distinct coordinates, i
Revised: For any x ∈ X , there exists f ∈ F such that f ( x ) has pairwise distinct coordinates, i

Original: e., for any indices i, j ∈ { 1 , . .
Revised: e., for any indices i, j ∈ { 1 , . .

Original: , p } with i (cid:54) =
Revised: , p } with i (cid:54) =

Original: j , f ( x ) i (cid:54) =
Revised: j , f ( x ) i (cid:54) =

Original: f ( x ) j .
Revised: f ( x ) j .

Original: Then the closure of F (for the topology of uniform convergence) is,
Revised: Then the closure of F (for the topology of uniform convergence) is,

Original: F = { f ∈ C
Revised: F = { f ∈ C

Original: I ( X, Y ) : ρ ( F ) ⊆
Revised: I ( X, Y ) : ρ ( F ) ⊆

Original: ρ ( f ) } .
Revised: ρ ( f ) } .

Original: Note that Assumptions 1 and 2 ensures that (17) is valid, while Assumption 3 ensures that F ( x )
Revised: Note that Assumptions 1 and 2 ensures that (17) is valid, while Assumption 3 ensures that F ( x )

Original: =
Revised: =

Original: R p .
Revised: R p .

Original: Unfortunately, in the equivariant case, the condition ρ ( F scal ) ⊆
Revised: Unfortunately, in the equivariant case, the condition ρ ( F scal ) ⊆

Original: ρ ( F ) is too strong and we now explain how we will relax it.
Revised: ρ ( F ) is too strong and we now explain how we will relax it.

Original: For the sake of simplicity, we consider here the particular setting adapted to graphs: let n ≥ 1 be a ﬁxed number (corresponding to the number of nodes), X be a compact set of graphs in R n 2 and
Revised: For the sake of simplicity, we consider here the particular setting adapted to graphs: let n ≥ 1 be a ﬁxed number (corresponding to the number of nodes), X be a compact set of graphs in R n 2 and

Original: Y = F n with F = R p for some p ≥ 1 .
Revised: Y = F n with F = R p for some p ≥ 1 .

Original: We deﬁne the action of the symmetric group S n on X by ( σ (cid:63) x ) σ ( i ) ,σ ( j ) =
Revised: We deﬁne the action of the symmetric group S n on X by ( σ (cid:63) x ) σ ( i ) ,σ ( j ) =

Original: x i.j and on Y by ( σ (cid:63) y ) σ ( i ) =
Revised: x i.j and on Y by ( σ (cid:63) y ) σ ( i ) =

Original: y i ∈
Revised: y i ∈

Original: R p .
Revised: R p .

Original: Hence the set of continuous equivariant functions C E ( X, Y ) agrees with Deﬁnition 1.
Revised: Hence the set of continuous equivariant functions C E ( X, Y ) agrees with Deﬁnition 1.

Original: Now consider the case where F ⊆ C E ( X, Y ) is a subalgebra of equivariant functions.
Revised: Now consider the case where F ⊆ C E ( X, Y ) is a subalgebra of equivariant functions.

Original: Then, f ∈ F scal needs to be invariant in order for f 1 to be equivariant and hence in F .
Revised: Then, f ∈ F scal needs to be invariant in order for f 1 to be equivariant and hence in F .

Original: As a result, we see that F scal will not separate points of X in the same orbit, i.e. x and σ (cid:63) x .
Revised: As a result, we see that F scal will not separate points of X in the same orbit, i.e. x and σ (cid:63) x .

Original: But these points will typically be separated by F , since for any f ∈ F , we have f ( σ (cid:63) x )
Revised: But these points will typically be separated by F , since for any f ∈ F , we have f ( σ (cid:63) x )

Original: = σ (cid:63) f
Revised: = σ (cid:63) f

Original: ( x ) which is not equal to f ( x ) unless f is invariant.
Revised: ( x ) which is not equal to f ( x ) unless f is invariant.

Original: We see that we need somehow to require a weaker separating power for F .
Revised: We see that we need somehow to require a weaker separating power for F .

Original: More formally, two isomorphic graphs will have permuted outputs through an equivariant function, but should not be considered as separated.
Revised: More formally, two isomorphic graphs will have permuted outputs through an equivariant function, but should not be considered as separated.

Original: Let Orb ( x ) =
Revised: Let Orb ( x ) =

Original: { σ (cid:63) x, σ ∈ S n } and Orb ( y ) =
Revised: { σ (cid:63) x, σ ∈ S n } and Orb ( y ) =

Original: { σ (cid:63) y, σ ∈ S n } .
Revised: { σ (cid:63) y, σ ∈ S n } .

Original: For any equivariant function f ∈ C E ( X, Y ) , for any z ∈ Orb ( x ) , we have f ( z ) ∈
Revised: For any equivariant function f ∈ C E ( X, Y ) , for any z ∈ Orb ( x ) , we have f ( z ) ∈

Original: Orb ( f ( x )) .
Revised: Orb ( f ( x )) .

Original: Then let π : Y → Y/ S n be the canonical projection π ( y ) =
Revised: Then let π : Y → Y/ S n be the canonical projection π ( y ) =

Original: Orb ( y ) .
Revised: Orb ( y ) .

Original: We deﬁne
Revised: We deﬁne

Original: [Equation]
Revised: [Equation]

Original: In particular, we see that if x (cid:48) ∈
Revised: In particular, we see that if x (cid:48) ∈

Original: Orb ( x ) then ( x, x (cid:48) )
Revised: Orb ( x ) then ( x, x (cid:48) )

Original: ∈
Revised: ∈

Original: ρ ( π ◦ F ) for any F ∈ C E ( X, Y ) .
Revised: ρ ( π ◦ F ) for any F ∈ C E ( X, Y ) .

Original: Moreover, two graphs x and x (cid:48) are ρ ( π ◦ F ) -distinct
Revised: Moreover, two graphs x and x (cid:48) are ρ ( π ◦ F ) -distinct

Original: if there exists a function f ∈ F such that ∀ σ , f ( σ (cid:63) x ) (cid:54) =
Revised: if there exists a function f ∈ F such that ∀ σ , f ( σ (cid:63) x ) (cid:54) =

Original: f ( x (cid:48) ) , i.e. the function f discriminates Orb ( x ) from Orb ( x (cid:48) ) in the sense that for any z ∈ Orb ( x ) and z (cid:48)
Revised: f ( x (cid:48) ) , i.e. the function f discriminates Orb ( x ) from Orb ( x (cid:48) ) in the sense that for any z ∈ Orb ( x ) and z (cid:48)

Original: ∈
Revised: ∈

Original: Orb ( x (cid:48) ) , we have f ( z ) (cid:54) =
Revised: Orb ( x (cid:48) ) , we have f ( z ) (cid:54) =

Original: f ( z (cid:48) ) .
Revised: f ( z (cid:48) ) .

Original: To obtain an equivalent of Proposition 20 with C E ( X, Y ) replacing C ( X, Y ) , we are able to relax assumption (17) to ρ ( F scal ) ⊆
Revised: To obtain an equivalent of Proposition 20 with C E ( X, Y ) replacing C ( X, Y ) , we are able to relax assumption (17) to ρ ( F scal ) ⊆

Original: ρ ( π ◦ F ) .
Revised: ρ ( π ◦ F ) .

Original: Our main general result in this direction is the following theorem (proved in Section D.7) which might be of independent interest:
Revised: Our main general result in this direction is the following theorem (proved in Section D.7) which might be of independent interest:

Original: Theorem 22.
Revised: Theorem 22.

Original: Let X be a compact space, Y
Revised: Let X be a compact space, Y

Original: = R p for some p , G be a ﬁnite group acting (continuously) on X and Y and F ⊆ C E ( X, Y ) a (non-empty) set of equivariant functions.
Revised: = R p for some p , G be a ﬁnite group acting (continuously) on X and Y and F ⊆ C E ( X, Y ) a (non-empty) set of equivariant functions.

Original: Denote by π : Y −→ Y/G the canonical projection on the quotient space Y/G .
Revised: Denote by π : Y −→ Y/G the canonical projection on the quotient space Y/G .

Original: Consider the following assumptions,
Revised: Consider the following assumptions,

Original: F is a sub-algebra of C ( X, Y ) and the constant function 1 is in F . 2.
Revised: F is a sub-algebra of C ( X, Y ) and the constant function 1 is in F . 2.

Original: The set of functions F scal ⊆
Revised: The set of functions F scal ⊆

Original: C ( X, R ) deﬁned by,
Revised: C ( X, R ) deﬁned by,

Original: [Equation]
Revised: [Equation]

Original: satisfy,
Revised: satisfy,

Original: [Equation]
Revised: [Equation]

Original: Then the closure of F (for the topology of uniform convergence) is,
Revised: Then the closure of F (for the topology of uniform convergence) is,

Original: [Equation]
Revised: [Equation]

Original: where F ( x ) = { f ( x ) , f ∈ F} .
Revised: where F ( x ) = { f ( x ) , f ∈ F} .

Original: Moreover, if I ( x ) = { ( i, j ) ∈ [ p ] 2 : ∀ y ∈ F ( x ) , y i =
Revised: Moreover, if I ( x ) = { ( i, j ) ∈ [ p ] 2 : ∀ y ∈ F ( x ) , y i =

Original: y j } , then we have:
Revised: y j } , then we have:

Original: [Equation]
Revised: [Equation]

Original: Example 23.
Revised: Example 23.

Original: We now demonstrate how Theorem 22 can be used to recover the universality results in Segol & Lipman (2020).
Revised: We now demonstrate how Theorem 22 can be used to recover the universality results in Segol & Lipman (2020).

Original: In this paper, the authors study equivariant neural network architectures working with unordered sets, corresponding in our case to X =
Revised: In this paper, the authors study equivariant neural network architectures working with unordered sets, corresponding in our case to X =

Original: Y = R n and the group being the symmetric group S n .
Revised: Y = R n and the group being the symmetric group S n .

Original: They show that the PointNet architecture cannot approximate any (continuous) equivariant function and that adding a single so-called transmission layer is enough to make this architecture universal.
Revised: They show that the PointNet architecture cannot approximate any (continuous) equivariant function and that adding a single so-called transmission layer is enough to make this architecture universal.

Original: Indeed, PointNet can only learn maps of the form x ∈ R n (cid:55)→ ( f ( x 1 ) . . .
Revised: Indeed, PointNet can only learn maps of the form x ∈ R n (cid:55)→ ( f ( x 1 ) . . .

Original: f ( x n )) , which are not universal in the class of equivariant functions, as shown by Segol & Lipman (2020, Lem. 3).
Revised: f ( x n )) , which are not universal in the class of equivariant functions, as shown by Segol & Lipman (2020, Lem. 3).

Original: Now, their transmission layer is a map of the form x ∈ R n (cid:55)→ ( 1 T x ) 1 .
Revised: Now, their transmission layer is a map of the form x ∈ R n (cid:55)→ ( 1 T x ) 1 .

Original: Therefore, in PointNetST, adding such a layer precisely adds a large class of functions to F = { ( f ( x 1 , (cid:80) i g ( x i )) , . . .
Revised: Therefore, in PointNetST, adding such a layer precisely adds a large class of functions to F = { ( f ( x 1 , (cid:80) i g ( x i )) , . . .

Original: , f ( x n , (cid:80) i g ( x i ))) , f ∈
Revised: , f ( x n , (cid:80) i g ( x i ))) , f ∈

Original: C ( R × R h , R ) , g ∈ C ( R , R h ) , h ≥ 1 } .
Revised: C ( R × R h , R ) , g ∈ C ( R , R h ) , h ≥ 1 } .

Original: F is still an algebra and as shown in Example 19, we have ρ ( F ) = { ( x, x ) , x ∈ X } .
Revised: F is still an algebra and as shown in Example 19, we have ρ ( F ) = { ( x, x ) , x ∈ X } .

Original: Moreover, we have F scal =
Revised: Moreover, we have F scal =

Original: C I ( X, R ) by Lem.
Revised: C I ( X, R ) by Lem.

Original: 33 in particular, we get ρ ( F scal )
Revised: 33 in particular, we get ρ ( F scal )

Original: = { ( x, σ (cid:63) x ) , x ∈ X } , so that we obviously have ρ ( F scal ) ⊆
Revised: = { ( x, σ (cid:63) x ) , x ∈ X } , so that we obviously have ρ ( F scal ) ⊆

Original: ρ ( π ◦ F ) .
Revised: ρ ( π ◦ F ) .

Original: In summary,
Revised: In summary,

Original: Theorem 22 implies the universality of PointNetST in C E ( X, Y ) .
Revised: Theorem 22 implies the universality of PointNetST in C E ( X, Y ) .

Original: D. A PRELIMINARY VERSION OF THE EQUIVARIANT APPROXIMATION THEOREM
Revised: D. A PRELIMINARY VERSION OF THE EQUIVARIANT APPROXIMATION THEOREM

Original: We start by proving a version of Theorem 22 with a slightly weaker condition:
Revised: We start by proving a version of Theorem 22 with a slightly weaker condition:

Original: Proposition 24.
Revised: Proposition 24.

Original: Let X be a compact space, Y
Revised: Let X be a compact space, Y

Original: = R p for some p , G be a ﬁnite group acting (continuously) on X and Y and F ⊆ C E ( X, Y ) a (non-empty) set of equivariant functions.
Revised: = R p for some p , G be a ﬁnite group acting (continuously) on X and Y and F ⊆ C E ( X, Y ) a (non-empty) set of equivariant functions.

Original: Consider the following assumptions,
Revised: Consider the following assumptions,

Original: F is a subalgebra C E ( X, Y ) .
Revised: F is a subalgebra C E ( X, Y ) .

Original: The set of real-valued functions F scal ⊆
Revised: The set of real-valued functions F scal ⊆

Original: C ( X, R ) deﬁned by,
Revised: C ( X, R ) deﬁned by,

Original: [Equation]
Revised: [Equation]

Original: Then the closure of F (for the topology of uniform convergence) is,
Revised: Then the closure of F (for the topology of uniform convergence) is,

Original: [Equation]
Revised: [Equation]

Original: where F ( x ) = { f ( x ) , f ∈ F} .
Revised: where F ( x ) = { f ( x ) , f ∈ F} .

Original: The proof of this theorem relies on two main ingredients.
Revised: The proof of this theorem relies on two main ingredients.

Original: First, following the elegant idea of Maehara & Hoang (2019), we augment the input space to transform the vector-valued equivariant functions into scalar maps.
Revised: First, following the elegant idea of Maehara & Hoang (2019), we augment the input space to transform the vector-valued equivariant functions into scalar maps.

Original: Second, we apply the ﬁne-grained approximation result Cor. 17.
Revised: Second, we apply the ﬁne-grained approximation result Cor. 17.

Original: Proof.
Revised: Proof.

Original: As uniform convergence implies point-wise convergence, the ﬁrst inclusion is immediate,
Revised: As uniform convergence implies point-wise convergence, the ﬁrst inclusion is immediate,

Original: [Equation]
Revised: [Equation]

Original: The rest of the proof is devoted to the other direction.
Revised: The rest of the proof is devoted to the other direction.

Original: For convenience, denote by Φ the family of linear forms associated to the canonical basis of R p , i.e.,
Revised: For convenience, denote by Φ the family of linear forms associated to the canonical basis of R p , i.e.,

Original: [Equation]
Revised: [Equation]

Original: Y, R ) .
Revised: Y, R ) .

Original: Deﬁne our augmented input space as ˜ X
Revised: Deﬁne our augmented input space as ˜ X

Original: =
Revised: =

Original: X × Φ .
Revised: X × Φ .

Original: As Φ is ﬁnite and X is compact, ˜ X is still a compact space.
Revised: As Φ is ﬁnite and X is compact, ˜ X is still a compact space.

Original: We now transform F , a class of equivariant functions from X to Y , into ˜ F a class of maps from ˜ X to R .
Revised: We now transform F , a class of equivariant functions from X to Y , into ˜ F a class of maps from ˜ X to R .

Original: Deﬁne
Revised: Deﬁne

Original: [Equation]
Revised: [Equation]

Original: We check that ˜ F is indeed a subset of C ( ˜ X, R ) .
Revised: We check that ˜ F is indeed a subset of C ( ˜ X, R ) .

Original: Indeed, as Φ is ﬁnite, it is equipped with the discrete topology.
Revised: Indeed, as Φ is ﬁnite, it is equipped with the discrete topology.

Original: Hence, each singleton { ϕ } for ϕ ∈ Φ is open in Φ and it sufﬁces to check the continuity in the ﬁrst variable with ϕ ﬁxed.
Revised: Hence, each singleton { ϕ } for ϕ ∈ Φ is open in Φ and it sufﬁces to check the continuity in the ﬁrst variable with ϕ ﬁxed.

Original: But, if f ∈ F , x (cid:55)→ ϕ
Revised: But, if f ∈ F , x (cid:55)→ ϕ

Original: ( f ( x )) is continuous as a composition of continuous maps.
Revised: ( f ( x )) is continuous as a composition of continuous maps.

Original: We can now apply Cor.
Revised: We can now apply Cor.

Original: 17 to ˜ F ⊆ C
Revised: 17 to ˜ F ⊆ C

Original: ( ˜ X, R ) .
Revised: ( ˜ X, R ) .

Original: Therefore, the closure of ˜ F in C ( ˜ X, R ) is,
Revised: Therefore, the closure of ˜ F in C ( ˜ X, R ) is,

Original: [Equation]
Revised: [Equation]

Original: We now show the equality of (19).
Revised: We now show the equality of (19).

Original: Take h in the right-hand side of (19), i.e. h ∈ C E ( X, Y ) such that ρ ( F ) ⊆
Revised: Take h in the right-hand side of (19), i.e. h ∈ C E ( X, Y ) such that ρ ( F ) ⊆

Original: ρ ( h ) and h ( x ) ∈ F ( x ) for all x ∈ X .
Revised: ρ ( h ) and h ( x ) ∈ F ( x ) for all x ∈ X .

Original: We show that ˜ h , deﬁned by ˜
Revised: We show that ˜ h , deﬁned by ˜

Original: h : ( x, ϕ )
Revised: h : ( x, ϕ )

Original: (cid:55)→ ϕ
Revised: (cid:55)→ ϕ

Original: ( h ( x )) , belongs to ˜ F using the result above.
Revised: ( h ( x )) , belongs to ˜ F using the result above.

Original: • As h is continuous, by the same argument as above, ( x, ϕ ) (cid:55)→ ϕ ( h ( x )) is continuous on ˜ X .
Revised: • As h is continuous, by the same argument as above, ( x, ϕ ) (cid:55)→ ϕ ( h ( x )) is continuous on ˜ X .

Original: • We check that ρ (cid:16) ˜ F (cid:17) ⊆
Revised: • We check that ρ (cid:16) ˜ F (cid:17) ⊆

Original: ρ (cid:16) ˜
Revised: ρ (cid:16) ˜

Original: h (cid:17) .
Revised: h (cid:17) .

Original: Take ( x, ϕ ) , ( y, ψ ) ∈
Revised: Take ( x, ϕ ) , ( y, ψ ) ∈

Original: ˜
Revised: ˜

Original: X such that, for all f ∈ F ,
Revised: X such that, for all f ∈ F ,

Original: [Equation]
Revised: [Equation]

Original: = ψ ( f (
Revised: = ψ ( f (

Original: [Equation]
Revised: [Equation]

Original: and we aim at showing that ϕ ( h ( x ))
Revised: and we aim at showing that ϕ ( h ( x ))

Original: =
Revised: =

Original: ψ ( h ( y )) .
Revised: ψ ( h ( y )) .

Original: To gain more information from (20), we apply it to functions of the form f 1 with f ∈ F scal .
Revised: To gain more information from (20), we apply it to functions of the form f 1 with f ∈ F scal .

Original: By deﬁnition of Φ , this translates to f ( x ) =
Revised: By deﬁnition of Φ , this translates to f ( x ) =

Original: f ( y ) , for any f ∈ F scal .
Revised: f ( y ) , for any f ∈ F scal .

Original: Therefore ( x, y ) ∈
Revised: Therefore ( x, y ) ∈

Original: ρ ( F scal )
Revised: ρ ( F scal )

Original: and so there exists g ∈ G
Revised: and so there exists g ∈ G

Original: such that ( g · x, y ) ∈
Revised: such that ( g · x, y ) ∈

Original: ρ ( F ) .
Revised: ρ ( F ) .

Original: Plugging this into (20) and using the equivariance of f ,
Revised: Plugging this into (20) and using the equivariance of f ,

Original: [Equation]
Revised: [Equation]

Original: As G acts continuously on Y , both ϕ and z (cid:55)→ ψ ( g · z ) are continuous and, as a consequence of the equality above, coincide on F ( x ) .
Revised: As G acts continuously on Y , both ϕ and z (cid:55)→ ψ ( g · z ) are continuous and, as a consequence of the equality above, coincide on F ( x ) .

Original: But we assumed that h ( x ) ∈ F ( x ) and therefore the equality also holds for h , i.e. ϕ ( h ( x ))
Revised: But we assumed that h ( x ) ∈ F ( x ) and therefore the equality also holds for h , i.e. ϕ ( h ( x ))

Original: = ψ ( g · h ( x )) .
Revised: = ψ ( g · h ( x )) .

Original: Finally, recalls that, by assumption, ρ ( F ) ⊆
Revised: Finally, recalls that, by assumption, ρ ( F ) ⊆

Original: ρ ( h ) .
Revised: ρ ( h ) .

Original: Therefore ( g · x, y ) ∈
Revised: Therefore ( g · x, y ) ∈

Original: ρ ( F ) implies
Revised: ρ ( F ) implies

Original: that
Revised: that

Original: h
Revised: h

Original: ( g · x ) = h ( y ) and
Revised: ( g · x ) = h ( y ) and

Original: , combined
Revised: , combined

Original: with the result
Revised: with the result

Original: above
Revised: above

Original: , ϕ ( h ( x )) =
Revised: , ϕ ( h ( x )) =

Original: ψ (
Revised: ψ (

Original: h
Revised: h

Original: ( y ))
Revised: ( y ))

Original: • We verify that, for x ∈ X ,
Revised: • We verify that, for x ∈ X ,

Original: ϕ ∈
Revised: ϕ ∈

Original: Φ , ϕ ( h ( x )) belongs to ˜ F ( x ) .
Revised: Φ , ϕ ( h ( x )) belongs to ˜ F ( x ) .

Original: Indeed, recall that h ( x ) ∈ F ( x ) .
Revised: Indeed, recall that h ( x ) ∈ F ( x ) .

Original: Therefore, as ϕ is continuous, ϕ ( h ( x )) is in ϕ ( F ( x )) which is included in ˜ F ( x ) .
Revised: Therefore, as ϕ is continuous, ϕ ( h ( x )) is in ϕ ( F ( x )) which is included in ˜ F ( x ) .

Original: This shows that ˜ h : ( x, ϕ ) (cid:55)→ ϕ
Revised: This shows that ˜ h : ( x, ϕ ) (cid:55)→ ϕ

Original: ( h ( x )) is in ˜ F .
Revised: ( h ( x )) is in ˜ F .

Original: Consequently, for any (cid:15) > 0 , there exists f ∈ F such that,
Revised: Consequently, for any (cid:15) > 0 , there exists f ∈ F such that,

Original: [Equation]
Revised: [Equation]

Original: If Y = R p is endowed with the inﬁnity norms on coordinates, by deﬁnition of Φ , this means that,
Revised: If Y = R p is endowed with the inﬁnity norms on coordinates, by deﬁnition of Φ , this means that,

Original: [Equation]
Revised: [Equation]

Original: Remark 25 .
Revised: Remark 25 .

Original: In the particular case of G ⊆ S n being a group of permutations acting on R p by, for g ∈ G , x ∈ R p ,
Revised: In the particular case of G ⊆ S n being a group of permutations acting on R p by, for g ∈ G , x ∈ R p ,

Original: [Equation]
Revised: [Equation]

Original: the functions of ˜ F are indeed invariant, as shown by Maehara & Hoang (2019).
Revised: the functions of ˜ F are indeed invariant, as shown by Maehara & Hoang (2019).

Original: For this, a left action on Φ is deﬁned by, for g ∈ G , ϕ ∈ Φ ,
Revised: For this, a left action on Φ is deﬁned by, for g ∈ G , ϕ ∈ Φ ,

Original: [Equation]
Revised: [Equation]

Original: In other words, the action of g on the linear form associated to the i th coordinate yields the linear form associated to the g ( i ) th coordinate.
Revised: In other words, the action of g on the linear form associated to the i th coordinate yields the linear form associated to the g ( i ) th coordinate.

Original: One can now check that the functions ˜ F are invariant.
Revised: One can now check that the functions ˜ F are invariant.

Original: D. C HARACTERIZING THE SUBALGEBRAS OF R p
Revised: D. C HARACTERIZING THE SUBALGEBRAS OF R p

Original: Before moving to our general result, we need to study the structure of the subalgebras of R p .
Revised: Before moving to our general result, we need to study the structure of the subalgebras of R p .

Original: For this, we will use the following simple lemma.
Revised: For this, we will use the following simple lemma.

Original: In the following lemma, R [ X 1 , . . .
Revised: In the following lemma, R [ X 1 , . . .

Original: , X p ] denotes the set of multivariate polynomials with p indeterminates (and real coefﬁcients).
Revised: , X p ] denotes the set of multivariate polynomials with p indeterminates (and real coefﬁcients).

Original: Lemma 26.
Revised: Lemma 26.

Original: Let
Revised: Let

Original: C ⊆ R p be a ﬁnite subset of R p .
Revised: C ⊆ R p be a ﬁnite subset of R p .

Original: There exists P ∈ R
Revised: There exists P ∈ R

Original: [ X 1 , . . .
Revised: [ X 1 , . . .

Original: , X p ] such that P | C , the restriction of P to C , is an injective map.
Revised: , X p ] such that P | C , the restriction of P to C , is an injective map.

Original: Proof.
Revised: Proof.

Original: Let x 1 , . . .
Revised: Let x 1 , . . .

Original: , x m ∈ R p be distinct vectors such that { x 1 , . . .
Revised: , x m ∈ R p be distinct vectors such that { x 1 , . . .

Original: , x m } = C .
Revised: , x m } = C .

Original: Similarly to Lagrange polynomials, deﬁne,
Revised: Similarly to Lagrange polynomials, deﬁne,

Original: [Equation]
Revised: [Equation]

Original: which is a well-deﬁned multivariate polynomial.
Revised: which is a well-deﬁned multivariate polynomial.

Original: Note that, seeing X =
Revised: Note that, seeing X =

Original: ( X 1 , . . .
Revised: ( X 1 , . . .

Original: , X p ) as a vector in R p , it can also be written as,
Revised: , X p ) as a vector in R p , it can also be written as,

Original: [Equation]
Revised: [Equation]

Original: By construction, P ( x i ) =
Revised: By construction, P ( x i ) =

Original: i and therefore P is an injective map on C
Revised: i and therefore P is an injective map on C

Original: Lemma 27.
Revised: Lemma 27.

Original: For a subalgebra A of R p , we deﬁne:
Revised: For a subalgebra A of R p , we deﬁne:

Original: [Equation]
Revised: [Equation]

Original: Then, we have :
Revised: Then, we have :

Original: [Equation]
Revised: [Equation]

Original: Proof.
Revised: Proof.

Original: Before proving the general case, we focus on the situation where A will turn out to be the whole R p .
Revised: Before proving the general case, we focus on the situation where A will turn out to be the whole R p .

Original: Assume that the two following conditions holds,
Revised: Assume that the two following conditions holds,

Original: [Equation]
Revised: [Equation]

Original: Our goal is to show that, under these additional assumptions, A = R p .
Revised: Our goal is to show that, under these additional assumptions, A = R p .

Original: We divide the proof in three parts, ﬁrst we show that 1 ∈ A using (24), giving us that A is closed under polynomials, then that there is x ∈ A with pairwise distinct coordinates thanks to (23) and ﬁnally that this implies that A is the whole space.
Revised: We divide the proof in three parts, ﬁrst we show that 1 ∈ A using (24), giving us that A is closed under polynomials, then that there is x ∈ A with pairwise distinct coordinates thanks to (23) and ﬁnally that this implies that A is the whole space.

Original: Note that if p = 1 , (24) and the linear space property of A immediately give the result.
Revised: Note that if p = 1 , (24) and the linear space property of A immediately give the result.

Original: • Here
Revised: • Here

Original: we
Revised: we

Original: prove that (24) implies that
Revised: prove that (24) implies that

Original: ∈
Revised: ∈

Original: A
Revised: A

Original: – First, we construct by induction x ∈
Revised: – First, we construct by induction x ∈

Original: A such that x i (cid:54) = 0 for any index
Revised: A such that x i (cid:54) = 0 for any index

Original: i .
Revised: i .

Original: More precisely, our induction hypothesis at step j ∈ { 1 , . .
Revised: More precisely, our induction hypothesis at step j ∈ { 1 , . .

Original: . , p } is,
Revised: . , p } is,

Original: [Equation]
Revised: [Equation]

Original: By (24), this holds for j = 1 .
Revised: By (24), this holds for j = 1 .

Original: Now, assume that it holds at j − 1 for some p ≥ j ≥ 2 and take x ∈
Revised: Now, assume that it holds at j − 1 for some p ≥ j ≥ 2 and take x ∈

Original: A such that x i (cid:54) =for any 1 ≤ i ≤ j − 1 and y ∈
Revised: A such that x i (cid:54) =for any 1 ≤ i ≤ j − 1 and y ∈

Original: A such that j j (cid:54) =
Revised: A such that j j (cid:54) =

Original: by (24).
Revised: by (24).

Original: By deﬁnition of x , the set,
Revised: By deﬁnition of x , the set,

Original: [Equation]
Revised: [Equation]

Original: is ﬁnite.
Revised: is ﬁnite.

Original: Thus, there exists λ ∈ R such that λx i
Revised: Thus, there exists λ ∈ R such that λx i

Original: + y
Revised: + y

Original: i (cid:54) = 

Revised: i (cid:54) =

Original: for any 1 ≤ i ≤ j − 1 and for i =
Revised: for any 1 ≤ i ≤ j − 1 and for i =

Original: j too as x j = 0 and y j (cid:54) = 0 .
Revised: j too as x j = 0 and y j (cid:54) = 0 .

Original: As A is a subalgebra, λx + y ∈ A
Revised: As A is a subalgebra, λx + y ∈ A

Original: and this concludes the induction step.
Revised: and this concludes the induction step.

Original: – Let x ∈
Revised: – Let x ∈

Original: A be the vector constructed, i.e. such that x i (cid:54) = 0 for every index i .
Revised: A be the vector constructed, i.e. such that x i (cid:54) = 0 for every index i .

Original: We prove that 1 ∈ A by constructing 1 from x .
Revised: We prove that 1 ∈ A by constructing 1 from x .

Original: Indeed, using Lagrange interpolation, take P ∈ R
Revised: Indeed, using Lagrange interpolation, take P ∈ R

Original: [ X ] such that P ( x i ) =
Revised: [ X ] such that P ( x i ) =

Original: 1 x i for every i .
Revised: 1 x i for every i .

Original: (Note that this noes not matter if some x i are equal, as the 1 x i would also be the same.)
Revised: (Note that this noes not matter if some x i are equal, as the 1 x i would also be the same.)

Original: Finally, as A is a subalgebra, xP ( x ) , which is to be understood coordinate-wise, is in A and so is 1 =
Revised: Finally, as A is a subalgebra, xP ( x ) , which is to be understood coordinate-wise, is in A and so is 1 =

Original: xP ( x ) .
Revised: xP ( x ) .

Original: • We show that the previous point and (23) imply that there exists a vector in A with pairwise distinct coordinates, i.e. that there exists
Revised: • We show that the previous point and (23) imply that there exists a vector in A with pairwise distinct coordinates, i.e. that there exists

Original: x ∈ A such that, for any i (cid:54) =
Revised: x ∈ A such that, for any i (cid:54) =

Original: j , x i (cid:54) =
Revised: j , x i (cid:54) =

Original: x j .
Revised: x j .

Original: Using (23), for any i < j , there exists x ij ∈
Revised: Using (23), for any i < j , there exists x ij ∈

Original: A such that x ij i (cid:54) = x ij j .
Revised: A such that x ij i (cid:54) = x ij j .

Original: We wish to combine the family ( x ij )
Revised: We wish to combine the family ( x ij )

Original: i<j into a single vector.
Revised: i<j into a single vector.

Original: For this we use Lem.
Revised: For this we use Lem.

Original: Seeing each collection ( x ijk ) i<j as vector of R p ( p − 1) / 2 , we deﬁne
Revised: Seeing each collection ( x ijk ) i<j as vector of R p ( p − 1) / 2 , we deﬁne

Original: C = { ( x ijk ) i<j : 1 ≤ k ≤ p } , which is a ﬁnite subset (of cardinal p ) of R p ( p − 1) / 2 .
Revised: C = { ( x ijk ) i<j : 1 ≤ k ≤ p } , which is a ﬁnite subset (of cardinal p ) of R p ( p − 1) / 2 .

Original: By
Revised: By

Original: Lem.
Revised: Lem.

Original: 26, there exists P ∈ R
Revised: 26, there exists P ∈ R

Original: [ X 1 , . . .
Revised: [ X 1 , . . .

Original: , X p ( p − 1) / 2 ] such that P is an injective map on C .
Revised: , X p ( p − 1) / 2 ] such that P is an injective map on C .

Original: As A is a subalgebra and 1 ∈ A , the vector,
Revised: As A is a subalgebra and 1 ∈ A , the vector,

Original: [Equation]
Revised: [Equation]

Original: is
Revised: is

Original: in
Revised: in

Original: A too.
Revised: A too.

Original: We
Revised: We

Original: now check that
Revised: now check that

Original: this vector
Revised: this vector

Original: has pairwise distinct coordinates
Revised: has pairwise distinct coordinates

Original: Let
Revised: Let

Original: l
Revised: l

Original: < k , then x lkl
Revised: < k , then x lkl

Original: (cid:54)
Revised: (cid:54)

Original: = x lkk and therefore ( x ijl )
Revised: = x lkk and therefore ( x ijl )

Original: i<
Revised: i<

Original: j
Revised: j

Original: (cid:54)
Revised: (cid:54)

Original: =
Revised: =

Original: x ijk ) i<j .
Revised: x ijk ) i<j .

Original: By construction
Revised: By construction

Original: of
Revised: of

Original: P , P (cid:16) ( x ijl ) i<
Revised: P , P (cid:16) ( x ijl ) i<

Original: j
Revised: j

Original: (cid:17) (cid:54) =
Revised: (cid:17) (cid:54) =

Original: P (cid:16) ( x ijk ) i<j (cid:17) , i.e. P (cid:0) ( x ij )
Revised: P (cid:16) ( x ijk ) i<j (cid:17) , i.e. P (cid:0) ( x ij )

Original: i<j (cid:1) l (cid:54) =
Revised: i<j (cid:1) l (cid:54) =

Original: P (cid:0) (
Revised: P (cid:0) (

Original: x ij )
Revised: x ij )

Original: i<j (cid:1) k .
Revised: i<j (cid:1) k .

Original: Thus, P (cid:0) ( x ij )
Revised: Thus, P (cid:0) ( x ij )

Original: i<j (cid:1)
Revised: i<j (cid:1)

Original: ∈
Revised: ∈

Original: A has pairwise distinct coordinates as required.
Revised: A has pairwise distinct coordinates as required.

Original: • Finally, we show that A = R p .
Revised: • Finally, we show that A = R p .

Original: This is a direct consequence of the point above and of Lagrange interpolation.
Revised: This is a direct consequence of the point above and of Lagrange interpolation.

Original: Indeed, take any y ∈ R p and denote by x ∈
Revised: Indeed, take any y ∈ R p and denote by x ∈

Original: A the vector that we just constructed with pairwise distinct coordinates.
Revised: A the vector that we just constructed with pairwise distinct coordinates.

Original: Therefore, by Lagrange interpolation, there exists P ∈ R
Revised: Therefore, by Lagrange interpolation, there exists P ∈ R

Original: [ X ] such that P ( x i ) =
Revised: [ X ] such that P ( x i ) =

Original: y i for every i ∈ { 1 , . . .
Revised: y i for every i ∈ { 1 , . . .

Original: , p } .
Revised: , p } .

Original: As A is a sublagebra and 1 ∈ A , P ( x ) ∈ A and hence y ∈ A .
Revised: As A is a sublagebra and 1 ∈ A , P ( x ) ∈ A and hence y ∈ A .

Original: Finally, we return to the general case.
Revised: Finally, we return to the general case.

Original: We introduce the set of indexes of I and J which appear in the result and use them to reduce the situation to the previous case.
Revised: We introduce the set of indexes of I and J which appear in the result and use them to reduce the situation to the previous case.

Original: Deﬁne,
Revised: Deﬁne,

Original: [Equation]
Revised: [Equation]

Original: ) ∈ ( J C ) 2 : ∀ x ∈ A , x i = x j } .
Revised: ) ∈ ( J C ) 2 : ∀ x ∈ A , x i = x j } .

Original: and denote by A (cid:48)
Revised: and denote by A (cid:48)

Original: = { x ∈ R p : ∀ ( i, j
Revised: = { x ∈ R p : ∀ ( i, j

Original: ) ∈
Revised: ) ∈

Original: I, x i = x j , ∀ j ∈ J, x j = 0 } , which is also a subalgebra.
Revised: I, x i = x j , ∀ j ∈ J, x j = 0 } , which is also a subalgebra.

Original: By deﬁnition, it holds that A ⊆ A (cid:48) .
Revised: By deﬁnition, it holds that A ⊆ A (cid:48) .

Original: By construction, I is an equivalence relation on J C and denote by J C /I
Revised: By construction, I is an equivalence relation on J C and denote by J C /I

Original: its equivalence classes.
Revised: its equivalence classes.

Original: Let p (cid:48) =
Revised: Let p (cid:48) =

Original: | J C /I | and choose i 1 , . .
Revised: | J C /I | and choose i 1 , . .

Original: , i p (cid:48) representatives of the equivalence classes.
Revised: , i p (cid:48) representatives of the equivalence classes.

Original: Consider the map,
Revised: Consider the map,

Original: [Equation]
Revised: [Equation]

Original: ϕ is an algebra homomoprhism so that ϕ ( A ) is a subalgebra of R p (cid:48) .
Revised: ϕ is an algebra homomoprhism so that ϕ ( A ) is a subalgebra of R p (cid:48) .

Original: But, by construction of I and J , ϕ ( A ) satisﬁes (23) and (24).
Revised: But, by construction of I and J , ϕ ( A ) satisﬁes (23) and (24).

Original: Whence, by our result in this particular case, ϕ ( A ) =
Revised: Whence, by our result in this particular case, ϕ ( A ) =

Original: R p (cid:48) .
Revised: R p (cid:48) .

Original: However, A ⊆ A (cid:48) implies that ϕ ( A ) ⊆
Revised: However, A ⊆ A (cid:48) implies that ϕ ( A ) ⊆

Original: ϕ ( A (cid:48) ) ⊆
Revised: ϕ ( A (cid:48) ) ⊆

Original: R p (cid:48) .
Revised: R p (cid:48) .

Original: Therefore, R p (cid:48) =
Revised: Therefore, R p (cid:48) =

Original: ϕ ( A ) =
Revised: ϕ ( A ) =

Original: ϕ
Revised: ϕ

Original: ( A (cid:48) ) .
Revised: ( A (cid:48) ) .

Original: But, by construction of ϕ , ϕ is actually an injective map on A (cid:48) .
Revised: But, by construction of ϕ , ϕ is actually an injective map on A (cid:48) .

Original: Therefore, we deduce from ϕ ( A ) =
Revised: Therefore, we deduce from ϕ ( A ) =

Original: ϕ
Revised: ϕ

Original: ( A (cid:48) )
Revised: ( A (cid:48) )

Original: that A =
Revised: that A =

Original: A (cid:48) , concluding the proof.
Revised: A (cid:48) , concluding the proof.

Original: D.
Revised: D.

Original: P ROOF OF THE MAIN EQUIVARIANT APPROXIMATION THEOREM
Revised: P ROOF OF THE MAIN EQUIVARIANT APPROXIMATION THEOREM

Original: We can now fully exploit the structure of subalgebra of F thanks to the results above, and in particular relax the second assumption of Prop. 24 to give our main theorem 22.
Revised: We can now fully exploit the structure of subalgebra of F thanks to the results above, and in particular relax the second assumption of Prop. 24 to give our main theorem 22.

Original: We ﬁrst prove the following lemma.
Revised: We ﬁrst prove the following lemma.

Original: Lemma 28.
Revised: Lemma 28.

Original: Under the assumptions of Thm. 22, for any H ⊆ G subgroup, for any x, y ∈ X ,
Revised: Under the assumptions of Thm. 22, for any H ⊆ G subgroup, for any x, y ∈ X ,

Original: [Equation]
Revised: [Equation]

Original: , f ( g · x ) = f
Revised: , f ( g · x ) = f

Original: ( y )
Revised: ( y )

Original: ⇐⇒ ∃ g ∈ H, ∀ f ∈ F , f ( g · x ) =
Revised: ⇐⇒ ∃ g ∈ H, ∀ f ∈ F , f ( g · x ) =

Original: f ( y ) .
Revised: f ( y ) .

Original: In particular, ( x, y ) ∈
Revised: In particular, ( x, y ) ∈

Original: ρ ( π ◦ F ) ⇐⇒ ∃ g ∈ G, ( g · x, y ) ∈
Revised: ρ ( π ◦ F ) ⇐⇒ ∃ g ∈ G, ( g · x, y ) ∈

Original: ρ ( F ) .
Revised: ρ ( F ) .

Original: Proof.
Revised: Proof.

Original: The reverse implication is immediate so we focus on the direct one and prove its contraposition, i.e.,
Revised: The reverse implication is immediate so we focus on the direct one and prove its contraposition, i.e.,

Original: [Equation]
Revised: [Equation]

Original: g
Revised: g

Original: [Equation] [Equation]
Revised: [Equation] [Equation]

Original: F , f
Revised: F , f

Original: [Equation]
Revised: [Equation]

Original: g · x
Revised: g · x

Original: [Equation]
Revised: [Equation]

Original: =
Revised: =

Original: [Equation]
Revised: [Equation]

Original: ⇒
Revised: ⇒

Original: [Equation]
Revised: [Equation]

Original: g ∈
Revised: g ∈

Original: [Equation] [Equation]
Revised: [Equation] [Equation]

Original: =
Revised: =

Original: [Equation]
Revised: [Equation]

Original: To prove this, we take advantage of F being a subalgebra and H being ﬁnite.
Revised: To prove this, we take advantage of F being a subalgebra and H being ﬁnite.

Original: Let H =
Revised: Let H =

Original: { g 1 , . .
Revised: { g 1 , . .

Original: . , g h } and deﬁne,
Revised: . , g h } and deﬁne,

Original: [Equation]
Revised: [Equation]

Original: As F is a subalgebra of C ( X, R p ) , A is a subalgebra of R p (cid:48) with p (cid:48)
Revised: As F is a subalgebra of C ( X, R p ) , A is a subalgebra of R p (cid:48) with p (cid:48)

Original: =
Revised: =

Original: p ( h + 1) .
Revised: p ( h + 1) .

Original: By Lem.
Revised: By Lem.

Original: [Equation]
Revised: [Equation]

Original: where I and J can be chosen to be,
Revised: where I and J can be chosen to be,

Original: [Equation]
Revised: [Equation]

Original: As I is an equivalence relation, an element of A is uniquely deﬁned by its coordinates on equivalence classes of I .
Revised: As I is an equivalence relation, an element of A is uniquely deﬁned by its coordinates on equivalence classes of I .

Original: Therefore, one can choose z ∈
Revised: Therefore, one can choose z ∈

Original: A such that z
Revised: A such that z

Original: i
Revised: i

Original: =
Revised: =

Original: z j ⇐⇒ ( i, j ) ∈
Revised: z j ⇐⇒ ( i, j ) ∈

Original: I .
Revised: I .

Original: By deﬁnition of A , there exists f ∗ ∈ F such that ( f ∗ ( g 1 · x ) , . . .
Revised: By deﬁnition of A , there exists f ∗ ∈ F such that ( f ∗ ( g 1 · x ) , . . .

Original: , f ∗ ( g h · x ) , f ∗ ( y ))
Revised: , f ∗ ( g h · x ) , f ∗ ( y ))

Original: =
Revised: =

Original: z .
Revised: z .

Original: We now check that f ∗ is indeed appropriate.
Revised: We now check that f ∗ is indeed appropriate.

Original: Take l ∈ { 1 , . .
Revised: Take l ∈ { 1 , . .

Original: , h } , we want to show that f ∗ ( g l · x ) (cid:54) =
Revised: , h } , we want to show that f ∗ ( g l · x ) (cid:54) =

Original: f ∗ ( y ) .
Revised: f ∗ ( y ) .

Original: By assumption, there exists f ∈ F such that f ( g l · x ) (cid:54) =
Revised: By assumption, there exists f ∈ F such that f ( g l · x ) (cid:54) =

Original: f ( y ) , i.e. there exists i ∈ { 1 , . .
Revised: f ( y ) , i.e. there exists i ∈ { 1 , . .

Original: . , p } such that f ( g l · x ) i (cid:54) =
Revised: . , p } such that f ( g l · x ) i (cid:54) =

Original: f ( y ) i .
Revised: f ( y ) i .

Original: Therefore, (( l − 1) p + i, hp + i ) cannot be in I so that z ( l − 1)
Revised: Therefore, (( l − 1) p + i, hp + i ) cannot be in I so that z ( l − 1)

Original: p
Revised: p

Original: + i (cid:54) =
Revised: + i (cid:54) =

Original: z hp +1 , i.e. f ∗ ( g l · x ) i (cid:54) =
Revised: z hp +1 , i.e. f ∗ ( g l · x ) i (cid:54) =

Original: f ∗ ( y ) i .
Revised: f ∗ ( y ) i .

Original: We now prove our main abstract theorem.
Revised: We now prove our main abstract theorem.

Original: Theorem 22.
Revised: Theorem 22.

Original: Let X be a compact space, Y
Revised: Let X be a compact space, Y

Original: = R p for some p , G be a ﬁnite group acting (continuously) on X and Y and F ⊆ C E ( X, Y ) a (non-empty) set of equivariant functions.
Revised: = R p for some p , G be a ﬁnite group acting (continuously) on X and Y and F ⊆ C E ( X, Y ) a (non-empty) set of equivariant functions.

Original: Denote by π : Y −→ Y/G the canonical projection on the quotient space Y/G .
Revised: Denote by π : Y −→ Y/G the canonical projection on the quotient space Y/G .

Original: Consider the following assumptions,
Revised: Consider the following assumptions,

Original: F is a sub-algebra of C ( X, Y ) and the constant function 1 is in F . 2.
Revised: F is a sub-algebra of C ( X, Y ) and the constant function 1 is in F . 2.

Original: The set of functions F scal ⊆
Revised: The set of functions F scal ⊆

Original: C ( X, R ) deﬁned by,
Revised: C ( X, R ) deﬁned by,

Original: [Equation]
Revised: [Equation]

Original: satisfy,
Revised: satisfy,

Original: [Equation]
Revised: [Equation]

Original: Then the closure of F (for the topology of uniform convergence) is,
Revised: Then the closure of F (for the topology of uniform convergence) is,

Original: [Equation]
Revised: [Equation]

Original: where F ( x ) = { f ( x ) , f ∈ F} .
Revised: where F ( x ) = { f ( x ) , f ∈ F} .

Original: Moreover, if I ( x ) = { ( i, j ) ∈ [ p ] 2 : ∀ y ∈ F ( x ) , y i =
Revised: Moreover, if I ( x ) = { ( i, j ) ∈ [ p ] 2 : ∀ y ∈ F ( x ) , y i =

Original: y j } , then we have:
Revised: y j } , then we have:

Original: [Equation]
Revised: [Equation]

Original: Proof of Thm. 22.
Revised: Proof of Thm. 22.

Original: By Lem.
Revised: By Lem.

Original: 28, the second assumption of Prop. 24 is also satisﬁed.
Revised: 28, the second assumption of Prop. 24 is also satisﬁed.

Original: To get the conclusion of Thm. 22, note that F ( x ) is now a linear subspace of a ﬁnite-dimensional vector space and therefore it is closed.
Revised: To get the conclusion of Thm. 22, note that F ( x ) is now a linear subspace of a ﬁnite-dimensional vector space and therefore it is closed.

Original: Thus, F ( x ) = F ( x ) which is a subalgebra.
Revised: Thus, F ( x ) = F ( x ) which is a subalgebra.

Original: Applying Lem.
Revised: Applying Lem.

Original: 27 to F ( x ) and noting that, necessarily J = ∅ as 1 ∈ F ( x ) by assumption, gives the result of Thm. 22.
Revised: 27 to F ( x ) and noting that, necessarily J = ∅ as 1 ∈ F ( x ) by assumption, gives the result of Thm. 22.

Original: D. P RACTICAL REDUCTIONS
Revised: D. P RACTICAL REDUCTIONS

Original: Though the results we proved above were formulated using classic hypotheses, such as requiring F to be a subalgebra, we can give much more compact versions for our setting.
Revised: Though the results we proved above were formulated using classic hypotheses, such as requiring F to be a subalgebra, we can give much more compact versions for our setting.

Original: We also reduce the assumption that ρ ( F scal ) ⊆
Revised: We also reduce the assumption that ρ ( F scal ) ⊆

Original: ρ ( π ◦ F ) to a more practical one.
Revised: ρ ( π ◦ F ) to a more practical one.

Original: We start with the invariant case.
Revised: We start with the invariant case.

Original: Corollary 29.
Revised: Corollary 29.

Original: Let X be a compact space, Y = F = R p be some ﬁnite-dimensional vector space,
Revised: Let X be a compact space, Y = F = R p be some ﬁnite-dimensional vector space,

Original: G be a ﬁnite group acting (continuously) on X and F ⊆ C
Revised: G be a ﬁnite group acting (continuously) on X and F ⊆ C

Original: I ( X, Y ) a (non-empty) set of invariant functions.
Revised: I ( X, Y ) a (non-empty) set of invariant functions.

Original: Assume that, for any h ∈ C ( F 2 , F ) and f, g ∈ F ,
Revised: Assume that, for any h ∈ C ( F 2 , F ) and f, g ∈ F ,

Original: [Equation]
Revised: [Equation]

Original: Then the closure of F is,
Revised: Then the closure of F is,

Original: [Equation]
Revised: [Equation]

Original: Proof.
Revised: Proof.

Original: We wish to apply Thm.
Revised: We wish to apply Thm.

Original: but for this we need G to act on Y .
Revised: but for this we need G to act on Y .

Original: Deﬁne a (trivial) action of G on Y by,
Revised: Deﬁne a (trivial) action of G on Y by,

Original: [Equation]
Revised: [Equation]

Original: With this action on Y , C E ( X, Y ) =
Revised: With this action on Y , C E ( X, Y ) =

Original: C I ( X, Y ) .
Revised: C I ( X, Y ) .

Original: Moreover, Y/G = Y , π : Y → Y/G is the identity
Revised: Moreover, Y/G = Y , π : Y → Y/G is the identity

Original: so that ρ ( π ◦ F ) =
Revised: so that ρ ( π ◦ F ) =

Original: ρ ( F ) .
Revised: ρ ( F ) .

Original: Our assumption clearly ensure that F is indeed a subalgebra and contains the constant function (cid:49) .
Revised: Our assumption clearly ensure that F is indeed a subalgebra and contains the constant function (cid:49) .

Original: All that is left to show to apply Thm.
Revised: All that is left to show to apply Thm.

Original: 22 is that the set of functions F scal ⊆
Revised: 22 is that the set of functions F scal ⊆

Original: C ( X, R ) deﬁned by,
Revised: C ( X, R ) deﬁned by,

Original: [Equation]
Revised: [Equation]

Original: satisﬁes,
Revised: satisﬁes,

Original: [Equation]
Revised: [Equation]

Original: Take ( x, y ) / ∈ ρ ( F ) and we show that ( x, y ) / ∈
Revised: Take ( x, y ) / ∈ ρ ( F ) and we show that ( x, y ) / ∈

Original: ρ ( F scal ) .
Revised: ρ ( F scal ) .

Original: Indeed, by deﬁnition there exists f ∈ F , i ∈ { 1 , . .
Revised: Indeed, by deﬁnition there exists f ∈ F , i ∈ { 1 , . .

Original: . , p } such that f ( x ) i (cid:54) =
Revised: . , p } such that f ( x ) i (cid:54) =

Original: f ( y ) i .
Revised: f ( y ) i .

Original: Let l ∈ C ( F , R ) deﬁned by l ( z )
Revised: Let l ∈ C ( F , R ) deﬁned by l ( z )

Original: =
Revised: =

Original: z
Revised: z

Original: i and h ∈ C ( F , F ) deﬁned by, for z ∈ F , h ( z ) =
Revised: i and h ∈ C ( F , F ) deﬁned by, for z ∈ F , h ( z ) =

Original: ( l ( z ) , . . .
Revised: ( l ( z ) , . . .

Original: , l ( z ))
Revised: , l ( z ))

Original: =
Revised: =

Original: l ( z ) (cid:49) .
Revised: l ( z ) (cid:49) .

Original: Then, by assumption h ◦ f ∈ F .
Revised: Then, by assumption h ◦ f ∈ F .

Original: But h ◦ f is ( l ◦ f ) 1 with l ◦ f ∈ C ( X, R ) , so that, by deﬁnition, l ◦ f ∈ F scal .
Revised: But h ◦ f is ( l ◦ f ) 1 with l ◦ f ∈ C ( X, R ) , so that, by deﬁnition, l ◦ f ∈ F scal .

Original: Moreover, l ◦ f ( x ) (cid:54) =
Revised: Moreover, l ◦ f ( x ) (cid:54) =

Original: l ◦ f ( y ) .
Revised: l ◦ f ( y ) .

Original: Therefore, ( x, y ) / ∈ ρ ( F scal ) .
Revised: Therefore, ( x, y ) / ∈ ρ ( F scal ) .

Original: Therefore, we can apply Thm.
Revised: Therefore, we can apply Thm.

Original: We get that,
Revised: We get that,

Original: [Equation]
Revised: [Equation]

Original: , and
Revised: , and

Original: F
Revised: F

Original: [Equation]
Revised: [Equation]

Original: with I ( x ) given by,
Revised: with I ( x ) given by,

Original: [Equation]
Revised: [Equation]

Original: To conclude the proof, we now show that I ( x ) =
Revised: To conclude the proof, we now show that I ( x ) =

Original: { ( i, i ) : i ∈ { 1 , . . .
Revised: { ( i, i ) : i ∈ { 1 , . . .

Original: , p }} , which will imply that
Revised: , p }} , which will imply that

Original: F ( x ) =
Revised: F ( x ) =

Original: R p
Revised: R p

Original: = Y .
Revised: = Y .

Original: Indeed, the constant function z (cid:55)→ (1 , 2 , . .
Revised: Indeed, the constant function z (cid:55)→ (1 , 2 , . .

Original: . , p ) is in F by assumption.
Revised: . , p ) is in F by assumption.

Original: Therefore, I ( x ) is reduced to { ( i, i ) : i ∈ { 1 , . . .
Revised: Therefore, I ( x ) is reduced to { ( i, i ) : i ∈ { 1 , . . .

Original: , p }} .
Revised: , p }} .

Original: In our previous version of our approximation result for node embedding, we did not allow features in the output as it would have made the statement and the proof a bit convoluted.
Revised: In our previous version of our approximation result for node embedding, we did not allow features in the output as it would have made the statement and the proof a bit convoluted.

Original: With this new assumption, this is much easier.
Revised: With this new assumption, this is much easier.

Original: Corollary 30.
Revised: Corollary 30.

Original: Let X be a compact space, Y = F n , with F = R p and G = S n the permutation group, acting (continuously) on X and acting on F n by, for σ ∈ S n , x ∈ F n ,
Revised: Let X be a compact space, Y = F n , with F = R p and G = S n the permutation group, acting (continuously) on X and acting on F n by, for σ ∈ S n , x ∈ F n ,

Original: [Equation]
Revised: [Equation]

Original: Let F ⊆ C E ( X, F n ) be a (non-empty) set of equivariant functions.
Revised: Let F ⊆ C E ( X, F n ) be a (non-empty) set of equivariant functions.

Original: Consider the following assumptions,
Revised: Consider the following assumptions,

Original: For any h ∈ C ( F 2 , F ) , f, g ∈ F , x (cid:55)→ ( h ( f ( x ) 1 , g ( x ) 1 ) , . . .
Revised: For any h ∈ C ( F 2 , F ) , f, g ∈ F , x (cid:55)→ ( h ( f ( x ) 1 , g ( x ) 1 ) , . . .

Original: , h ( f ( x ) n , g ( x ) n ))
Revised: , h ( f ( x ) n , g ( x ) n ))

Original: ∈ F .
Revised: ∈ F .

Original: If f ∈ F , x (cid:55)→ (cid:32) n (cid:88) i =
Revised: If f ∈ F , x (cid:55)→ (cid:32) n (cid:88) i =

Original: f ( x ) i , n (cid:88) i
Revised: f ( x ) i , n (cid:88) i

Original: = f
Revised: = f

Original: ( x ) i . .
Revised: ( x ) i . .

Original: . , n (cid:88)
Revised: . , n (cid:88)

Original: i =
Revised: i =

Original: f ( x ) i (cid:33)
Revised: f ( x ) i (cid:33)

Original: [Equation]
Revised: [Equation]

Original: Then the closure of F (for the topology of uniform convergence) is,
Revised: Then the closure of F (for the topology of uniform convergence) is,

Original: [Equation]
Revised: [Equation]

Original: For this we need a handy lemma, whose proof relies on a result about multi-symmetric polynomials from Maron et al. (2019a).
Revised: For this we need a handy lemma, whose proof relies on a result about multi-symmetric polynomials from Maron et al. (2019a).

Original: Lemma 31.
Revised: Lemma 31.

Original: Let F = R p be some ﬁnite-dimensional space.
Revised: Let F = R p be some ﬁnite-dimensional space.

Original: Take x 1 , . . .
Revised: Take x 1 , . . .

Original: , x n , y 1 , . . .
Revised: , x n , y 1 , . . .

Original: , y n ∈ F such that, for any σ ∈ S n , ( x 1 , . . .
Revised: , y n ∈ F such that, for any σ ∈ S n , ( x 1 , . . .

Original: , x n ) (cid:54) =
Revised: , x n ) (cid:54) =

Original: ( y σ (1) , . . .
Revised: ( y σ (1) , . . .

Original: y σ ( n ) )
Revised: y σ ( n ) )

Original: Then, there exists h ∈ C ( F , R ) such that, n (cid:88)
Revised: Then, there exists h ∈ C ( F , R ) such that, n (cid:88)

Original: i =
Revised: i =

Original: h ( x i ) (cid:54) =
Revised: h ( x i ) (cid:54) =

Original: n (cid:88)
Revised: n (cid:88)

Original: i =
Revised: i =

Original: h ( y i ) .
Revised: h ( y i ) .

Original: Moreover, h can be written as h ( x ) =
Revised: Moreover, h can be written as h ( x ) =

Original: h 1 ( x 1 )
Revised: h 1 ( x 1 )

Original: h 2 ( x 2 ) . . .
Revised: h 2 ( x 2 ) . . .

Original: h p ( x p ) for any x ∈ F with h 1 , . . .
Revised: h p ( x p ) for any x ∈ F with h 1 , . . .

Original: , h p ∈
Revised: , h p ∈

Original: C ( R , R ) .
Revised: C ( R , R ) .

Original: Proof.
Revised: Proof.

Original: By Maron et al.
Revised: By Maron et al.

Original: (2019a, Prop. 1), there exists α 1 , . .
Revised: (2019a, Prop. 1), there exists α 1 , . .

Original: . , α p non-negative integers such that (cid:80)
Revised: . , α p non-negative integers such that (cid:80)

Original: [Equation]
Revised: [Equation]

Original: Taking h : x (cid:55)→ x
Revised: Taking h : x (cid:55)→ x

Original: α 1 i 1 x α 2 i 2 . . .
Revised: α 1 i 1 x α 2 i 2 . . .

Original: x α p ip yields the result.
Revised: x α p ip yields the result.

Original: Proof of Cor.
Revised: Proof of Cor.

Original: We want to apply Thm. 22.
Revised: We want to apply Thm. 22.

Original: With our ﬁrst assumption, the ﬁrst assumption of
Revised: With our ﬁrst assumption, the ﬁrst assumption of

Original: Thm. 22 is easily veriﬁed.
Revised: Thm. 22 is easily veriﬁed.

Original: We now focus on the second one.
Revised: We now focus on the second one.

Original: As in the statement of Thm. 22, deﬁne F scal ⊆
Revised: As in the statement of Thm. 22, deﬁne F scal ⊆

Original: C ( X, R ) by,
Revised: C ( X, R ) by,

Original: F scal = { f ∈ C ( X, R ) : f 1 ∈ F} .
Revised: F scal = { f ∈ C ( X, R ) : f 1 ∈ F} .

Original: We have to show that ρ ( F scal ) ⊆
Revised: We have to show that ρ ( F scal ) ⊆

Original: ρ ( π ◦ F ) .
Revised: ρ ( π ◦ F ) .

Original: For this, take x, y / ∈ ρ ( π ◦ F ) .
Revised: For this, take x, y / ∈ ρ ( π ◦ F ) .

Original: There exists f ∈ F such that for any σ ∈ S n , σ · f ( x ) (cid:54) =
Revised: There exists f ∈ F such that for any σ ∈ S n , σ · f ( x ) (cid:54) =

Original: f ( y ) .
Revised: f ( y ) .

Original: We have to ﬁnd l ∈ F scal such that l ( x ) (cid:54) =
Revised: We have to ﬁnd l ∈ F scal such that l ( x ) (cid:54) =

Original: l ( y ) .
Revised: l ( y ) .

Original: In other words, from a function in F n which discriminates between x and y we have to build a function in R .
Revised: In other words, from a function in F n which discriminates between x and y we have to build a function in R .

Original: First, we exhibit a function which discriminates between x and y .
Revised: First, we exhibit a function which discriminates between x and y .

Original: Apply Lem.
Revised: Apply Lem.

Original: 31 to the vectors f ( x ) and f ( y ) : there exists h 0 ∈ C ( F , R ) such that (cid:80) ni
Revised: 31 to the vectors f ( x ) and f ( y ) : there exists h 0 ∈ C ( F , R ) such that (cid:80) ni

Original: =1 h 

Revised: =1 h

Original: ( f ( x i ))
Revised: ( f ( x i ))

Original: (cid:54) =
Revised: (cid:54) =

Original: (cid:80) ni
Revised: (cid:80) ni

Original: =1 h 

Revised: =1 h

Original: ( f ( y i )) .
Revised: ( f ( y i )) .

Original: To ﬁt the assumptions, we build h ∈ C ( F , F ) from h 0 by h : x ∈ F (cid:55)→ ( h 0 ( x ) , . .
Revised: To ﬁt the assumptions, we build h ∈ C ( F , F ) from h 0 by h : x ∈ F (cid:55)→ ( h 0 ( x ) , . .

Original: , h 0 ( x )) ∈ F .
Revised: , h 0 ( x )) ∈ F .

Original: Take g ∈ C ( F , F ) such that g ( z ) =
Revised: Take g ∈ C ( F , F ) such that g ( z ) =

Original: ( z 1 , . . .
Revised: ( z 1 , . . .

Original: , z 1 ) for any z ∈ F and l ∈ C
Revised: , z 1 ) for any z ∈ F and l ∈ C

Original: ( X, R ) deﬁned by, for w ∈ X , l ( w )
Revised: ( X, R ) deﬁned by, for w ∈ X , l ( w )

Original: = (cid:80) ni
Revised: = (cid:80) ni

Original: =1 h 

Revised: =1 h

Original: ( f ( w i )) .
Revised: ( f ( w i )) .

Original: Then, l ( x ) (cid:54) =
Revised: Then, l ( x ) (cid:54) =

Original: l ( y ) .
Revised: l ( y ) .

Original: All we have to do is show that l ∈ F scal , i.e., l 1 ∈ F .
Revised: All we have to do is show that l ∈ F scal , i.e., l 1 ∈ F .

Original: This is where the two assumptions we made come into play.
Revised: This is where the two assumptions we made come into play.

Original: Indeed, the ﬁrst one implies that h ◦ f ∈ F and the second gives
Revised: Indeed, the ﬁrst one implies that h ◦ f ∈ F and the second gives

Original: [Equation]
Revised: [Equation]

Original: Finally, the ﬁrst assumption ensure that,
Revised: Finally, the ﬁrst assumption ensure that,

Original: [Equation]
Revised: [Equation]

Original: But this last function is none other than l 1 , which shows that l ∈ F scal as required.
Revised: But this last function is none other than l 1 , which shows that l ∈ F scal as required.

Original: We have successfully veriﬁed the hypothesis of Thm. 22.
Revised: We have successfully veriﬁed the hypothesis of Thm. 22.