Can Hybrid Geometric Scattering Networks Help Solve the Maximum Clique Problem?
Anonymous Author(s)
AffiliationAddress email
Abstract
We propose a geometric scattering-based graph neural network (GNN) for approximating solutions of the NP-hard maximum clique (MC) problem.
We construct aloss function with two terms, one which encourages the network to find a large setof nodes and the other which acts as a surrogate for the constraint that the nodesform a clique.
We then use this loss to train an efficient GNN architecture thatoutputs a vector representing the probability for each node to be part of the MC andapply a rule-based decoder to make our final prediction.
The incorporation of thescattering transform alleviates the so-called oversmoothing problem that is oftenencountered in GNNs and would degrade the performance of our proposed setup.
Our empirical results demonstrate that our method outperforms representative GNNbaselines in terms of solution accuracy and inference speed as well as conventionalsolvers like GUROBI with limited time budgets.
Furthermore, our scattering modelbuilds efficient representation and only consists of ∼ 0.1% of parameter counts ofprevious GNN baseline model.
Introduction
The success of Graph Neural Networks (GNNs) for a variety of machine learning tasks [Scarselliet al., 2008, Gori et al., 2005, Kipf and Welling, 2016, Gilmer et al., 2017] has sparked interests inusing GNNs to solve graph combinatorial optimization (CO) problems [Li et al., 2018a, Karalias and
Loukas, 2020].
For example, Joshi et al.
[2019] introduces a GNN-based method for approximatelysolving the travelling salesman problem (TSP), while Karalias and Loukas [2020] approximatesolutions of the maximum clique (MC) problem.
Solving such problems in an end-to-end fashionvia GNNs is challenging for several reasons.
First of all, many CO problems are provably either
NP-Hard or NP-complete.
Therefore, learning CO solvers in a supervised or semi-supervised fashioncan be computationally infeasible since the cost of generating the truth labels grows exponentiallywith the problem size.
Second, the solution to the CO problems must satisfy a number of constraintsor limitations (e.g. belonging to a clique).
Although these constraints can be imposed during trainingtime, there is still no guarantee that these constraints are still satisfied at test time.
Furthermore,solving graph combinatorial problems largely depends on the expressive power of GNNs.
Many
GNNs aggregate information via local averaging, which can be interpreted as a smoothing operation.
This degrades the expressive power of GNNs and results in the so-called oversmoothing problem [Liet al., 2018b] where neighboring nodes have similar representations and are difficult to distinguishfrom each other.
This is problematic if a node that is not in the solution set borders many points thatare in the solution set.
The purpose of this paper is to use a geometric scattering-based GNN to fast approximate the solutionof the maximum clique (MC) problem, that is to find the largest complete subgraph contained withina large graph G .
Finding maximum clique has wide application in pattern recognition and knowledge
[Figure]
discovery. For instance, finding the maximum clique of the related connection graphs can be used tohandle object recognition tasks[Horaud and Skordas, 1989]. It is also tightly related to the biologicalfield as well, such as help solving genome-scale elucidation of biological networks[Zhang et al. The use of geometric scattering, rather than a more traditional GNN is motivated by recentwork [Wenkel et al., 2022] showing that the geometric scattering transform can help overcome theoversmoothing problem via the use of band-pass wavelet filters in conjunction with GCN-type filters.
This is particularly important in the context of the MC problem because it is critical to distinguish apoint which connects to many members of the clique from an actual member of the clique.
As weshall show, our method, which utilizes band-pass wavelet filters is able to better detect the borderbetween the MC and the rest of the graph.
This parallels the traditional use of wavelets as edgedetectors in image processing[Grossmann, 1988].
Our method uses a two-phase strategy shown in Figure 1.
We first use an unsupervised learningmodel to generate an efficient representation of how likely each node is to be part of the maximumclique.
Then, this representations is fed into a constraint-preserving decoder to generate the maximumclique.
Ideally, given a graph G with n nodes, we would build a representation p ∈ [0 , 1] n where thenodes in the maximum clique are assigned the value 1 (True) while all other nodes are assigned 0(False), so that we can ‘cut’ the MC from the graph via O ( n ) operations.
In practice, we will build arepresentation which assigns large probabilities to the maximum clique nodes and small probabilitiesto the other nodes.
Figure 2 (a) presents a situation where oversmoothing does not occur.
The signalp is much larger on the nodes (1 , 2 , 3 , 4 , 6) , which form the maximum clique, than it is on the othernodes (5 , 7 , 8) .
Therefore, the decoder can easily find the MC by, e.g., setting a threshold.
Figure2 (b), on the other hand, illustrates a situation in which oversmoothing does occur and p has lessvariability.
Indeed, there are no blue colored nodes at all.
In this setting, it is nearly impossible todistinguish node 7 , which does not belong to the maximum clique, from the nodes 1 , 6 , and thereforeextracting the MC from p is difficult if not impossible.
[Figure]
Background and
Work2.1 Graph Signal Processing
Consider a graph G =
( V, E ) with a set of nodes (or vertices) V :=
{ v 1 , . .
. , v n } and a set ofundirected edges E ⊂ V × V .
For a function x : V → R defined on V , we will, in minor abuse ofnotation identify x with the vector where x [ i ] =
x ( v i ) .
We let W ∈ R n × n denote the adjacencymatrix of G and define the symmetric normalized grah Laplacian by L :=
I n − D − 1 / 2 W D 1 / 2 ,where d i := (cid:80) nj =1 W [ v i , v j ] is the degree of the node v i and D :=
diag( d 1 , ..., d n ) ∈ R n × n isthe degree matrix.
We will let q i
and λ i denote the (normalized) eigenvectors and eigenvaluesof L with L q i
= λ
i q i , and make use of the eigendecomposition L = Q Λ Q ⊤ , where Q is theorthogonal matrix whose i -th column is the normalized eigenvector q i , and Λ := diag( λ 1 , ..., λ n ) .
Notably, the eigenvectors q 1 , . . .
q n can be seen as a generalization of Fourier modes in the contextof graph domains with the corresponding eigenvalues representing the increasing frequencies 0 ≤λ 1 ≤ · · · ≤ λ n ≤ 2 .
This link may be established by viewing frequency as a measure of variation,in particular the variation of (degree-normalized)
Fourier modes across edges is λ i =
q ⊤ i
L q i = 75 (cid:80) { u,v }∈ E (˜ q i [ u ] − ˜
q i [ v ]) 2 for ˜ q i :=
D − 1 / 2 q i .
The graph Fourier transform of a signal vector x isthen defined as by ˆ x [ i ] = ⟨ x , q i ⟩ and the inverse Fourier is given by x = (cid:80) n i =
ˆ x [ i ] q i .
Compactly,we write ˆ x = Q ⊤ x and x = Q
ˆ x .
In the Euclidean setting, it is known that convolution in the spatial domain corresponds to multiplication in the Fourier domain.
This fact has motivated work such as Shuman et al.
[2016], whichdefines the convolution of a signal x with a filter g to be the unique vector verifying ( (cid:91) g ⋆ x )
[ i ]
=ˆ g
[ i ] ˆ x
[ i ] , which implies that g ⋆ x = (cid:80) ni =1 ˆ g [ i ] ˆ x [ i ] q i = (cid:80) ni
{+=
 ˆ g+}
[ i ] ⟨ q i , x ⟩ q i = Q (cid:98) G Q ⊤ x , where(cid:98) G :=
diag( ˆ g ) =
diag( ˆ g [1] , . .
, ˆ g [ n ]) .
When incorporating this notion of convolution into a graphneural network, a common choice [e.g., Defferrard et al., 2016] is to require the coefficients ˆ g [ i ]to be polynomials of the eigenvalues
λ
i , i ∈ [ n ] , i.e., ˆ g [ i ] := (cid:80) k γ k λ ki in which case we have(cid:98) G = (cid:80) k γ k Λ k .
This allows one to implement convolution in the spatial domain by verifying thatg
⋆ x = (cid:80) k γ k L k x .
In particular, the entire method may be implemented without the need todiagonalize a matrix which is extremely expensive for large graphs.
2.2 Graph Convolutional Networks
In Kipf and Welling [2016], the authors set ˆ g [ i ] :
= θ (2 − λ i ) , where θ is a real number, which yields
[Equation]
As the eigenvalues of the above convolutional filter lie in [0,2] and could lead to vanishingor exploding gradients, Kipf and Welling [2016] apply a renormalization trick which replaces
I n + D − 1 / 2 W D − 1 / 2 with A :
= ( D + I ) − 1 / 2 ( W + I ) ( D + I ) − 1 / 2 .
The layer-wise propagationrule of GCN is then defined by x ℓj = σ (cid:0) (cid:80) N ℓ − 1 i =1 θ ℓij A x ℓ − 1 i (cid:1) , where x ℓ − 1 i ∈ R n is the i -th featurevector in layer ℓ − 1 , θ ℓij is a trainable parameter and σ ( . ) is a nonlinear activation function, x ℓj is thej -th feature activation vector in layer ℓ .
We can further write this compactly in matrix notation, thisgives
[Equation]
where Θ ℓ ∈ R N ℓ − 1 × N ℓ is the weight-matrix of layer ℓ and X ℓ ∈ R n × N ℓ contains the activationsoutput by layer ℓ .
The GCN model updates node features at every node by averaging over the nodefeatures of the node itself and its neighbors, which enforces similarity throughout node neighborhoods.
This makes nodes increasingly difficult to discriminate for ‘deeper’ models and is widely referredto as the oversmoothing problem [Li et al., 2018b].
From a graph spectral theory perspective,oversmoothing is related to low-pass filtering as the filter in Eq.
1 puts larger weights on the lowfrequency spectrum (as 0 ≤ λ i ≤ 2 ).
This motivates the idea of using GCN-type low-pass filters inconjunction with band-pass filters, that can be implemented, for example, using graph scattering [Minet al., 2020], which we discuss in the following.
2.3 Graph Scattering
The graph scattering transform [Gama et al., 2019, Zou and Lerman, 2019, Gao et al., 2019] is awavelet-based model for machine learning on graphs.
Unlike the one-hop localized low-pass filtersused in GCN, which aim to promote smoothness between neighboring nodes, these wavelets areband-pass filters, which in turn incorporate long-range dependencies through the large spatial supportof the used aggregations.
The scattering transform is based upon raising the lazy random walk matrix
P := 12 (cid:0) I n + W D − 1 (cid:1) to different powers in order to capture the diffusion geometry of the graph G at various time scales.
In particular, subtracting such powers allows us to detect changes in thesediffusion geometries.
Following the lead of Coifman and Maggioni [2006], for k ∈ N 0 , we define awavelet matrix Ψ k ∈ R n × n at scale 2 k by
[Equation]
Intuitively, at every node, these diffusion wavelets can be seen as a comparison operation thatcalculates the difference of the averaged features of two neighborhoods of different sizes (namelysizes 2 k − 1 and 2 k ).
The geometric scattering transform is constructed through an alternating cascadeon wavelet filterings and pointwise nonlinearities.
Towards this end, we let σ : R → R be a nonlinearfunction which is nonexpansive in the sense that | σ ( x ) − σ ( y ) | ≤ | x − y | , and for a scattering pathp :
= ( k 1 , . . .
, k m ) , we define
[Equation]
Model
Our method is centered around three main components, (i) a hybrid scattering-GCN model M (Sec. 3.1-3.3) that transforms a small set of simple node-level statistics represented by the matrix
X ∈ R n × d to a probability vector p ∈ [0 , 1] n representing the probabilities that each node is part of the maximum clique, (ii) an easy-to-optimize unsupervised loss function L (Sec. 3.4) that is small for “good” probability vectors p ⋆ ; and (iii) a rule-based decoder D (Sec. 3.5) that maps the probability vector to the maximum clique C ⋆ ⊂ V of G .
[Equation]
3.1 Embedding Module
The input of our model is a node feature matrix X ∈ R n × d , containing d features for each node.
In our experiments, we set d = 3 and let the features be the eccentricity, the clustering coefficient, and the logarithm of the degree of each vertex.
The encoder transforms the node features to d h -dimensional embeddings H 0 using a multi-layer perceptrone (MLP) m
emb : R d → R d h , i.e.,
H 0 :
= m emb ( X ) .
We will use H 0 as the input to the diffusion model introduced in the next subsection.
3.2 Diffusion Module
The aggregation module consists of a cascade of K ∈ N aggregation (or diffusion) layers withoperations that are chosen for each node via an attention mechanism.
Our network is based uponthe framework introduced in Wenkel et al.
However, our method differs from Wenkel et al.[2022] by storing every intermediate node representation H ℓ , 0 ≤ ℓ ≤ K , in a list of readouts .
In each layer ℓ , every node has access to node representations from set a of filters F that containsa selection of low-pass and band-pass filters.
Similar to Wenkel et al.
[2022], our band-pass filtersf low ,r are modified GCN filters that have the form
[Equation]
(5) and are parameterized by the power r ≥ 1 of the matrix A .
Similarly, we define the scattering filterf band ,k of order k according to
[Equation]
Next, we want to obtain data-driven scores s f ( v ) that determine the importance of filter f ∈ F fornode v ∈ V .
We set H ℓf := f
( H ℓ − 1 ) and calculate the scores via an attention mechanism, setting
[Equation]
where ∥ denotes the concatenation operation. The attention mechanism, at every node, calculates thedot product of the learnable attention vector a ℓ ∈ R 2 d h with the filtered node representation of thenode concatenated to its previous representation.
We normalize the scores across the filters usingthe softmax function, i.e., α f ( v ) =
softmax F ( s f ( v )) , and use the corresponding attention vectorsα ℓf ∈ R n , f ∈ F to update node representations via
[Equation]
where ⊙ denotes the element-wise multiplication.
Notably, the softmax function enforces that everynode is likely to focus on one single filter.
We note that the method is compatible with multi-headattention but omit further details as it is not necessary for the present experiments.
Finally, wetransform the aggregated node representations via an MLP m ℓ : R d h → R d h , i.e., H ℓ := m ℓ ( H ℓ agg ) .
3.3 Output Module
In the output module, we combine the information from the readouts H ℓ , 0 ≤ ℓ ≤ K .
We first concatenate the readouts horizontally to
[Equation]
where H cat ∈ R d h ( K +1) and then transform H cat to a vector h ∈ R n using an MLP, i.e., h :
=m out ( H cat ) .
Finally, to obtain a probability vector, we apply min-max normalization, i.e., p :=(
h − min( h ) · 1 n ) / (max( h ) − min( h )) , and interpret p [ i ] as the probability that the i -th node ispart of the maximum clique.
3.4 Training Loss
We now derive an unsupervised training loss for the maximum clique problem.
Letting Ω denotethe set of cliques, we note that the task of finding the maximum clique is equivalent to identifyingthe clique C ∈ Ω that contains the most edges, i.e., maximizes the objective function L ⋆ ( C ) :
= 155 (cid:80) u,v ∈ C w u,v w.r.t to nodes u and v , where w u,v = 1 if node u and v are connected otherwise 0.
Our approach relies on producing a vector p such that p v ≈ 1 if v is in the maximum clique andp v ≈ 0 otherwise.
We consider the vector x ∈ R n where x v = 1 if v is in the maximum clique andx v = [-
-]0 otherwise.
We then model x v as a Bernoulli random variable where P ( x v = 1) =
p v .
Ourgoal then becomes to maximize the expectation
[Equation]
Maximizing (9) encourages p to be large on the support of the largest clique.
However, we alsowant the support of p to be concentrated within a clique.
This naturally motivates the constrainedoptimization problem of minimizing
[Equation]
subject to: supp
[Equation]
p ) is contained in a clique
[Equation]
However, the constraint that p is contained in a clique is hard to enforce directly.
In order toconstruct a surrogate, we consider the complement graph G =
( V , E ) .
The vertices of G aretaken to be the same as G , i.e., V = V , while the edges are defined by the rule that for u ̸ = vthere is an edge between u and v in G if and only if there is not an edge between them in G , i.e.,
E = { ( u, v ) ∈
V × V : u ̸ = v }\ E .
We denote the adjacency matrix of G by W = ( w u,v ) u,v ∈ V ,and note thatw u,v := (cid:26) 1 if ( u, v ) / ∈ E and u ̸ = v, 0 otherwise.
In particular, W has zeros on the diagonal and can be computed from W via W = 1 n × n − ( I + W ) .
The following Lemma indicates that a second loss term L 2 ( p ) :
= p ⊤ W p can be used to ensure thatmass is primarily concentrated in a set of nodes that form a clique.
Lemma 1. Consider a graph G =
( V, E ) , a signal p ≥ 0 and define supp( p ) :
= { v ∈ V : p v > 0 } .
Then, L 2 ( p ) =
p ⊤ W p =
if and only if there exists a clique C ⊂ V such that the support of p iscontained in C .
Proof.
We first assume p ⊤ W p = 0 and suppose for the sake of a contradiction that there existsno clique C ⊂ V with supp( p ) ⊂ C .
Then, there exist at least two nodes u 0 , v 0 ∈ supp( p ) with{ u 0 , v 0 } ̸∈ E .
Now, p ⊤ W p = (cid:80) { u,v }̸∈ E p u
p v ≥ p u 0 p v 0 > 0 , which is a contradiction.
For the opposite direction, we assume that there exists a clique C ⊂ V such that supp( p )
⊂
C .
Hence, every pair of distinct nodes in supp( p ) is connected by an edge and consequently w u,v = 0 forall u, v ∈ supp( p ) .
Thus, p ⊤ W p = (cid:80) u,v ∈ V w u,v p u p v = (cid:80) u,v ∈ supp( p ) w u,v p
u p v = 0 .
In light of (9) and Lemma 1, we now define our training loss as
[Equation] [Equation]
We note that the loss function decomposes into two terms.
The first term, L 1 , encourages the allocationof mass at highly connected nodes, while the second term L 2 encourages most of the mass of p to becontained in a clique.
Moreover, β is a hyper-parameter which is used to balance the contributions ofthe two terms.
We can also rewrite L 2 as p T W p =
( (cid:80) nv =1 p v ) 2 − p T W p − (cid:80) nv =1 p 2 v .
3.5 Decoder
In order to extract the maximum clique from p , we use a greedy decoder detailed in in Alg. 1.
Ouralgorithm creates a set Ω
= { ˆ C j } κj =1 of κ cliques and then selects the ˆ C j with largest cardinality.
To construct each ˆ C j , we first create a sorting function π : V → { 1 , . .
, | V |} which puts thenodes in descending order, so p π − 1 (1) ≥ . . .
≥ p π − 1 ( n ) .
Each ˆ C j starts as a single vertex π − 1 ( j ) .
We then consider τ − j additional candidate, π − 1 ( j + 1) , . .
, π − 1 ( τ ) and accept each candidatevertex v canditate if ˆ C j ∪ { v candidate } forms a clique.
This is tested by verifying that x ⊤ W x = 0 forx :
= (cid:80) v ∈ ˆ C j ∪{ v candidate } 1 v .
We remark that in order for our decoder to predict the true maximumclique exactly, we must choose the parameters τ and κ such that there exists a j , 1 ≤ j ≤ κ , suchthat j ≤ π − 1 ( i ) ≤ τ for all v i in the MC.
On simple datasets, such as IMDB, it suffices to set κ = 1 .
More complex datasets such as Twitter require larger values of κ .
Importantly, we note that the ˆ C jcan be computed in parallel, so increasing the value of κ does not create major scalability issues.
Wenote that setting the hyperparameters of the decoder requires some prior knowledge of the largestsizes of the maximum cliques present in the data, which we tune using the validation set.
Results
We empirically experiment on three popular graph learning baselines, namely IMDB, COLLAB [Yanardag and Vishwanathan, 2015] and TWITTER [Yan et al., 2008].
We compare both against otherneural network based methods and against an integer-programming-based solver.
For the the neuralnetworks, Erdös’ GNN [Karalias and Loukas, 2020] and RUN-CSP
[Toenshoff et al., 2019], theauthors provide two implementations, one optimized towards computational speed (fast), and theother optimized for best approximation score (accurate).
The integer-programming-based solver, GUROBI 9.0[Gurobi Optimization, LLC, 2022], is in general more accurate than neural networkbased methods.
However, it is slower on larger graphs.
In order to highlight the contribution ofour hybrid, scattering model, we also provide results for a pure low-pass model, which only uses GCN-type layers [Kipf and Welling, 2016] followed by our decoder module.
We also compare ourresults with traditional heuristic[Grosso et al., 2008]. We provide our results in Tab. 1, where wetrack each model’s test approximation score and the average time needed to classify one graph of thedataset.
For a given graph, the approximation score is computed by dividing the size of the cliquefound by the algorithm by the true MC.
So if the MC of a graph has size 10 and the algorithm returnsa clique of size 9, it would be given a score of 90%.
We then average these scores over all graphs ineach dataset. 4.2 Experiments
For the IMDB dataset, our hybrid approach and most other methods (except RUN-CSP) solve the MCproblem with a perfect (100%) approximation score.
However, our model significantly outperformsthe other neural network-based models (Erdös’ GNN and RUN-CSP) with respect to computationtime (although the highly optimized GUROBI solver is slightly faster than our method on this datasetdue to the small graph size).
On COLLAB, our model almost reaches perfect performance (99.9%) in0.029 S/G, outperforming the other neural network based methods in both speed and approximationscore.
Some versions of the Gurobi solver are able to achieve 99.9% and 100% approximation scores,however, they are slower than our method on this dataset.
Lastly, we experimented on the TWITTERdataset, the most challenging of the datasets considered here.
Among neural networks, RUN-CSP(accurate) does exceptionally well at 98.7% approximation score our method is second best with 95%,while running faster than all compared neural networks, in particular, 6 times faster than RUN-CSP. Although RUN-CSP (accurate) achieves high accuracy, GUROBI runs faster and achieves betterperformance. (GUROBI achieves 0.996 in 0.34 S/G).
Figure 3 illustrates the differences between our hybrid scattering model and the low-pass GCN-basedmodel[Kipf and Welling, 2016] and helps explain why the scattering-based model performs betteracross all three datasets.
It shows that the probability vector p generated by our hybrid scatteringmodel is a more discriminative representation than the one produced by the low-pass model.
Whenusing the low-pass model, most nodes are assigned high probabilities (marked in red).
The decodercannot tell which nodes have higher priorities given such a smooth output.
This leads to the decoderaccepting nodes which are not part of the MC.
If the new clique is not a sub-clique of the MC, thisin turn can lead to the decoder rejecting nodes which are part of the MC.
The scattering model onthe other hand produces a less smooth representation.
The nodes outside the MC are assigned lowerprobabilities so that our decoder can successfully find the MC.
This is consistent with previous work[Wenkel et al., 2022] showing that the graph scattering transform helps overcome the oversmoothnessproblem in node classification.
We emphasize that both of these models are trained with the same lossfunction and the same rule-based decoder to predict the maximum clique.
Therefore, the differencesillustrated in Figure 3 are the direct result of using scattering.
More comparison between the scatteringmodel and low-pass model can be found in the appendix.
There are two hyper-parameters in traditional heuristic, the restart times R h and the maximum numberof iterations S h .
The heuristic randomly selects a node as the start point.
These randomly selectedstart points may lead to sub-optimal solutions and we use restart in conjunction with the heuristic toescape from local optima[Gomes et al., 2008]. However, the restart strategy will bring additionaltime cost. Instead, our GNN generates an efficient representation and we always start from the nodeswith high probabilities. We further apply a greedy decoder based on the GNN representation.
Ourmethod avoids the restart strategy and enables us to quickly obtain a good approximation.
[Table]
[Table] We set the number of samplers, κ , equal to 1, 1 and 10 on IMDB, COLLAB and TWITTER, respectively. We mark the two best neural network models with highest prediction accuracy bold and underlined and the best one. In case of the same performance, we use lower computation time as a tiebreaker.
We provide results for GUROBI with 4 different time budgets. We also provide heuristic’s performance[Grosso et al., 2008]. To further explore how our model compares to the current state-of-the-art commercial solver GUROBI9.0, we introduce a new dataset that contains three classes of graphs with different difficulties (easy,medium and hard) for the maximum clique prediction.
The dataset generation is based on theapproach in Xu et al.
[2005], where we set the hardness parameter to 0.2, 0.5 and 0.8, respectively, togenerate the easy, medium and hard cases.
Each hardness class case contain 1,000 graphs.
We noticethat our model outperforms GUROBI under limited time budget.
The results are shown in Table 2.
Notably, the scattering model is very efficient when dealing with medium hardness cases, where ourmodel is 20 times faster than GUROBI when achieving similar approximation score.
[Table]
our model. Noted that the running time can be larger than the specified time limit when running GUROBI. Overall, we remark that our hybrid model provides considerable speedup compared to both theother neural networks and to GUROBI for large graphs.
We also note that our model takes fewerparameters than the Erdös’ GNN model. Erdös’ GNN model uses a multihead GNN (8 heads, with64 hidden units per head) for each layer, whereas the MLPs used in our method have 8 hidden unitson all three datasets, without using any multi-head attention mechanism. This in turn suggests thatthe use of both GCN filters and scattering filters combined yields powerful feature extractors thathelp learn informative node representations with relatively few parameters. In fact, the Erdös’ GNNmodel contains 1,880,709 parameters for each dataset, while our model contains 1297 parameters for
Twitter, 705 for IMDB and 1001 for COLLAB, respectively.
We only use ∼ 0.07%, 0.04% and 0.05%of the parameter counts that Erdös’ model takes and achieve better performance in terms of bothrunning time and accuracy. Furthermore, since the L -layer GCN-type model has time complexity
O ( LEd + Lnd 2 ) , where L is the number of layers, E is the number of edges and d is the number ofhidden units, it is important to reduce the hidden units number per layer because the time complexityis quadratic to d . As mentioned before, our scattering model only use 8 hidden units (withoutmulti-head attention), which results in lower time complexity and thus faster inference. We provide experiments on large graphs in the supplementary material.  Conclusion
In this paper, we show the expressive power of GNNs can be critical for solving graph CO problems.
Our results suggest that low-pass models that generally enforce smoothness over graph neighborhoods due to oversmoothing, make the nodes indistinguishable, which leads to unfavorable noderepresentations for the MC problem.
Inspired by the geometric scattering transform, we proposean efficient scattering-based GNN to overcome the oversmoothing problem.
We further construct atwo-term loss function which encourages our network both to find a large set of nodes and to find aset of nodes which are contained within a clique, which, in conjunction mimic the otherwise hard tooptimize problem of finding maximum cliques.
Our scattering model largely reduces the number ofparameters and is able to generate efficient representations that enable us to learn probabilities foreach node to be part of the maximum clique at a fast speed.
Based on these node representations weapproximate the correct maximum cliques using a constraint-preserving greedy decoder.
Overall, ourperformance is competitive with other neural network-based methods.