Count-GNN: Graph Neural Networks for Subgraph Isomorphism Counting

Abstract: The prevalence of graph structures has attracted a surge of research interest in graph data. As many graph-based tasks exploit recurring subgraph patterns on graphs, subgraph isomorphism counting becomes an important problem. Classical methods usually boil down to a backtracking framework that needs to navigate a huge search space with prohibitive computational cost due to the NP-completeness of the problem. Some recent studies resort to graph neural networks (GNNs) to learn a low-dimensional representation for both the query subgraph and the input graph, in order to predict the number of query subgraph isomorphisms on the input graph. However, typical GNNs employ a node-centric message passing mechanism that receives and aggregates messages on nodes. While effective on node-oriented tasks, they become inadequate in complex structure matching for isomorphism counting. Moreover, given an input graph, the space of query subgraph is enormous, and thus expecting a single model to fit the diverse range of query subgraphs is unrealistic. In this paper, we propose a novel GNN called Count-GNN for subgraph isomorphic counting, to deal with the above challenges at two levels. At the edge level, we resort to an edge-centric message passing scheme, where messages on edges are propagated and aggregated based on the edge adjacency. By treating edges as first-class citizens, Count-GNN is able to preserve finer-grained structural information, given that an edge is an atomic unit of encoding graph structures. At the graph level, we modulate the graph representation conditioned on the query subgraph, so that the model can be adapted to each unique query for better matching with the input graph. To demonstrate the effectiveness and efficiency of Count-GNN, we conduct extensive experiments on a number of benchmark graphs. Results show that Count-GNN achieves superior performance in comparison to the state-of-the-art baselines.

Reviews and author responses:

 I thank the authors for responding to my comments. I believe that some concrete example or case study where the model facilitates counting subgraphs within an acceptable tolerance is needed to strengthen the paper. I will maintain my current score.

 Thank you for addressing my comments.
I maintain my score and confidence in assessment.

 > Technical contribution.

To cope with the challenges in the problem of subgraph isomorphism counting, Count-GNN mainly consists of two key components, i.e., edge-centric representation learning and modulation-based graph-level aggregation. 
On the one hand, our proposed edge-centric representation learning module is quite different with the suggested approaches [9,10]. In particular, [9] proposes EdgeNet, which is a GNN architecture that allows different nodes to use different parameters to weigh the information of different neighbors. By extrapolating this strategy to more iterations between neighboring nodes, EdgeNet learns edge- and neighbor-dependent weights to capture local details. Thus, they consider the node-centric representation learning yet not the edge-centric representation learning. Another work [10] focuses on node representation learning and resorts to attention mechanism to aggregate messages from neighboring nodes, and they utilize the attention weights to update the edge features, which is different with our edge-centric representation learning.
On the other hand, for our proposed modulation-based graph-level aggregation, different queries are often characterized by distinct structures, which implies that different parts of the input graph will be triggered to match different queries.
Therefore, aggregating all edges in the input graph uniformly cannot retain sufficiently specific structural properties w.r.t. each query. In other words, using a fixed whole-graph representation for the input graph cannot tailor to each query individually for effective structure matching. Thus, we propose to modulate the input graph conditioned on the query, to adapt its whole-graph representation to each query uniquely.

- [9] EdgeNets:Edge Varying Graph Neural Networks. arXiv 2020
- [10] Exploiting Edge Features for Graph Neural Networks. CVPR 2019

> Suggestions of some related studies.

We thank the reviewer for this suggestion. Indeed, our model generally follows similar model design with the line of graph similarity learning approaches, as the reviewer indicated. Besides, the combinatorial approaches open opportunities not only for the graph matching task, but also possibly for the task of subgraph isomorphism counting. We have added the discussion of these studies in Sect.5 (Related Work) of the new version. Due to the difference between the problem of graph matching and subgraph isomorphism counting, we do not import them as baselines since they cannot be directly employed to count the isomorphism on the graph. We believe this direction is quite promising in dealing with the subgraph isomorphism counting, and will investigate in the future work.

> Other minor suggestions.

We thank the reviewer's suggestions. We have modified the paper accordingly.

 > The target audience of this work.

We thank the reviewer for this suggestion. Indeed, it is challenging to clarify the practical scenarios for the neural network based approaches in coping with subgraph isomorphism counting without a theoretical guarantee on the approximation. It is worthy to note that, the scenarios that our model is practicable are mainly those strict in time yet tolerant of the accuracy. For our proposed Count-GNN, the empirical errors are usually within a tolerable margin, providing a useful trade-off between the accuracy and computational cost since many applications do not necessarily need an exact count.

> Novelty.

Yes, we agree that our model is standing on the shoulder of some existing typical modules, as many high-impact approaches did. However, our work resorts to some novel components to cope with the specific challenges existing in the significant problem of subgraph isomorphism counting (not exactly for graph-level representation learning), which provides innovative yet effective solutions to this traditional yet under-explored problem. 
More precisely, our proposed Count-GNN resorts to two main components, i.e., edge-centric representation learning and modulation-based graph-level aggregation, to handle the challenges in subgraph isomorphism counting, forming a novel and effective framework. 


> Minor comments.

We thank the reviewer for the correction. Accordingly, we have made the following updates in the new version: (1) we updated to \#P-complete in the new version; (2) we also added the discussion for the suggested paper which considers fixed-parameter tractable algorithm into Sect.5 in the new version.

 > NP complete -> \#P-complete.

We thank the reviewer for this correction. We have modified this point in the new version.

> The suggested several related papers.

We thank the reviewer's suggestion. We have include the discussion of these related studies in the new version, please see Sect.5.

 > The importance of counting. 

**Subgraph isomorphism counting** is quite important in many real-world scenarios. In these settings, they usually employ the counting of substructures, such as motifs or metagraphs, to depict the structural information of graphs for further usage. For example, for graph learning approaches, some require the counting of specific structures to generate graph kernels [2] or calculate spectral information [3]; for applications on biomolecular networks, some of them utilize the isomorphism counting to generate molecular fingerprints [4,5], or calculate the similarity between molecules [6].
On the other hand, the **exact match** is a finer family of approaches for substructure search on graph. However, in many scenarios as discussed above, we only need a coarser approach to count the subgraph isomorphisms, rather than to precisely search each subgraph isomorphism, which may cost more.

- [2] Efficient graphlet kernels for large graph comparison. Artificial Intelligence and Statistics 2009
- [3] From local measurements to network spectral properties: Beyond degree distributions. CDC 2010
- [4] The generation of a unique machine description for chemical structures-a technique developed at chemical abstracts service. Journal of Chemical Documentation 1965
- [5] Comparing structural fingerprints using a literature-based similarity benchmark. Journal of Cheminformatics 2016
- [6] Biomolecular network motif counting and discovery by color coding. Bioinformatics 2008

> Why the proposed approach is superior to the existing ones?

Compared to existing approaches, our model mainly relies on two main components from the perspectives of edge and graph to promote the performance, i.e., edge-centric representation learning and modulation-based graph-level aggregation. In particular, for the joint representation of query graph, we directly employ a typical readout function (as shown in Eq.(3)) to aggregate the edge-centric representations. For the input graph (or called candidate graph), different queries are often characterized by distinct structures, which implies that different parts of the input graph will be triggered to match different queries. Therefore, aggregating all edges in the input graph uniformly cannot retain sufficiently specific structural properties w.r.t. each query. In other words, using a fixed whole-graph representation for the input graph cannot tailor to each query individually for effective structure matching. Thus, we propose to modulate the input graph conditioned on the query, to adapt its whole-graph representation to each query uniquely. 
In summary, these two modules form the main contribution of this paper, and based on them, our model can outperform the state-of-the-art baselines.


> The implications of the prediction error in the experiments.

To comprehensively demonstrate the effectiveness of the proposed Count-GNN, here we also report the results on metric **MAPE** for Table.2, as follows,

| Methods | SMALL | LARGE | MUTAG |
| :-----| :----: | :----: | :----: |
| GCN | 0.5278 | 0.3392 | $\underline{0.3442}$ |
| GraphSAGE | 0.2562 | 0.3492 | 0.7723 |
| GAT | 0.2368 | 0.3807 | 1.5371 |
| GIN | 0.2174 | 0.3630 | 1.3388 |
| RGCN-SUM | 0.5679 | 8.0188 | 0.6669 |
| RGCN-DN | 0.3402 | 1.0815 | 0.4274 |
| RGIN-SUM | **0.1286** | 0.5518 | 0.8100 |
| RGIN-DN | 0.2625 | **0.2194** | 0.4471 |
| Count-GNN | $\underline{0.1556}$ | $\underline{0.2511}$ | **0.2915** |

From the above table, we can observe that the proposed Count-GNN can also achieve promising performance with the metric MAPE. Though it cannot perform the best on datasets SMALL and LARGE, it is still the runner-up.
In practice, the scenarios that our model is practicable are mainly those strict in time yet tolerant of the accuracy. Besides, to some extent the prediction average twice or 1/2 of the ground-truth can be tolerant in some scenarios which are quite strict to the time, and these errors are much smaller than those of the baselines. In summary, for Count-GNN, the empirical errors are usually within a tolerable margin, providing a useful trade-off between the accuracy and computational cost since many applications do not necessarily need an exact count.


> Two references.

We thank the reviewer's suggestion. The two related studies [7,8] mainly investigate the problem of graph similarity. On the other hand, the subgraph isomorphism counting aims to count the possible number of a given query subgraph appearing on a larger input graph. Thus, they address different problems, and the direct usage of these approaches cannot cope with the problem of subgraph isomorphism counting. We have included this discussion in the new version, please see **Other related studies** in Sect.5.

- [7] SimGNN: A Neural Network Approach to Fast Graph Similarity Computation. WSDM 2019
- [8] Graph Matching Networks for Learning the Similarity of Graph Structured Objects. ICML 2019.

 > Motivation.

1) Previous GNN-based subgraph isomorphism counting models utilize a family of node-centric message passing mechanisms, which propagate and aggregate messages for nodes. While this kind of mechanisms can be effective for node-oriented tasks, it falls short of matching complex structures for isomorphism counting. In particular, they usually rely on message aggregation for local view representations centering on nodes, failing to fundamentally capture the subtle structures especially the link adjacency which is the atomic element in network structures, exposing a bottleneck of prior studies in dealing with the task of subgraph isomorphism counting. 
On the other hand, Count-GNN is built upon an **edge-centric** GNN that propagates and aggregates messages for edges based on the edge adjacency. Given that edges constitute the atomic unit of graph structures, any subgraph is composed of one or more edge chains. Thus, treating them as first-class citizens can better capture fine-grained structural information. We also provide these details in the Introduction of the new version.

2) We illustrated the motivation for the modulation part in **Input graph representation** of Sect.3.3. In particular, a straightforward way for the input graph representation learning is to rely on Eq.(3) as well, which regards all edges uniformly. However, on an input graph, the space of possible query graphs is enormous. Thus, different queries are often characterized by distinct structures, which implies that different parts of the input graph will be triggered to match different queries. Therefore, aggregating all edges in the input graph uniformly cannot retain sufficiently specific structural properties w.r.t. each query. In other words, using a fixed whole-graph representation for the input graph cannot tailor to each query individually for effective structure matching. Thus, we propose to modulate the input graph conditioned on the query, to adapt its whole-graph representation to each query uniquely.


> Typos and clarity.

Thank you for the suggestions. In the new version, we have changed the subscripts of all edge embeddings into the form of ${\langle u, v \rangle}$'s. Besides, we also made bold for both the $\gamma$ and $\beta$ vectors in the new version.


> Comparison with paper [1] (Bouritsas et al. 2020).

Paper [1] investigates the graph-level representation learning based on subgraph isomorphism counting. In particular, they aim to extract the number of subgraphs surrounding a target node to form a specific structural feature vector to promote the expressive power of GNNs. To this end, they propose a model named GSN, a topologically-aware message passing scheme based on substructure encoding. In GSN, the subgraph isomorphism counting is achieved by a predetermined counting method, thus the counting can be further utilized for the expressiveness promotion. Therefore, [1] is not particularly designed for the task of subgraph isomorphism counting, and cannot be directly utilized to deal with this problem. Specifically, it can be regarded as one possible application of our proposed Count-GNN model. We have included this discussion in Sect.5 in the new version.

[1] Improving Graph Neural Network Expressivity via Subgraph Isomorphism Counting. arXiv 2020

The paper present a graph neural network model for subgraph isomorphism counting. The proposed GNN model is based on aggregating and propagating edge features. The edge embeddings of the input graph are generated based on the embeddings of the query graph. The isomorphism counting is performed in a semi-supervised manner. Experimental results on two synthetic dataset and one TU dataset (MUTAG) has been presented. Overall the paper is well-written. The main contribution of the paper is the proposed graph neural network model for subgraph isomorphism counting based on edge embeddings generated by propagating edge features. 

The main concerns to arrive at the recommendation are as follows.

1. Motivation: While it is stressed enough in the paper that node-centric embeddings fall short of matching complex structures for isomorphism counting, the paper fails to answer and convince how exactly does it fall short and how does the proposed method address this issue? Also, the motivation for the proposed module on modulation conditioned on queries is not well explained in the paper. In essence, the rationale for the proposed method is not clear.

2. Scalability and datasets: Apparently generating input graph representation based on queries is computationally too intensive, particularly when the input graph is large with many edges with subgraph isomorphism counting is mostly interesting for large-sized graphs. Towards this end, the experiments presented with datasets (two synthetic and one small MUTAG dataset) are too limited. Please present results on OGB datasets? This becomes evermore important, when  Buritsas et al. 2020 solves the same problem, but large OGB datasets. Also, please provide comparison with Buritsas et al. 2020, along with more elaborate claims on contributions with w.r.t.  Buritsas et al. 2020.

3. Expressivity and power: Since Buritsas et al. 2020 presents results about expressiveness and power of the GSN, a suggestion would be to consider presenting theoretical characterization of the proposed model.

4. Typos and clarity: There are few typos or unclear text in the paper (e.g., error, the need for subgraph patterns etc.). The notation for edge embeddings {\bf h}_<.,u> and {\bf h}_e seem similar, but please define {\bf h}_e. In equations (5) and (6), why are \gamma_e and \beta_e scalars, whiles boldface letters are used for vectors.
 The proposed model for subgraph isomorphism counting is not well-motivation, and the claimed shortcomings for existing methods based node-embeddings are not justified. Comparison in terms of experiments as well contributions with a closed related paper Buritsas et al. 2020 that solves the same problem and provides a theoretical characterization is missing. Experimental validation is limited, datasets are not diverse to support the model (there are many TU datasets and preferably OGB datasets that may be used, for instance).

This paper proposes Count-GNN to predict how many times a small query graph appears in the target graph. This paper proposes two strategies in enhancing existing GNNs to achieve the goal. First, instead of using node-centric message passing, an edge-centric message passing is used. Second, a synergy between query graph and target graph is proposed to make the graph representation between both. It is shown that the proposed approach outperforms baselines.

 Strengths:
1. The paper is targeting a very hard graph problem, and attempts to find approximate solution via graph representation learning.
2. The paper is in general well written and easy to follow.
3. The proposed approach has demonstrated better performance than baselines.

Weaknesses:
1. It needs to be better motivated why the count itself is important while the exact match is not. 
2. Although the solution has shown its superiority empirically, there is not too much theoretical analysis why it is the case. For example, what is so different of the current edge-centric message passing from node-centric message passing where edge information is also included. And why this difference has a better expressiveness in addressing subgraph isomorphism problem.
3. The joint representation of both query graph and candidate graph is not new, and why the proposed approach is superior to the existing ones?
4. It is better advised the implications of the prediction error in the experiments. First, MAE is dependent on the absolute size of the graphs etc, better to change it to MAPE. Second, according to Q-error, the prediction is in average twice or 1/2 of the ground truth. How can we leverage the predicted result in practice?

References:
[1 Bai et al., SimGNN: A Neural Network Approach to Fast Graph Similarity Computation, WSDM 2019
[2] Li et al., Graph Matching Networks for Learning the Similarity of Graph Structured Objects, ICML 2019. In all, this paper provides some interesting study on using GNNs to estimate the subgraph isomorphism count problem. But the paper is purely empirical and lacks of theoretical justification. Also, some important baselines are missing. 

Consider the task of counting the subgraphs of a graph, which are isomorphopic to a given query.
The authors propose an edge based GNN where a representation is learned for each directed edge and where the message passing step consists of aggregating representations of edges which connect to the source vertex of the considered edge.
The final representations so learned are then modulated using a similarly learnt representation of the query to predict the counts. ## Strengths

- The method is common sense and well motivated.
- Except a few minor issues the paper is well written and the description of the method is clear
- The experiments are reasonably fair and thorough. The results conclusively show that the proposed method works significantly better than existing baselines.

## Minor Comments

- The authors mention multiple times that the subgraph counting problem is NP complete, the subgraph matching problem is NP complete, the counting form of the problem is therefore #P-complete
- The authors' method is similar to the GAT on the dual graph as proposed by [DPGCN], which should be cited and ideally compared with empirically.
- Please consider citing the following as newer SOTA methods for counting subgraphs
  - Marco Bressan, Stefano Leucci, and Alessandro Panconesi. 2021. Faster Motif Counting via Succinct Color Coding and Adaptive Sampling. ACM Trans. Knowl. Discov. Data 15, 6, Article 96 (July 2021), 27 pages. DOI:https://doi.org/10.1145/3447397
  - Teixeira, Carlos HC, et al. "Sequential Stratified Regeneration: MCMC for Large State Spaces with an Application to Subgraph Count Estimation." arXiv preprint arXiv:2012.03879 (2020).
  - Pinar, Ali, C. Seshadhri, and Vaidyanathan Vishal. "Escape: Efficiently counting all 5-vertex subgraphs." Proceedings of the 26th international conference on world wide web. 2017.

[DPGCN] - https://geometry.stanford.edu/papers/msblgb-dpgcn-19/msblgb-dpgcn-19.pdf I recommend acceptance based on the novelty and simplicity of the work along with the superior empirical performance of the proposed method.

The authors propose a graph neural network (GNN) model for subgraph isomorphism counting. Previous GNN models for this problem are "node-centric" (i.e., messages are passed and embeddings are learned w.r.t. nodes) whereas the authors propose an "edge-centric" model (i.e., messages are passed and embeddings are learned w.r.t. edges)---the intuition being that focusing on the edges is more important for isomorphism, since we preserve graph structure information.

The authors compare their model to recently-proposed relevant work and, by means of an ablation study, show the value of an edge-centric approach to representation learning for the target problem. Strengths:
- Model is explained clearly, and the motivation for an edge-centric approach is reasonable.
- Experimental results show clear improvement both in performance and/or speed over competing models.
- Model and codebase could be of interest to subject matter experts.

Weaknesses:
- Experiments are performed in relatively small networks
- It is not clear who the target audience is for this work. Model doesn't seem to adequately please any particular group (see below for details)
- No case study to justify the practical value of the proposed method
- Some citations missing, in particular work on fixed-parameter tractable algorithms


Extended review:
As mentioned in the "Strenghts", I think that the authors do a good job at explaining their model and motivating the problem. It should be relatively straightforward for an interested party to reproduce the results, especially with the provided code. The experimental results, in particular the ablation study, provide compelling evidence to support the claims about the advantages of an edge-centric approach.

Where I believe this paper falls short is in having a clear target audience and providing more evidence that their model is relevant to that audience. 

On one side, we have people who want an exact solution or at least an approximate solution with some theoretical guarantee on the approximation; this includes theoreticians and also practitioners who need to have some sense of the accuracy of the method. The paper is NOT targeted to this audience, since the authors hint heavily at the fact that empirical errors of GNN approaches are within a tolerable margin. Furthermore, any kind of theoretical analysis is challenging in this type of models.

That leaves us with subject matter experts in applications where an approximate count, without rigorous guarantees on the approximation, suffices. For this crowd, it is not clear how impactful the results in Table 2 and 3 would be. Looking at the MAE, existing methods have an error of 10 (common) to 70+ (rare) on average. Is missing 10 subgraphs or up to 70 in some cases a significant hurdle in practice? Moreover, if we look at the Q-error, for instance GCN-SMALL, a 14.8 MAE translates to a 2.1 Q-error, which means that the true count in these graphs is somewhat between 7 and 28 subgraphs. Is that a significant error?

I think the paper would greatly benefit from a case study where the authors show that the proposed method is improving the "quality of life" for a subject matter expert. If the proposed method helps to new insights that were not possible with previous counting methods, that would be very compelling evidence. Otherwise, it is challenging to assess the relevance of the method for practitioners.

As a third candidate audience for this work, we have machine learning experts working an graph representation learning. I do not believe that the work introduces any new insights to this crowd, since the components and ideas used in the model have been used before.

Minor comments:
- The authors state that subgraph isomorphism counting is NP-complete. Technically, it is #P-complete.
- One way to deal with the computational complexity in counting problems is to consider fixed-parameter tractable algorithms. I would encourage the authors to cite this line of work. See for instance Alon, Noga, Raphael Yuster, and Uri Zwick. "Color-coding." Journal of the ACM (JACM) 42.4 (1995): 844-856. 
- On page 4, line 4, it should be h^0_{u,v} \neq h^0_{v,u} Given my review above, I believe this paper is slightly below the acceptance threshold, and I would encourage the authors to consider the suggestions above in a future revision.

The problem of counting isomorphic subgraphs is studied in this paper, which is an important and fundamental problem in machine learning but challenging due to the NP-complete challenge of graph isomorphism problems. In this paper, the main technical contribution made by the authors should be the edge-centric message-passing network for graph isomorphism counting, in replacement of the existing node-centric message-passing neural network, because of the insight from the authors that edges are more important than nodes in graph isomorphism. The other parts of the network are kept to be simple neural network building blocks to perform the regression task on the number of isomorphic subgraphs. The experiment results show that the proposed method can outperform existing GNNs, but there is still much room for improvement compared to the optimal ground truth labels. ### Strengths
1. The proposed method seems to have a simple model structure but the experiment results are good compared to other GNN-based methods.
1. The experiment results are detailed with dataset configuration, ablation study, and a study on the sensitivity of the parameters.
1. The experiment setups are detailed and the code is provided to reproduce the reported results.

### Weaknesses
1. The major technical contribution made in this paper, according to my own understanding, is the edge-centric graph neural network. The experiment results on various datasets show that the edge-centric GNN is more powerful compared to other node-centric GNNs. However, it is not a novel idea for incorporating edge features in GNNs, just to name a few:
    * https://arxiv.org/abs/2001.07620
    * https://openaccess.thecvf.com/content_CVPR_2019/papers/Gong_Exploiting_Edge_Features_for_Graph_Neural_Networks_CVPR_2019_paper.pdf

    The other modules designed in this paper seem to be standard neural network building blocks, to perform the regression task towards the ground truth number of isomorphic graphs. Therefore, the technical contribution of this paper is somewhat trivial.
1. The proposed pipeline of this paper (also the pipeline in RGCN/RGIN, Liu et al, KDD 2020) seems to be an extension from the line of graph similarity learning papers to the subgraph isomorphism problem. Some representative graph similarity learning papers are
    * http://arxiv.org/abs/1904.12787
    * http://arxiv.org/abs/1904.01098
    * http://arxiv.org/abs/1808.05689
    * https://arxiv.org/abs/2007.03092

    And the above papers are usually more technically novel by explicitly considering the matching step. Besides, a drawback of these methods that perform a regression task at the output step (including yours) is that the network cannot provide the matching result which is crucial for certain applications. This line of works does not touch the combinatorial nature of graph similarity/isomorphism either. Therefore, I am also suggesting another line of works that can provide graph matching results and/or explicitly deals with the graph similarity problem as a combinatorial problem: 
    * https://arxiv.org/abs/2011.15039
    * https://arxiv.org/abs/2002.03129
    * https://arxiv.org/abs/1904.00597

### Other Remarks
1. It seems improper to add the "QED" sign in Definition 1.
2. Perhaps a typo in Sec 3.1: messages on edges are propagated to and aggregated for each edge based on the edge adjacency, as shown in **Fig. 2(b)c**. The strength of this paper is that the experiment results are detailed with ablation studies. However, my major concern about this paper is the technical contribution made by this paper, and I would personally regard this paper as a combination of existing techniques. I have provided some suggestions in the "weakness" part, and I am wishing to see a technically improved version of this paper in the coming venues.


Meta-review: This work proposes a graph neural network (GNN) model for subgraph isomorphism counting. The work cannot be accepted in its current form. The reviewers provided good comments to authors that can improve their work.

Strengths/Weaknesses:

(+) Experimental results are reasonably detailed with an extensive ablation study. 

(-) Experimental results are limited to two synthetic and one small MUTAG dataset.

(-) The reviewers found the work to be missing a motivation. The authors are correct that subgraph isomorphism frequencies and counts are extremely important in many areas of science, including graph representation learning, the reviewers found it unclear why GNNs are the right tool for the task (in lieu of a specialized randomized algorithm, for instance). 

(-) Regarding specialized algorithms, there are state-of-the-art randomized algorithms that are not search-based via backtracking cited in the paper, such as Motivo (Bressan et al., 2020) and PSRW (Wang et al., 2014) and their derivatives (as one of the reviewers pointed out).

(-) The reviewers were not particularly excited with the technical depth of the paper.

The work definitively has potential. I would encourage the authors to consider better motivating the work.

Decision: Reject