Title: Random Search Neural Networks for Efficient and Expressive Graph Learning
Venue: NeurIPS 2025
Accepted: yes
Keywords: graph neural networks

random walks

long-range dependencies

molecule graph
Abstract: Random walk neural networks (RWNNs) have emerged as a promising approach for graph representation learning, leveraging recent advances in sequence models to process random walks. However, under realistic sampling constraints, RWNNs often fail to capture global structure even in small graphs due to incomplete node and edge coverage, limiting their expressivity. To address this, we propose \textit{random search neural networks} (RSNNs), which operate on random searches, each of which guarantees full node coverage. Theoretically, we demonstrate that in sparse graphs, only $O(\log |V|)$ searches are needed to achieve full edge coverage, substantially reducing sampling complexity compared to the $O(|V|)$ walks required by RWNNs (assuming walk lengths scale with graph size). Furthermore, when paired with universal sequence models, RSNNs are universal approximators. We lastly show RSNNs are probabilistically invariant to graph isomorphisms, ensuring their expectation is an isomorphism-invariant graph function. Empirically, RSNNs consistently outperform RWNNs on molecular and protein benchmarks, achieving comparable or superior performance with up to 16$\times$ fewer sampled sequences. Our work bridges theoretical and practical advances in random walk based approaches, offering an efficient and expressive framework for learning on sparse graphs.

Random Search Neural Networks for Efficient and Expressive Graph Learning

Abstract

Random walk neural networks (RWNNs) have emerged as a promising approach for graph representation learning, leveraging recent advances in sequence models to process random walks. However, under realistic sampling constraints, RWNNs often fail to capture global structure even in small graphs due to incomplete node and edge coverage, limiting their expressivity. To address this, we propose random search neural networks (RSNNs), which operate on random searches, each of which guarantees full node coverage. Theoretically, we demonstrate that in sparse graphs, only O (log | V | ) searches are needed to achieve full edge coverage, substantially reducing sampling complexity compared to the O ( | V | ) walks required by RWNNs (assuming walk lengths scale with graph size). Furthermore, when paired with universal sequence models, RSNNs are universal approximators. We lastly show RSNNs are probabilistically invariant to graph isomorphisms, ensuring their expectation is an isomorphism-invariant graph function. Empirically, RSNNs consistently outperform RWNNs on molecular and protein benchmarks, achieving comparable or superior performance with up to 16 × fewer sampled sequences. Our work bridges theoretical and practical advances in random walk based approaches, offering an efficient and expressive framework for learning on sparse graphs.

1 Introduction

Early work on random walk-based graph representations focused on using skip-gram objectives to learn node embeddings from sampled walks [1, 2]. Building on these ideas and leveraging recent advances in sequence modeling, random walk neural networks (RWNNs) have emerged as a powerful paradigm for modern graph learning [3-8], overcoming the limitations of message-passing neural networks (MPNNs) [9-11] and graph transformers [12-14] by representing graphs as collections of random walks processed by sequence models. This advancement aligns with the broader research goal of identifying effective and expressive methods for graph representation learning [15-17]. However, despite their success, RWNNs encounter critical expressivity challenges under realistic conditions due to incomplete node and edge coverage, limiting their capacity to capture structure even in small graphs (Figure 1). In our analysis, we establish that, under partial coverage, RWNNs are strictly less expressive than traditional MPNNs, highlighting the importance of complete coverage and bridging the theoretical expressivity of the two paradigms.

To illustrate the limitations of RWNNs, consider the graph composed of connected six-cycles and side chains shown in Figure 1. Capturing the full structure of this graph requires traversing every node and edge. However, since the node and edge cover times for a random walk can scale as O ( | V || E | ) [18], RWNNs require either prohibitively long walks or an impractically large number of samples to guarantee complete coverage. Under realistic sampling constraints where the walk's number of steps is significantly less than O ( | V || E | ) , random walks obtain only partial graph reconstruction: as shown in Figure 1(a), subgraphs induced by short random walks can miss critical structural components,

(a) Random Walk Extraction

=

⋃

⋃

(b) Random Search Extraction

Induced spanning tree from

random search 2

Search obtains complete reconstruction

with log sampling (# searches > log(|V|))

Legend

Induced subgraph from

random walk 2

Walks obtain incomplete reconstruction

under realistic sampling (# steps << |V||E|)

Traversed Node

Missed Node

Missed Edge

Induced subgraph from

random walk 1

Induced spanning tree from

random search 1

=

such as the side chains connected to the six-cycles. This incomplete coverage significantly hinders RWNNexpressivity. Current methods attempt to address this limitation through non-backtracking walks [5, 6] and minimum-degree local rules (MDLR) [7], reducing node and edge cover time to O ( | V | 2 ) . Nonetheless, these approaches retain quadratic complexity with respect to graph size, making comprehensive coverage costly and impractical for even small and medium graphs.

To overcome these challenges in small and medium sized graphs, we introduce random search neural networks (RSNNs) , which represent graphs as collections of random searches. Critical to our analysis is the insight that subgraphs induced by searches are spanning trees as opposed to arbitrary subgraphs induced by random walks. Each spanning tree inherently ensures full node coverage, reducing the task to achieving edge coverage across the union of induced trees. Leveraging this insight, our analysis demonstrates that RSNNs require only a logarithmic number of searches for complete edge coverage, specifically in sparse graphs where such searches are computationally feasible. This is a substantial improvement over the linear number of walks required by RWNNs, assuming walk lengths scale with graph size. As shown in Figure 1(b), the union of just a few spanning trees enables complete reconstruction of the graph, including nodes and edges missed by walk-induced subgraphs. When equipped with maximally expressive sequence models, RSNNs achieve universal approximation efficiently. Furthermore, we show that RSNNs are probabilistically invariant to graph isomorphisms, ensuring their expectation is an isomorphism-invariant predictor. Empirically, we focus on sparse molecular and protein graph classification datasets, domains in which RWNNs have shown significant improvement over existing GNNs. Across both domains, we demonstrate that RSNNs consistently outperform existing RWNN approaches. In summary, we make the following contributions:

Characterizing RWNN Expressive Limitations. Our analysis characterizes the expressive power of RWNNs, bridging the expressivity of RWNNs and MPNNs. We demonstrate that RWNNs under partial node and edge coverage are strictly less expressive than MPNNs, motivating the design of sampling strategies that guarantee full coverage.

New Model: Random Search Neural Networks. We propose random search neural networks (RSNNs), a new approach that operates on random searches, whose induced subgraphs are spanning trees, substantially reducing the sample size required for complete node and edge coverage in sparse graphs.

Efficient Coverage, Universal Approximation, & Isomorphism Invariance. We demonstrate that RSNNs can achieve universal approximation efficiently with logarithmic sampling in sparse graphs. RSNNs are also probabilistically invariant to graph isomorphims, ensuring their expectation is an isomorphism-invariant function on graphs.

Extensive Empirical Analysis. Focusing on sparse molecular and protein graph benchmarks, we demonstrate that RSNNs consistently outperform existing RWNNs.

2 Background and Preliminaries

We establish notation for graphs and random walks and next review MPNNs and RWNNs, the primary class of models under investigation. Importantly, we later bridge the expressivity of MPNNs and RWNNs. We lastly review random walk cover times, highlighting how RWNNs require prohibitively long walks or impractically large numbers of walks to guarantee full graph coverage.

2.1 Notation and Random Walks on Graphs

We define a graph G = ( V, A , X ) , where V is the set of nodes, A ∈ { 0 , 1 } | V |×| V | is the adjacency matrix representing the set of edges E , and X ∈ R | V |× d is the node feature matrix. For each node i ∈ V , we denote its feature vector as x i and its set of immediate (one-hop) neighbors as N ( i ) . We define the augmented neighborhood ˆ N ( i ) , obtained by adding a self-loop to node i .

A random walk of length ℓ on G produces a sequence of nodes W = ( w 0 , . . . , w ℓ ) by first sampling an initial node w 0 ∈ V according to a uniform distribution P 0 , and then iteratively transitioning to subsequent nodes by sampling neighbors according to a given random walk algorithm. We let W ℓ ( G ) denote the set of all possible random walks of length ℓ on G , and let P ( W ( G ) , P 0 ) represent a probability distribution over these walks. Lastly, we define P m ( W ( G )) = { W 1 , . . . , W m } as a realization of a set of m independently sampled random walks from P ( W ( G ) , P 0 ) .

2.2 Message-passing Neural Networks and GNN Expressivity

Standard GNNs adopt a message-passing approach, where each layer iteratively updates a node's representation by aggregating the features of its neighbors [19]. Formally, the initial message-passing layer can be defined as the following propagation rule at the node level for all i ∈ V ,

where f agg is a permutation-invariant function. Because of this aggregation step, MPNNs incur fundamental expressivity limitations and cannot distinguish certain classes of non-isomorphic graphs [15, 20]. We compare the expressivity of GNNs by the pairs of graphs they can distinguish [21], introducing the following notation. For two GNNs f 1 and f 2 , we write

̸

Thus, any pair indistinguishable by f 1 is also indistinguishable by f 2 , so f 1 is at least as expressive as f 2 . The relation is strict, f 2 ≺ f 1 , if f 2 ⪯ f 1 and there exist graphs G,H with f 1 ( G ) = f 1 ( H ) while f 2 ( G ) = f 2 ( H ) . f 1 and f 2 are equally expressive, written f 1 ≃ f 2 , if f 2 ⪯ f 1 and f 1 ⪯ f 2 . These relations coincide with notions of approximation power. For example, if f 2 ≺ f 1 , every target approximable by f 2 is approximable by f 1 , and there exist targets approximable by f 1 but not f 2 .

2.3 Random Walk Neural Networks

RWNNs are a novel class of neural network on graphs that leverage sequence models to process random walks sampled from the graph. Typically, an RWNN is characterized by four key components: (1) a random walk algorithm that generates node sequences, (2) a recording function that encodes the walks into structured representations, (3) a reader neural network that processes these representations, and (4) an aggregator that combines the representations or predictions from multiple walks.

For our analysis, we assume the following representative general version of RWNN [3-8]. Specifically, we consider the random walk algorithm as uniform random walks of fixed length ℓ , denoted by P m ( W ( G )) := P ( W ℓ ( G ) , U ( V )) , where U ( V ) denotes the uniform distribution over V . Given a sampled walk W ∈ P m ( W ( G )) , we define the recording function f emb : W ℓ ( G ) → R ℓ × d as follows:

where h V : V → R d is a node embedding function. Here, h PE [ i ] serves as an optional position encoding that supplies extra structural context for each node in the walk (Appendix B); when such encoding is employed, it is further processed by the learnable projection mapping proj : R d pe → R d . Subsequently, we assume walk embeddings produced by f emb are processed by a sequence model,

denoted by f seq : R ℓ × d → R ℓ × d . Finally, embeddings from the sequence model are aggregated by a permutation-invariant function. The choice for the function can be simple functions such as taking the mean over random walk representations such as in Wang and Cho [3], Kim et al. [7], or it can be more complex as in Tönshoff et al. [5], Chen et al. [6], which updates a node's representation as the aggregation of its representations across all walks using the aggregation function f agg : R m × ℓ × d → R | V |× d :

where N i ( P m ( W ( G ))) represents the number of occurrences of node i in the union of walks in P m ( W ( G )) . The RWNN layer is defined as the composition f l RWNN = f l agg ◦ f l seq , while the overall architecture f RWNN is defined as the stacking of RWNN layers. In the node classification setting, the final node representation f agg [ i ] produced by the last RWNN layer is directly utilized for predictions. In graph classification, an additional global pooling function aggregates these node representations into a single representation for the graph.

2.4 Random Walk Cover Times

RWNN expressivity depends on how much of the graph its random walks visit (Section 3). Here, we review results on random walk node cover times, C V ( G ) , the expected number of steps a walk takes to visit all nodes. For a connected graph G = ( V, E ) , the cover time of a general uniform random walk satisfies C V ( G ) = O ( | V || E | ) [22]; in particular, for sparse graphs ( | E | = Θ( | V | ) ) this gives C V ( G ) = O ( | V | 2 ) . Minimum-degree local rule (MDLR) walks further achieve C V ( G ) = O ( | V | 2 ) on all graphs, which is optimal among first-order walks [7, 23]. Non-backtracking walks can also empirically reduce cover times on graphs [5, 6]. Even with these improvements, guaranteeing full node and edge coverage by random walks can require prohibitively long walks or impractically large numbers of walks. We therefore replace walks entirely with searches (Section 4), significantly improving on the number of samples required for full coverage in comparison to random walks.

3 Expressive Power of Random Walk Neural Networks

In this section, we characterize the expressive power of RWNNs. Our main result establishes that without additional positional or structural encodings, RWNNs with access to the complete multiset of random walks whose lengths scale up to the cover time are exactly as expressive as MPNNs. In practice, however, such assumptions are unrealistic: guaranteeing full node and edge coverage requires walk lengths on the order of the cover time, rendering full coverage computationally infeasible. We then show that in the partial-coverage regime, RWNNs are strictly less expressive than MPNNs. This limitation motivates our random search neural network (RSNN), which achieves full coverage and thus maximal expressivity at significantly lower sampling cost.

3.1 The Role of Coverage: RWNNs vs. MPNNs

We first analyze the ideal setting in which the RWNN has access to complete walk sets up to the cover time. In this regime, RWNN expressive power matches that of MPNNs.

Theorem 3.1 (RWNN-MPNN Equivalence Under Full Coverage (FC)) . Let G be a graph. Let f FC RWNN denote an RWNN with injective f seq and f agg with no additional positional encodings, applied to the complete multiset of walks W ≤ ℓ ( G ) with lengths up to ℓ = C E ( G ) , the edge cover time of G . Let f MPNN be an MPNN with injective f agg . Then, for all graphs G,H ,

Hence, f FC RWNN ≃ f MPNN (i.e., f FC RWNN and f MPNN are equal in expressive power).

Although Theorem 3.1 shows that full-coverage RWNNs and MPNNs are equal in expressivity, RWNNs under full coverage can be more effective empirically. RWNNs leverage expressive sequence models capable of capturing long-range dependencies when given full graph structure in complete sequences. MPNNs instead rely on iterative neighborhood aggregation and are limited in depth by oversmoothing [24] and oversquashing [25], which in practice reduce their expressivity and

capabilities to capture long-range signals. This contrasts our theoretical setup where we assume MPNNs have unlimited depth, allowing them to match full-coverage RWNN expressivity.

Constructing complete walk sets with lengths up to the cover time, however, is typically computationally infeasible. RWNNs can thus fall short of MPNNs under realistic budgets despite their inherent advantages. Indeed, as an immediate consequence of Theorem 3.1, when RWNNs operate under partial coverage, their expressive power is strictly less than that of MPNNs.

Corollary 3.2 (RWNNs Under Partial Coverage (PC)) . Let f PC RWNN denote an RWNN of the same architectural class as in Theorem 3.1 but applied to a multiset of random walks that attains only partial node/edge coverage of the input graph. Then, for all graphs G,H ,

and there exist non-isomorphic graphs G ̸ ∼ = H such that

̸

Hence, f PC RWNN ≺ f MPNN (partial-coverage RWNNs are strictly less expressive than MPNNs).

Corollary 3.2 reveals a fundamental limitation of RWNNs: under partial coverage, their expressive power falls below that of classical message passing. Thus, to attain maximal theoretical expressivity, it is essential to design sampling strategies that efficiently guarantee complete coverage. In order to realize the advantages of RWNNs while obtaining maximal expressivity, we introduce RSNNs (Section 4), which replace walks with searches to guarantee full node coverage by construction and achieve full edge coverage with a small number of searches on sparse graphs.

Insights of the analysis. In proving Theorem 3.1, we introduce a walk-based color refinement, Walk Weisfeiler-Lehman ( WWL ; Definition A.3), which updates each node using the multiset of walks that visit it. We demonstrate that WWL upper bounds RWNN expressivity (Lemma A.5). Next, we establish that WWL operates on the same object as classical WL : unfolding trees (Definition A.6). We lastly leverage this insight to establish that WWL and WL have equal distinguishing power (Theorem A.9). In essence, this construction aligns the Weisfeiler-Lehman hierarchy with RWNNs , unifying the expressive power of two seemingly distinct model classes: RWNNs, which process random walks with sequence models, and MPNNs, which process multisets of node neighborhoods with graph convolution. Formal definitions and details are in Appendix A.

4 Random Search Neural Networks (RSNNs)

Motivated by our analysis of RWNNs, we propose a new sampling strategy that efficiently achieves the necessary conditions for maximal expressivity: full node and edge coverage. Since random walks require either prohibitively long walks or an impractically large number of walks to guarantee full coverage, we introduce random search neural networks (RSNNs), which represent graphs as collections of random searches. Notably, a single search guarantees full node coverage, and under the sparse graph assumption, only O (log( | V | )) searches are needed to capture all edges. This significantly reduces the sampling complexity compared to the O ( | V | ) requirement for traditional RWNNs, assuming walk lengths scale on the order of O ( | V | ) . When paired with a maximally expressive sequence model, RSNNs emerge as universal approximators on graphs. Moreover, we provably show RSNNs are probabilistically invariant to graph isomorphisms. Hence, the predictor obtained by averaging over searches is an isomorphism-invariant graph function. While the computational cost of a full search can be significantly larger than a short random walk, we focus on sparse graphs where search is computationally feasible, addressing the limitations of RWNNs in these classes of graphs.

4.1 Search via Random DFS

RSNNs leverage a random depth-first search (DFS) procedure to obtain sequences from an input graph G . We utilize a DFS rather than a breadth-first search to better preserve continuity in the sequence. RSNN obtains a random DFS by sampling a uniform root and independent random neighbors at each vertex; this induces a distribution over DFS sequences we denote S DFS ( G ) . RSNN collects m independent searches to form the set P m ( S DFS ( G )) = { S 1 , . . . , S m } . Once these searches are obtained, RSNNs leverage all the advances of RWNNs but with new benefits. We apply the recording function (Equation (1)) to each search, including positional encodings from Tönshoff et al. [5] to distinguish between disconnected nodes and true connections in the sequence. Search embeddings are then processed with a sequence model and the node aggregation function (Equation (2)) (Figure 2).

{

{

{

f

seq

f

emb

f

agg

Aggregation

Random DFS Extraction

Original

{

.

.

.

.

.

.

m

m

1

1

4.2 From Efficient Graph Coverage to Universal Approximation in RSNNs

In this section, we establish the theoretical foundations of RSNNs by demonstrating how our random search strategy efficiently obtains full graph coverage. Central to our analysis is the observation that the subgraphs induced by DFS sequences are spanning trees. Leveraging this insight, we prove the following key lemma showing that for sparse graphs with bounded degree, a logarithmic number of random searches is sufficient to guarantee full node and edge coverage with high probability.

Lemma 4.1 (Logarithmic Sampling Yields Full Edge Coverage) . Let G = ( V, E ) be a connected graph with | E | ≤ C | V | for some constant C and a bounded maximum degree d max . Let S 1 , S 2 , . . . , S m be m independent random searches sampled from G , and let T 1 , T 2 , . . . , T m be their corresponding induced spanning trees. Then, for small δ ≪ 1 , if the union of T 1 , T 2 , . . . , T m contains every edge in E with probability at least 1 -δ .

In contrast to RWNNs, which require m = O ( | V | ) random walks of length ℓ = O ( | V | ) , RSNNs achieve complete coverage with m = O (log( | V | )) searches of length ℓ = O ( | V | ) . With full node and edge coverage, RSNNs are able to capture all the information necessary to represent any function on graphs. Intuitively, this means that under our sampling strategy, RSNNs are universal approximators: they can approximate any graph function arbitrarily well, provided they are paired with a universal sequence model such as transformers or LSTMs [26, 27].

Theorem 4.2 (Universal Approximation by RSNNs on Sparse Graphs with Bounded Degree) . Let ϵ > 0 and let f : G → R d be any continuous graph-level function, where G is the space of sparse graphs with | E | = O ( | V | ) , maximum degree at most d max , and size at most | V | ≤ n max . Assume m satisfies Equation (3) , so that full coverage is achieved with probability at least 1 -δ . Then, with probability at least 1 -δ there exists an RSNN configuration such that

4.3 From Expressivity to Invariance: Isomorphism Invariance of RSNNs

Having established the expressive capabilities of RSNNs, we now turn to invariance. For graphs, the target symmetry is isomorphism invariance: for all G ∼ = H , an isomorphism-invariant graph function satisfies f ( G ) = f ( H ) . Graph functions that capture the symmetry enjoy learning and generalization benefits. Because RSNNs are randomized functions, we adopt the notion of probabilistic invariance [7,

28]: for all G ∼ = H , the random outputs satisfy f ( G ) d = f ( H ) . Intuitively, a randomized graph function is probabilistically invariant to graph isomorphisms if its distribution is unchanged under any graph isomorphism. We demonstrate that the randomized DFS procedure used by RSNNs is probabilistically invariant; consequently, the RSNN predictor f RSNN is invariant in distribution, and its expectation Φ( G ) := E [ f RSNN ( G )] is an isomorphism-invariant function on graphs.

Theorem 4.3 (Probabilistic Isomorphism-Invariance of RSNN) . A randomized search procedure on a graph G produces a sequence S G = ( s G 0 , . . . , s G | V ( G ) | ) of visited vertices. We say the procedure is

probabilistically invariant to graph isomorphisms if for all graph isomorphisms π ,

Learning the invariance. In addition to being invariant in expectation, we show that RSNNs can learn the optimal invariant predictor throughout training even under limited sampling budgets, where the expectation is only approximated (e.g., m = 1 sampled search for each forward pass in the parameter update). At inference, the invariant predictor can then be computed exactly or estimated by the Monte Carlo estimator. Our result follows Murphy et al. [29, 30]. For RSNN parameters W , define the model output on a graph G and a sampled search set S ∼ S DFS ( G ) as f RSNN ( G,S ; W ) .

The randomized DFS procedure used in RSNNs satisfies the above definition. Hence, RSNNs satisfy probabilistic invariance: for all G ∼ = H , f RSNN ( G ) d = f RSNN ( H ) , and the averaged predictor Φ( G ) := E [ f RSNN ( G ) ] is an invariant function on graphs: Φ( G ) = Φ( H ) for all G ∼ = H .

Corollary 4.4 (SGD converges to the invariant objective) . Let ℓ ( · , y ) be differentiable and define

At each step t , sample a mini-batch B t = { ( G ( i ) t , y ( i ) t ) } B i =1 i.i.d. from D and, for each i , draw a single S ( i ) t ∼ S DFS ( G ( i ) t ) independently of W t ; update

Then E [ 1 B ∑ B i =1 ∇ W ℓ ( f RSNN ( G ( i ) t , S ( i ) t ; W t ) , y ( i ) t ) ] = ∇ W L ( W t ) , i.e., the mini-batch gradient is an unbiased estimator of ∇ L ( W t ) . Under standard SGD conditions, W t converges almost surely to an optimizer W ⋆ of the invariant objective.

Inference. Given a fixed point W ⋆ and a new test graph G ′ , the invariant prediction is E S [ f RSNN ( G ′ , S ; W ⋆ )] , which can be exactly computed or approximated with the estimator 1 m ∑ m j =1 f RSNN ( G ′ , S j ; W ⋆ ) where S 1 , . . . , S m i.i.d. ∼ S DFS ( G ′ ) .

4.4 Runtime Complexity

We compare the sampling costs of RSNNs and RWNNs. In our approach, each random search corresponds to a DFS traversal. Assuming a sparse graph, a single DFS has a worst-case cost of O ( | V | ) , and obtaining m searches requires O ( m | V | ) time, efficient and computationally feasible in small to medium-sized graphs. In contrast, RWNNs generate m random walks of length ℓ , with total sampling cost O ( mℓ ) . When ℓ ≪| V | , random walk sampling can be faster than random search extraction. However, as we have shown, short walks fail to capture global structure, leading to reduced expressivity. Thus, while RSNN sampling is more expensive when ℓ is small, its increased coverage and performance can justify its cost, especially in graphs where full structure is critical.

5 Experiments & Results

Through empirical evaluation we aim to answer the following research questions, extending our theory by testing RSNNs on datasets with factors not explicitly addressed in the theoretical analysis (e.g., class imbalance, rich node features), and testing RSNNs against models beyond our theory such as canonical approaches (e.g., SMILES, Fingerprints) used commonly in molecular analysis.

RQ1 (Discriminative performance): How does RSNN discriminative performance compare to standard baselines and RWNNs across sparse graph benchmark tasks?

RQ2 (Node and edge coverage): Do RSNNs achieve higher node and edge coverage than RWNNs as the number of sampled searches m increases, and does this increased coverage translate into improved task performance?

RQ3 (Generalization to larger and denser graphs): How do RSNNs perform on larger and denser graphs, where attaining full edge coverage is computationally expensive?

5.1 Experimental Setup

Datasets. We focus our analysis on molecular and protein benchmarks, domains where RWNNs have demonstrated strong empirical performance and where efficient coverage, long-range dependencies, and high expressivity are essential [5, 7, 31]. Importantly, RSNNs are not intended as a solution across all domains, but as a principled alternative for sparse graphs requiring representations that capture global structure. Specifically, we evaluate on four small-scale molecular graph classification datasets from MoleculeNet [32]: CLINTOX , SIDER , TOX21 , and BBBP . These benchmarks span diverse molecular tasks such as toxicity and adverse reaction prediction, with graph sizes ranging from tens to hundreds nodes. We also include four protein graph classification datasets from ProteinShake [33]: EC Subclass , EC Mechanism , SC Class , and SC Family . Protein graphs are significantly larger than molecules, ranging up to thousands of nodes, making it more difficult to capture global structure. To assess scalability, we evaluate on large-scale molecular benchmarks with hundreds of thousands of graphs from Open Graph Benchmark [34]: PCBA-1030 , PCBA-1458 , and PCBA-4467 . Lastly, to test generalization to larger and denser graphs, we evaluate on NeuroGraph-task , a brain graph benchmark, where the task is to predict one of seven mental states (e.g., emotion processing). We provide descriptive statistics for all datasets in Tables 1 and 2.

Baselines. First, we compare to standard molecular learning baselines: (1) SMILES , a sequence model applied to canonical SMILES [35]; (2) GCN [36] and (3) GIN [15], message-passing GNNs; and (4) GT [12], a graph transformer model. In addition, we compare to (5) Fingerprint , a multilayer perceptron trained on hand-crafted chemical descriptors known to be effective in molecular tasks [37]. Importantly, SMILES and Fingerprint are not applicable in protein graphs. Second, we consider four RWNN variants as baselines for comparison: (6) RWNN-base , which employs uniform random walks of length ℓ with mean aggregation over walk representations [4]; (7) RWNN-anon , which augments the base model with a node anonymization strategy from Wang and Cho [3]; (8) RWNN-mdlr , which uses minimum-degree local rule walks from [7], anonymization, and mean aggregation; (9) CRAWL [5], which applies non-backtracking walks with node-level aggregation. We consider three sequence models for f seq: (a) GRU [38], (b) LSTM[26], and (c) transformer [27].

Training and Evaluation. To ensure fair comparisons, all RWNNs and RSNN are configured with the same number of samples m , and RWNN walk lengths are set to ℓ = | V | , the number of nodes per graph, so that asymptotic runtimes are equivalent across methods. On molecular benchmarks, we sample a new set of m walks for each forward pass during training, and on protein benchmarks, we precompute the set of m walks before training. Following each dataset's protocol, performance is computed as AUC or accuracy. We report median (min, max) performance over five random splits (60/20/20), which is more robust than mean and standard deviation for small sample sizes. All models are trained on a machine equipped with 8 × NVIDIA GeForce GTX 1080 Ti GPUs; if a model does not converge within 24 hours, we omit it from evaluation. All remaining details are in Appendix D 1 .

5.2 RQ1 & RQ2: Discriminative Performance and Coverage

First, RSNNs significantly outperform standard baselines across all benchmarks, demonstrating their effectiveness for molecular and protein learning (Table 1). Notably, at m = 16 , RSNNs match or exceed the performance of Fingerprint models, which do not rely on learned representations and instead use features designed by domain experts. For all RWNNs and RSNN, we present results using GRU, which performs best empirically, and include additional results for LSTMs and transformers in the Appendix C, where we observe similar trends. Compared to existing RWNNs, RSNNs exhibit greater expressivity at low sampling budgets; with a single search ( m = 1 ), RSNN significantly outperforms all RWNN variants at the same budget. Moreover, across all molecular benchmarks, RSNNs at m = 1 match or exceed the best-performing RWNNs at m = 16 , highlighting their sample efficiency. While performance differences narrow at m = 16 on molecular benchmarks, RSNNs retain a substantial lead on larger protein graphs, underscoring their expressivity in structurally complex settings. On large-scale molecular benchmarks, training both RWNNs and RSNNs with m> 1 becomes computationally infeasible, exceeding the 24-hour time budget. At m = 1 , however, RSNNs maintain strong performance and substantially outperform RWNNs (Table 2), demonstrating RSNNs' robustness under sampling constraints when computation is limited.

We compare how node/edge coverage and performance varies with the number of walks or searches for RSNNs and CRAWL, the strongest RWNN baseline (Figure 3). Across all benchmarks, we observe a strong correlation between coverage and model performance. On molecular graphs, RSNNs achieve full node and high edge coverage with a single search ( m = 1 ), resulting in strong initial performance. This aligns with our theoretical analysis: each RSNN search guarantees node coverage by construction, and

only a few searches are needed to achieve full edge coverage in sparse graphs. In contrast, CRAWL begins with low node and edge coverage and only reaches RSNN-level performance at m = 16 , once coverage converges, highlighting RWNN limitations under small sampling budgets. On larger protein graphs, both coverage and performance improve more gradually, but RSNNs retain a consistent performance advantage across all m , underscoring the benefit of efficient coverage in larger graphs.

5.3 RQ3: Generalization to Larger and Denser Graphs

To assess generalization beyond small and sparse regimes, we evaluate on a NeuroGraph benchmark of brain graphs. These graphs are substantially larger than molecules and denser than proteins, making full edge coverage expensive for both walks and searches. We compare RSNN against CRAWL. RSNN outperforms CRAWL at m = 4 , 16 , indicating that RSNNs can leverage structure even when full coverage is expensive and that their performance advantage remains on larger and denser graphs.

1.0

1.0

1.0

Coverage

0.8

Coverage

0.8

Coverage

0.8

RSNN -Node Coverage

0.6

0.6

RSNN -Node Coverage

0.6

RSNN-Edge Coverage

RSNN-Edge Coverage

RSNN-Edge Coverage

CRAWL-Node Coverage

0.4

CRAWL-Node Coverage

0.4

CRAWL-Node Coverage

0.4

CRAWL-Edge Coverage

0.2

CRAWL-Edge Coverage

0.2

CRAWL-Edge Coverage

0.90

0.20

0.5

AUROC

0.85

0.15

0.4

0.80

0.10

RSNN

RSNN

RSNN

0.75

CRAWL

0.05

CRAWL

0.3

CRAWL

4

8

16

1

4

8

16

1

4

8

16

Number of Walks/Searches (m)

Number of Walks/Searches (m)

Number of Walks/Searches (m)

(a) BBBP Molecular Graph

(b) Structural Family Protein Graph

(c) Enzyme Subclass Protein Graph

6 Discussion and Conclusion

We present the first theoretical analysis of RWNNs under realistic sampling constraints, showing that their expressivity is fundamentally limited without full node and edge coverage, even in small graphs. We prove that under partial coverage, RWNNs are strictly less expressive than traditional MPNNs. To address this, we introduce RSNNs, which use random depth-first search to guarantee full node coverage and edge coverage with only a logarithmic number of samples in sparse graphs. When paired with expressive sequence models, we show that RSNNs are universal approximators. Furthermore, RSNNs are also probabilistically invariant to graph isomorphisms. Empirically, RSNNs consistently outperform RWNNs on both molecular and protein benchmarks, requiring up to 16 × fewer samples to achieve comparable performance.

Our work builds on recent work in RWNNs that combines random walks with expressive sequence models [3-8]. These works explore various walk strategies, including uniform walks [3, 4], nonbacktracking walks [5, 6], minimum-degree local rule walks [7], and learnable walks [8], and propose architectural improvements to enhance expressivity and performance. We critically examine the expressivity of RWNNs under realistic sampling constraints, relaxing prior assumptions that walks are as long as cover times. Based on our analysis, we propose to replace random walks entirely with random searches, leading to RSNNs, a more sample-efficient and expressive alternative.

Our work is not without limitations. In particular, RSNNs are tailored to sparse, medium-sized graphs. How to scale RSNN to extremely large, densely connected graphs remains an open question. In such settings, full-depth searches may become prohibitively expensive, and edge coverage may scale less efficiently. A promising direction is to explore truncated searches that capture key structural signal while reducing computation. This raises new questions about how coverage and expressivity behave under partial searches, particularly in dense regimes where full coverage is infeasible.

Despite the focused scope, our results are promising: RSNNs match or exceed RWNN performance with significantly fewer samples and maintain a clear advantage across benchmarks. These findings underscore the value of replacing random walk sampling with search-based sampling in graph learning. More broadly, this work highlights the importance of moving beyond local neighborhoods toward sampling strategies that capture global structure. By leveraging efficient coverage through random searches, RSNNs offer a principled, expressive, and sample-efficient framework for learning on sparse graphs, laying the foundation for future exploration in other settings.

References

Bryan Perozzi, Rami Al-Rfou, and Steven Skiena. Deepwalk: Online learning of social representations. In Proceedings of the 20th ACM SIGKDD international conference on Knowledge discovery and data mining , 2014.

Aditya Grover and Jure Leskovec. node2vec: Scalable feature learning for networks. In Proceedings of the 22nd ACM SIGKDD international conference on Knowledge discovery and data mining , 2016.

Yuanqing Wang and Kyunghyun Cho. Non-convolutional graph neural networks. In Advances in Neural Information Processing Systems , 2024.

Yanchao Tan, Zihao Zhou, Hang Lv, Weiming Liu, and Carl Yang. Walklm: A uniform language model fine-tuning framework for attributed graph embedding. In Advances in neural information processing systems , 2023.

Jan Tönshoff, Martin Ritzert, Hinrikus Wolf, and Martin Grohe. Walking out of the weisfeiler leman hierarchy: Graph learning beyond message passing. Transactions in Machine Learning Research , 2023.

Dexiong Chen, Till Hendrik Schulz, and Karsten Borgwardt. Learning long range dependencies on graphs via random walks. In International Conference on Learning Representations , 2025.

Jinwoo Kim, Olga Zaghen, Ayhan Suleymanzade, Youngmin Ryou, and Seunghoon Hong. Revisiting random walks for learning on graphs. In International Conference on Learning Representations , 2025.

Karolis Martinkus, Pál András Papp, Benedikt Schesch, and Roger Wattenhofer. Agent-based graph neural networks. In International Conference on Learning Representations , 2023.

Qimai Li, Zhichao Han, and Xiao-Ming Wu. Deeper insights into graph convolutional networks for semi-supervised learning. In Proceedings of the AAAI conference on artificial intelligence , volume 32, 2018.

Jiong Zhu, Yujun Yan, Lingxiao Zhao, Mark Heimann, Leman Akoglu, and Danai Koutra. Beyond homophily in graph neural networks: Current limitations and effective designs. In Advances in Neural Information Processing Systems , 2020.

Michael Ito, Danai Koutra, and Jenna Wiens. Understanding gnns and homophily in dynamic node classification. In International Conference on Artificial Intelligence and Statistics . PMLR, 2025.

Vijay Prakash Dwivedi and Xavier Bresson. A generalization of transformer networks to graphs. In AAAI Workshop on Deep Learning on Graphs: Methods and Applications , 2021.

Ladislav Rampášek, Michael Galkin, Vijay Prakash Dwivedi, Anh Tuan Luu, Guy Wolf, and Dominique Beaini. Recipe for a general, powerful, scalable graph transformer. In Advances in Neural Information Processing Systems , 2022.

Michael Ito, Jiong Zhu, Dexiong Chen, Danai Koutra, and Jenna Wiens. Learning laplacian positional encodings for heterophilous graphs. In International Conference on Artificial Intelligence and Statistics . PMLR, 2025.

Keyulu Xu, Weihua Hu, Jure Leskovec, and Stefanie Jegelka. How powerful are graph neural networks? In International Conference on Learning Representations , 2019.

Xiyuan Wang and Muhan Zhang. How powerful are spectral graph neural networks. In International Conference on Machine Learning , 2022.

Liheng Ma, Chen Lin, Derek Lim, Adriana Romero-Soriano, Puneet K Dokania, Mark Coates, Philip Torr, and Ser-Nam Lim. Graph inductive biases in transformers without message passing. In International Conference on Machine Learning , 2023.

Romas Aleliunas, Richard M Karp, Richard J Lipton, László Lovász, and Charles Rackoff. Random walks, universal traversal sequences, and the complexity of maze problems. In 20th Annual Symposium on Foundations of Computer Science (sfcs 1979) , pages 218-223. IEEE Computer Society, 1979.

Justin Gilmer, Samuel S Schoenholz, Patrick F Riley, Oriol Vinyals, and George E Dahl. Neural message passing for quantum chemistry. In International Conference on Machine Learning , 2017.

Ryoma Sato. A survey on the expressive power of graph neural networks. arXiv preprint arXiv:2003.04078 , 2020.

Waiss Azizian and Marc Lelarge. Expressive power of invariant and equivariant graph neural networks. In International Conference on Machine Learning , 2021.

László Lovász. Random walks on graphs. Combinatorics, Paul erdos is eighty , 2(1-46):4, 1993.

Roee David and Uriel Feige. Random walks with the minimum degree local rule have o(nˆ2) cover time. SIAM Journal on Computing , 47(3):755-768, 2018.

Ming Chen, Zhewei Wei, Zengfeng Huang, Bolin Ding, and Yaliang Li. Simple and deep graph convolutional networks. In International conference on machine learning , pages 1725-1735. PMLR, 2020.

Kenta Oono and Taiji Suzuki. Graph neural networks exponentially lose expressive power for node classification. In International Conference on Learning Representations , 2020.

Sepp Hochreiter and Jürgen Schmidhuber. Long short-term memory. Neural computation , 9(8): 1735-1780, 1997.

Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. Attention is all you need. Advances in neural information processing systems , 30, 2017.

Benjamin Bloem-Reddy and Yee Whye Teh. Probabilistic symmetries and invariant neural networks. Journal of Machine Learning Research , 21(90):1-61, 2020.

Ryan L Murphy, Balasubramaniam Srinivasan, Vinayak Rao, and Bruno Ribeiro. Janossy pooling: Learning deep permutation-invariant functions for variable-size inputs. In International Conference on Learning Representations , 2018.

Ryan Murphy, Balasubramaniam Srinivasan, Vinayak Rao, and Bruno Ribeiro. Relational pooling for graph representations. In International Conference on Machine Learning , pages 4663-4673. PMLR, 2019.

Vijay Prakash Dwivedi, Ladislav Rampášek, Michael Galkin, Ali Parviz, Guy Wolf, Anh Tuan Luu, and Dominique Beaini. Long range graph benchmark. In Advances in Neural Information Processing Systems , 2022.

Zhenqin Wu, Bharath Ramsundar, Evan N Feinberg, Joseph Gomes, Caleb Geniesse, Aneesh S Pappu, Karl Leswing, and Vijay Pande. Moleculenet: a benchmark for molecular machine learning. Chemical science , 9(2):513-530, 2018.

Tim Kucera, Carlos Oliver, Dexiong Chen, and Karsten Borgwardt. Proteinshake: Building datasets and benchmarks for deep learning on protein structures. In Advances in Neural Information Processing Systems , 2023.

Weihua Hu, Matthias Fey, Marinka Zitnik, Yuxiao Dong, Hongyu Ren, Bowen Liu, Michele Catasta, and Jure Leskovec. Open graph benchmark: Datasets for machine learning on graphs. Advances in neural information processing systems , 33:22118-22133, 2020.

David Weininger. Smiles, a chemical language and information system. 1. introduction to methodology and encoding rules. Journal of chemical information and computer sciences , 28 (1):31-36, 1988.

Thomas N. Kipf and Max Welling. Semi-supervised classification with graph convolutional networks. In International Conference on Learning Representations , 2017.

David Rogers and Mathew Hahn. Extended-connectivity fingerprints. Journal of chemical information and modeling , 50(5):742-754, 2010.

Kyunghyun Cho, Bart Van Merriënboer, Caglar Gulcehre, Dzmitry Bahdanau, Fethi Bougares, Holger Schwenk, and Yoshua Bengio. Learning phrase representations using rnn encoderdecoder for statistical machine translation. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing , 2014.

Christopher Morris, Martin Ritzert, Matthias Fey, William L Hamilton, Jan Eric Lenssen, Gaurav Rattan, and Martin Grohe. Weisfeiler and leman go neural: Higher-order graph neural networks. In Proceedings of the AAAI conference on artificial intelligence , volume 33, pages 4602-4609, 2019.

Christopher Morris, Gaurav Rattan, and Petra Mutzel. Weisfeiler and leman go sparse: Towards scalable higher-order graph embeddings. Advances in Neural Information Processing Systems , 33:21824-21840, 2020.

Nils M Kriege. Weisfeiler and leman go walking: Random walk kernels revisited. Advances in Neural Information Processing Systems , 35:20119-20132, 2022.

Diederik Kingma and Jimmy Ba. Adam: A method for stochastic optimization. In International Conference on Learning Representations , 2015.

Luis Müller and Christopher Morris. Aligning transformers with weisfeiler-leman. In International Conference on Learning Representations , 2024.

Luis Müller, Daniel Kusuma, Blai Bonet, and Christopher Morris. Towards principled graph transformers. Advances in Neural Information Processing Systems , 37:126767-126801, 2024.

Caterina Graziani, Tamara Drucks, Fabian Jogl, Monica Bianchini, Franco Scarselli, T Gartner, et al. The expressive power of path-based graph neural networks. In International Conference on Machine Learning , volume 235, pages 16226-16249. ML Research Press, 2024.

NeurIPS Paper Checklist

The checklist is designed to encourage best practices for responsible machine learning research, addressing issues of reproducibility, transparency, research ethics, and societal impact. Do not remove the checklist: The papers not including the checklist will be desk rejected. The checklist should follow the references and follow the (optional) supplemental material. The checklist does NOT count towards the page limit.

Please read the checklist guidelines carefully for information on how to answer these questions. For each question in the checklist:

You should answer [Yes] , [No] , or [NA] .

[NA] means either that the question is Not Applicable for that particular paper or the relevant information is Not Available.

Please provide a short (1-2 sentence) justification right after your answer (even for NA).

The checklist answers are an integral part of your paper submission. They are visible to the reviewers, area chairs, senior area chairs, and ethics reviewers. You will be asked to also include it (after eventual revisions) with the final version of your paper, and its final version will be published with the paper.

The reviewers of your paper will be asked to use the checklist as one of the factors in their evaluation. While "[Yes] " is generally preferable to "[No] ", it is perfectly acceptable to answer "[No] " provided a proper justification is given (e.g., "error bars are not reported because it would be too computationally expensive" or "we were unable to find the license for the dataset we used"). In general, answering "[No] " or "[NA] " is not grounds for rejection. While the questions are phrased in a binary way, we acknowledge that the true answer is often more nuanced, so please just use your best judgment and write a justification to elaborate. All supporting evidence can appear either in the main paper or the supplemental material, provided in appendix. If you answer [Yes] to a question, in the justification please point to the section(s) where related material for the question can be found.

IMPORTANT, please:

1. Claims

Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?

Answer: [Yes]

Justification: Yes, they are properly reflected.

Guidelines:

The answer NA means that the abstract and introduction do not include the claims made in the paper.

The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.

The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.

It is fine to include aspirational goals as motivation as long as it is clear that these goals are not attained by the paper.

2. Limitations

Question: Does the paper discuss the limitations of the work performed by the authors?

Answer: [Yes]

Justification: Yes, we discuss limitations of the work in Section 6.

Guidelines:

The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.

The authors are encouraged to create a separate "Limitations" section in their paper.

The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.

The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with a few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.

The authors should reflect on the factors that influence the performance of the approach. For example, a facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or a speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.

The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.

If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.

While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an important role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.

3. Theory assumptions and proofs

Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?

Answer: [Yes]

Justification: Yes, all theoretical results are clearly stated and we place all the mathematical proofs in the Appendix.

Guidelines:

The answer NA means that the paper does not include theoretical results.

All the theorems, formulas, and proofs in the paper should be numbered and crossreferenced.

All assumptions should be clearly stated or referenced in the statement of any theorems.

The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch to provide intuition.

Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.

Theorems and Lemmas that the proof relies upon should be properly referenced.

4. Experimental result reproducibility

Question: Does the paper fully disclose all the information needed to reproduce the main experimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?

Answer: [Yes]

Justification: All experimental results are described in the main paper and the Appendix

Guidelines:

The answer NA means that the paper does not include experiments.

If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.

If the contribution is a dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.

Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.

While NeurIPS does not require releasing code, the conference does require all submissions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example

(a) If the contribution is primarily a new algorithm, the paper should make it clear how to reproduce that algorithm.

(b) If the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully.

(c) If the contribution is a new model (e.g., a large language model), then there should either be a way to access this model for reproducing the results or a way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).

(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.

5. Open access to data and code

Question: Does the paper provide open access to the data and code, with sufficient instructions to faithfully reproduce the main experimental results, as described in supplemental material?

Answer: [Yes]

Justification: Yes, all data are publicly available and code is available at:

https://github.com/MLD3/RandomSearchNNs

Guidelines:

The answer NA means that paper does not include experiments requiring code.

Please see the NeurIPS code and data submission guidelines ( https://nips.cc/ public/guides/CodeSubmissionPolicy ) for more details.

While we encourage the release of code and data, we understand that this might not be possible, so 'No' is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for a new open-source benchmark).

The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines ( https: //nips.cc/public/guides/CodeSubmissionPolicy ) for more details.

The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.

The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only a subset of experiments are reproducible, they should state which ones are omitted from the script and why.

At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).

Providing as much information as possible in supplemental material (appended to the paper) is recommended, but including URLs to data and code is permitted.

6. Experimental setting/details

Question: Does the paper specify all the training and test details (e.g., data splits, hyperparameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?

Answer: [Yes]

Justification: We leave all training and test details in the main paper and the Appendix.

Guidelines:

The answer NA means that the paper does not include experiments.

The experimental setting should be presented in the core of the paper to a level of detail that is necessary to appreciate the results and make sense of them.

The full details can be provided either with the code, in appendix, or as supplemental material.

7. Experiment statistical significance

Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?

Answer: [Yes]

Justification: All error bars are explained in Section 5

Guidelines:

The answer NA means that the paper does not include experiments.

The authors should answer "Yes" if the results are accompanied by error bars, confidence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.

The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).

The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)

The assumptions made should be given (e.g., Normally distributed errors).

It should be clear whether the error bar is the standard deviation or the standard error of the mean.

It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have a 96% CI, if the hypothesis of Normality of errors is not verified.

For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative error rates).

If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.

8. Experiments compute resources

Question: For each experiment, does the paper provide sufficient information on the computer resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?

Answer: [Yes]

Justification: Yes, all details are in the Appendix.

Guidelines:

The answer NA means that the paper does not include experiments.

The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.

The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.

The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make it into the paper).

9. Code of ethics

Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https://neurips.cc/public/EthicsGuidelines ?

Answer: [Yes]

Justification: The paper conforms in every respect to the code of ethics.

Guidelines:

The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.

If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics.

The authors should make sure to preserve anonymity (e.g., if there is a special consideration due to laws or regulations in their jurisdiction).

10. Broader impacts

Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?

Answer: [NA]

Justification: Our work is a general graph representation learning framework with no positive/negative societal impacts beyond any general machine learning framework for graphs (message-passing graph neural networks, graph transformers, random walk neural networks).

Guidelines:

The answer NA means that there is no societal impact of the work performed.

If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.

Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.

The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.

The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.

If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).

11. Safeguards

Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?

Answer: [NA]

Justification: No new data or models are released with a high risk of misuse.

Guidelines:

The answer NA means that the paper poses no such risks.

Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.

Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.

We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort.

12. Licenses for existing assets

Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?

Answer: [Yes]

Justification: Yes, all assets are properly credited.

Guidelines:

The answer NA means that the paper does not use existing assets.

The authors should cite the original paper that produced the code package or dataset.

The authors should state which version of the asset is used and, if possible, include a URL.

The name of the license (e.g., CC-BY 4.0) should be included for each asset.

For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.

If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswithcode.com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of a dataset.

For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.

If this information is not available online, the authors are encouraged to reach out to the asset's creators.

13. New assets

Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?

Answer: [Yes]

Justification: All assets are properly documented.

Guidelines:

The answer NA means that the paper does not release new assets.

Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.

The paper should discuss whether and how consent was obtained from people whose asset is used.

At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.

14. Crowdsourcing and research with human subjects

Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?

Answer: [NA]

Justification: No crowdsourcing or research with human subjects was used.

Guidelines:

The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.

Including this information in the supplemental material is fine, but if the main contribution of the paper involves human subjects, then as much detail as possible should be included in the main paper.

According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.

15. Institutional review board (IRB) approvals or equivalent for research with human subjects

Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?

Answer: [NA]

Justification: All data used in this work are publicly available datasets.

Guidelines:

The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.

Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.

We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.

For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.

16. Declaration of LLM usage

Question: Does the paper describe the usage of LLMs if it is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, or formatting purposes and does not impact the core methodology, scientific rigorousness, or originality of the research, declaration is not required.

Answer: [NA]

Justification: LLMs do not impact the core methodology of the research.

Guidelines:

The answer NA means that the core method development in this research does not involve LLMs as any important, original, or non-standard components.

Please refer to our LLM policy ( https://neurips.cc/Conferences/2025/LLM ) for what should or should not be described.

A Mathematical Proofs

A.1 Random Walk Neural Network Expressive Power

We begin with the 1-Weisfeiler-Lehman (WL) color refinement, which iteratively updates each node's color by hashing its current color together with the multiset of its neighbors' colors (Section A.1.1). WL is known to upper-bound MPNN expressivity [15, 39]. We then introduce Walk Weisfeiler-Lehman (WWL), a walk-based refinement that updates a node from the multiset of colored walks of length ≤ ℓ that terminates at it (Section A.1.2). We establish monotonicity of WWL in the number of refinement rounds t , the maximum walk length ℓ , and the initialization π (0) (i.e., richer initial features yield finer partitions). We further show that WWL upper-bounds the expressive power of RWNNs (without positional/ID signals). Finally, using unfolding trees , which simultaneously captures the nodes visible to t rounds of message passing and encodes all root-terminating walks of length ≤ t , we prove the main expressivity results that unify MPNNs and RWNNs: equivalence under full coverage and strict separation under partial coverage (Section A.1.3). We provide a more detailed review of existing WL variants and their relation to WWL in Appendix E.

A.1.1 Weisfeiler-Lehman (WL)

We begin with the 1-dimensional Weisfeiler-Lehman (WL) color-refinement procedure, which upperbounds the expressive power of message-passing GNNs and, as we will show later, also upper-bounds RWNN expressive power. Intuitively, WL iteratively refines node labels by hashing each node's current label together with the multiset of its neighbors' labels.

Definition A.1 (1-WL color refinement) . Let G = ( V, E ) be an unlabeled graph and let N ( u ) denote the neighbors of u ∈ V . Initialize a coloring π (0) WL : V → Σ with a constant value (e.g., π (0) WL ( u ) = 1 for all u ). For t ≥ 0 , update

where Hash is injective and maps pairs of the form (current color, neighbor colors) to Σ . The process stabilizes at the first t ⋆ for which π ( t ⋆ ) WL = π ( t ⋆ +1) WL ; we denote the stable coloring by π ( ∞ ) WL .

To compare two graphs G and H , run 1-WL on each. If the stable color multisets differ (e.g., some color has a different node count), the graphs are certified non-isomorphic. If the stable colorings agree, the test is inconclusive (the graphs may still be non-isomorphic). For the remainder of the analysis, we write α ⪯ β for node-level colorings α, β : V → Σ to mean that β refines α : if β ( u ) = β ( v ) then α ( u ) = α ( v ) . These notions coincide with graph-level distinguishability: applying an injective readout on multiset colors α and β for α ⪯ β yields graph-level functions f α and f β such that f α ⪯ f β . Thus, distinguishability at the node-level translates to distinguishability at the graph-level.

WLhas been used to quantify the expressive power of MPNNs. For any standard message-passing NN, its expressive power is no greater than that of WL. Moreover, if its aggregation function is injective on the multisets of node neighbors, its expressive power matches that of WL.

Lemma A.2 (MPNN vs. 1-WL Expressivity [15, 39]) . Let f MPNN be a MPNN with a permutationinvariant readout, and let π WL denote the 1-WL coloring. Then

Moreover, if the multiset aggregator f agg used by f MPNN is injective, then

A.1.2 Walk Weisfeiler-Lehman (WWL)

Building on WL, we now align Weisfeiler-Lehman to random walk models by defining a node-level WWLscheme that refines a node's label from the multiset of colored walks incident to it.

Definition A.3 (WWL at length ℓ ) . Let G = ( V, E ) be a graph and ℓ ∈ N . For L ≥ 1 , let

be the set of lengthL walks, and write W ≤ ℓ = ⋃ ℓ ℓ =1 W L , the union of all walks of length ≤ ℓ . For a node u ∈ V , define its terminating-walk neighborhood

Given an initial coloring π (0) : V → Σ (e.g., uniform or from node features), define for any walk col ( t ) WWL ℓ ( W ) = ( π ( t ) WWL ℓ ( w 0 ) , . . . , π ( t ) WWL ℓ ( w L ) ) , the colored walk obtained by applying π ( t ) WWL ℓ to each node in the walk. The WWL ℓ update is, for all u ∈ V ,

Lemma A.4 (Monotonicity in t , ℓ , and π 0 ) . Fix ℓ ≤ ℓ ′ , t ≤ t ′ , and initial colorings π 0 ⪯ π ′ 0 . Then

Consequently, combining each result yields π ( t ) WWL ℓ ( π 0 ) ⪯ π ( t ′ ) WWL ℓ ′ ( π ′ 0 ) for t ≤ t ′ , ℓ ≤ ℓ ′ , π 0 ⪯ π ′ 0 .

Proof. Monotonicity in t . At each step, π ( t +1) WWL ℓ ( u ) = Hash ( π ( t ) WWL ℓ ( u ) , · ) includes the current color as an input. By injectivity of Hash , if two nodes receive the same new color then they had the same current color. Thus π ( t ) WWL ℓ ⪯ π ( t +1) WWL ℓ , and induction gives the stated inequality for t ≤ t ′ .

Monotonicity in ℓ . Let ℓ ≤ ℓ ′ . For each node u , the multiset of colored terminating walks of lengths ≤ ℓ is obtained from the corresponding multiset for lengths ≤ ℓ ′ by the projection that discards all walks of length > ℓ . Therefore, if two nodes are equal under WWL ℓ ′ , they are equal under WWL ℓ as well. Injectivity of Hash yields π ( t ) WWL ℓ ⪯ π ( t ) WWL ℓ ′ .

Monotonicity in the initialization π 0 . Assume π 0 ⪯ π ′ 0 . Then there exists a color-forgetting map ρ with π 0 = ρ ◦ π ′ 0 . Apply ρ pointwise to every color in each colored walk: for any terminating walk W = ( w 0 , . . . , w L ) ,

( π 0 ( w 0 ) , . . . , π 0 ( w L )) = ( ρ ( π ′ 0 ( w 0 )) , . . . , ρ ( π ′ 0 ( w L )) ) . Hence the multiset of π 0 -colored walks at any node is the image, under this deterministic transformation, of the multiset of π ′ 0 -colored walks. Consequently, equality of the π ′ 0 -based walk multisets implies equality of the π 0 -based walk multisets, and injectivity of Hash gives π (1) WWL ℓ ( π 0 ) ⪯ π (1) WWL ℓ ( π ′ 0 ) . The same argument iterates, since each WWL round recomputes colors from the previous round's coloring via the same construction, yielding π ( t ) WWL ℓ ( π 0 ) ⪯ π ( t ) WWL ℓ ( π ′ 0 ) for all t .

The following lemma is an analogue to expressive results on MPNNs and 1-WL. Intuitively, WWL ℓ upper bounds RWNN expressivity, and RWNNs can match WWL ℓ if their aggregator is injective.

Lemma A.5 (RWNN vs. WWL ℓ Expressivity) . Let f ℓ RWNN be a random walk neural network that, for each node u , aggregates over the multiset of all terminating walks of lengths 1 , . . . , ℓ ending at u , via a permutation-invariant aggregator and a sequence encoder applied to each walk. Let π WWL ℓ denote the WWL ℓ coloring.

( Upper bound ) For any choice of encoders/aggregators,

That is, if two graphs are indistinguishable by WWL ℓ , they are indistinguishable by f ℓ RWNN .

( Tightness under injectivity ) Suppose the sequence encoder f seq is injective on length-aware color sequences and the nodewise multiset aggregator f agg is injective. Then

Proof. (Upper bound). We prove by induction on t that π ( t ) WWL ℓ ( u ) = π ( t ) WWL ℓ ( v ) implies f ℓ,t RWNN ( u ) = f ℓ,t RWNN ( v ) .

Base case t = 0 . Both procedures start from the same initialization (e.g., uniform or fixed features), so the claim holds trivially.

Inductive step. Assume the claim holds at depth t . Take u, v with π ( t +1) WWL ℓ ( u ) = π ( t +1) WWL ℓ ( v ) . By injectivity of the WWL hash, the entire inputs to the hash coincide, hence

where each π ( t ) WWL ℓ ( W ) is the length-aware color sequence along the walk W . Thus there is a bijection between terminating walks at u and v that preserves these sequences. By the induction hypothesis, matched nodes with equal WWL color at round t have equal RWNN representations at depth t . Therefore, for each matched walk pair, the inputs to the per-walk sequence encoder f seq agree elementwise, so per-walk encodings match; applying the same permutation-invariant multiset aggregator f agg yields f ℓ,t +1 RWNN ( u ) = f ℓ,t +1 RWNN ( v ) . This completes the induction and the upper bound.

̸

(Equivalence under injectivity). Assume f seq is injective on length-aware sequences and f agg is injective on multisets. Let u, v satisfy π ( t +1) WWL ℓ ( u ) = π ( t +1) WWL ℓ ( v ) . By injectivity of the WWL hash, either their current colors at round t differ, or their multisets { { π ( t ) WWL ℓ ( W ) : W ∈ W ≤ ℓ ( · ) } } differ. In the first case, including (an injective transform of) the current node state in the RWNN update separates u and v . In the second case, there is no bijection between the two multisets of colored sequences; since f seq is injective on sequences and f agg is injective on multisets, the aggregated RWNN representations must differ at round t +1 . Combining with the upper bound, we conclude f ℓ,t RWNN ≃ π ( t ) WWL ℓ for all t .

A.1.3 RWNN-MPNN Equivalence Under Full Coverage (Theorem 3.1, Corollary 3.2)

Unfolding Trees. We introduce the unfolding tree from Morris et al. [40], Kriege [41], which makes explicit the bridge between Weisfeiler-Lehman (WL) refinement and random walks. For a node u in G , the unfolding tree at depth ℓ enumerates, with multiplicities, all vertices seen by successive layers of message passing around u . Equivalently, every leaf-to-root path in the unfolding tree corresponds to a walk in G that terminates at u . Hence the unfolding tree simultaneously encodes (i) all messages propagated in a message-passing view and (ii) all terminating walks of length ≤ ℓ at u . We will leverage this structure to relate the expressive power of WL and WWL.

Definition A.6 (Unfolding tree [40, 41]) . Let G = ( V, E ) be a graph, ℓ ∈ N , and u ∈ V . The unfolding tree of depth ℓ rooted at u , denoted T G [ ℓ, u ] , is the rooted tree defined recursively as follows:

T G [0 , u ] consists of a single root node labeled by u .

For ℓ ≥ 1 , T G [ ℓ, u ] has root labeled by u ; for each neighbor v ∈ N G ( u ) , attach as a child a fresh copy of T G [ ℓ -1 , v ] .

The first key fact ties WL colors to unfolding trees: WL's ℓ -round color of a node is exactly the isomorphism type of its depthℓ unfolding tree.

Lemma A.7 (WL ↔ unfolding tree [41]) . Let G,H be graphs, u ∈ V ( G ) , v ∈ V ( H ) , and ℓ ≥ 1 . Then

Unfolding trees also capture terminating walks: every leaf-to-root path in T G [ ℓ, u ] reads off a unique lengthℓ walk in G ending at u , and conversely. Let

Lemma A.8 (Leaf-to-root paths ↔ terminating walks [41]) . For any u ∈ V ( G ) and ℓ ≥ 1 ,

Theorem A.9. For any graphs G,H and any ℓ ≥ 1 , WWL ℓ test has exactly the same distinguishing power as the classical 1-dimensional Weisfeiler-Lehman test. Formally,

Proof. π ( ∞ ) WL ⪯ π ( ∞ ) WWL ℓ . For ℓ = 1 , the set of terminating length-1 walks at a node u is exactly its neighbor set N ( u ) . Hence the WWL 1 update coincides with the WL update, and for every round t

By Lemma A.4 (monotonicity in ℓ ), π ( t ) WWL 1 ⪯ π ( t ) WWL ℓ for all ℓ ≥ 1 and all t . Passing to the limit,

π ( ∞ ) WWL ℓ ⪯ π ( ∞ ) WL . Initializing WWL with the WL limit, π (0) = π ( ∞ ) WL , it suffices to show that one WWLupdate makes no further splits. Fix u ∈ V ( G ) and v ∈ V ( H ) with π ( ∞ ) WL ( u ) = π ( ∞ ) WL ( v ) . By Lemma A.7, there is a root-preserving isomorphism σ : T G [ ℓ, u ] ∼ = - - → T H [ ℓ, v ] . By Lemma A.8, leaf-to-root paths in these depthℓ trees biject with the terminating walks of lengths 1 , . . . , ℓ at u and v , respectively; σ also preserves WL ∞ colors at every node of two unfolding trees. To show this, suppose for contradiction, that there exists x ∈ T G [ ℓ, u ] with π ( ∞ ) 1-WL ( x ) = π ( ∞ ) 1-WL ( σ ( x )) . Since 1-WL stabilizes in finite time on the finite disjoint union G ⊎ H , there exists a finite witness round k ⋆ ∈ N such that π ( k ⋆ ) 1-WL ( x ) = π ( k ⋆ ) 1-WL ( σ ( x )) . Let d be the distance from x to the root u in T G [ ℓ, u ] . By the 1-WL update rule, a mismatch at a node at round k ⋆ forces a mismatch at its parent at round k ⋆ +1 (the multiset of child colors differs), and inductively a mismatch at the root after d further rounds:

̸

This contradicts π ( ∞ ) 1-WL ( u ) = π ( ∞ ) 1-WL ( v ) . Hence σ must preserve 1-WL colors at every node. Consequently, the leaf-to-root paths in T G [ ℓ, u ] and T H [ ℓ, v ] correspond bijectively under σ with identical colored sequences. Thus, the corresponding multisets of WL ∞ -colored terminating-walk sequences at u and v coincide. Together with π (0) ( u ) = π (0) ( v ) , the entire inputs to the WWL hash agree at u and v , so by injectivity of Hash we obtain π (1) WWL ℓ ( π ( ∞ ) WL ) ( u ) = π (1) WWL ℓ ( π ( ∞ ) WL ) ( v ) . Hence π ( ∞ ) WL is a fixed point of WWL. By Lemma A.4, WWL is monotone in t and π 0 ; since uniform ⪯ π ( ∞ ) WL , it follows that π ( ∞ ) WWL ℓ ⪯ π ( ∞ ) WL . Combined with π ( ∞ ) WL = π ( ∞ ) WWL 1 ⪯ π ( ∞ ) WWL ℓ , we conclude π ( ∞ ) WWL ℓ ≃ π ( ∞ ) WL .

We now leverage the preceding results to prove the main expressivity statements.

Theorem A.10 (RWNN-MPNN equivalence under full coverage) . Let G be a graph. Let f FC RWNN denote an RWNN with injective f seq and f agg with no additional positional encodings, applied to the complete multiset of walks W ≤ ℓ ( G ) with lengths up to ℓ = C E ( G ) , the edge cover time of G . Let f MPNN be an MPNN with injective f agg . Then, for all graphs G,H ,

Hence, f FC RWNN ≃ f MPNN (i.e., f FC RWNN and f MPNN are equal in expressive power).

Proof. By the standard 1-WL result for message passing (Theorem A.2), an MPNN with injective aggregation satisfies f MPNN ≃ π WL . By the RWNN/WWL correspondence (Lemma A.5), a fullcoverage RWNN with injective f seq and f agg satisfies f FC RWNN ≃ π WWL ℓ . Finally, by the equivalence π WWL ℓ ≃ π WL (Theorem A.9), we conclude f FC RWNN ≃ f MPNN .

Corollary A.11 (RWNNs under partial coverage) . Let f PC RWNN be an RWNN of the same architectural class as in Theorem 3.1, but applied to a multiset of terminating walks that achieves only partial node/edge coverage of the input. Then, for all graphs G,H ,

̸

̸

and there exist non-isomorphic graphs G ̸ ∼ = H such that

̸

Hence f PC RWNN ≺ f MPNN .

Proof. Coverage monotonicity (a direct consequence of injectivity and permutation invariance of the aggregator on multisets) implies that removing walks cannot increase distinguishing power, i.e., f PC RWNN ⪯ f FC RWNN ≃ f MPNN , which yields the implication f MPNN ( G ) = f MPNN ( H ) ⇒ f PC RWNN ( G ) = f PC RWNN ( H ) . For strictness, start from two isomorphic graphs and form G by adding one isolated vertex and H by adding one pendant vertex (a new vertex attached to an existing node). Then 1-WL (hence an MPNN) distinguishes G and H . However, if f PC RWNN is applied to walk multisets that exclude all walks visiting the added vertex in both graphs, the remaining covered walks coincide, so f PC RWNN ( G ) = f PC RWNN ( H ) . Thus f PC RWNN ≺ f MPNN .

A.2 Random Search Neural Network Expressive Power (Lemma 4.1, Theorem 4.2)

We first establish a coverage lemma: for any edge e = { u, v } in a connected graph G with maximum degree d max , a randomized DFS (uniform start; i.i.d. tie-breaking) includes e in its spanning tree with probability at least 1 /d max , i.e., Pr [ e ∈ T DFS ( G ) ] ≥ 1 /d max . Building on this, we show that sampling O ( d max log | E | ) independent DFS trees suffices to achieve full edge coverage with high probability; in bounded-degree sparse graphs ( d max = O (1) and | E | = Θ( | V | ) ), this reduces to O (log | V | ) searches. Equipped with such full coverage, standard universal components, and shared anonymous integer tags, RSNNs are universal approximators on graphs in the specified family.

Lemma A.12 (Edge inclusion probability under random DFS) . Consider the following random-DFS procedure on a graph G : fix a uniform distribution over the root vertex; independently for each vertex x , draw a uniformly random permutation π x of its neighbors; run depth-first search that, upon first visiting x , explores neighbors in the order π x . Let T DFS be the resulting DFS spanning tree. For an edge e = ( u, v ) , define

Proof. Let A be the event that u is discovered by DFS before v . On A , when u is first processed, v is unvisited. The edge ( u, v ) will be taken as a tree edge iff, in the random neighbor order π u , the vertex v appears before all neighbors S u ( e ) that can lead from u to v without using e . The positions of the other neighbors of u are irrelevant: exploring any neighbor not on a path to v first cannot reach v before DFS returns to u . Since π u is a uniform permutation, the probability of this sufficient event is exactly 1 / ( τ u ( e ) + 1) . A symmetric argument on A c (i.e., when v is discovered before u ) gives the bound 1 / ( τ v ( e ) + 1) . Unconditionally,

Lemma A.13 (Logarithmic Sampling Yields Full Edge Coverage) . Let G = ( V, E ) be a connected, unweighted graph with | E | ≤ C | V | for some constant C and a bounded maximum degree d max . Let S 1 , S 2 , . . . , S m be m independent random searches sampled from G , and let T 1 , T 2 , . . . , T m be their corresponding induced spanning trees. Then, for small δ ≪ 1 , if

the union of T 1 , T 2 , . . . , T m contains every edge in E with probability at least 1 -δ .

Proof. By Lemma A.12 the probability that any edge e appears in any DFS is at least p e ≥ 1 d max . Hence, the probability that a single DFS tree does not contain e is at most 1 -p e ≤ 1 -1 d max . Since the spanning trees T 1 , T 2 , . . . , T m are sampled independently, the probability that e is missing from all m trees is at most ( 1 -1 d max ) m . By the union bound over all | E | edges, the probability that there exists at least one edge which is not covered by the union of the m trees is at most

We require this probability to be at most δ :

Taking the natural logarithm on both sides gives:

Since ln ( 1 -1 d max ) < 0 , dividing by this term (and reversing the inequality) yields

Thus, with m chosen accordingly, the union of the m spanning trees contains every edge of G with probability at least 1 -δ .

Definition A.14 (Anonymous identity and parent tags) . Let G = ( V, E, X ) be a graph and let S (1) be the first search sampled on G according to the RSNN search algorithm. Write S (1) = ( w 0 , w 1 , . . . , w T ) and let ( v (1) , v (2) , . . . , v ( n ) ) be the vertices of G ordered by their first-visit time along S (1) , i.e. v ( i ) is the i -th distinct vertex encountered in S (1) . Define the anonymous identity tag assignment τ : V → [ n ] by

We use the same tag assignment τ for all searches in the RSNN search set on G . Because S (1) is sampled in a manner equivariant to vertex relabellings, the induced random tag assignment τ is permutation-invariant in distribution . Now let S ( j ) be any search produced by the RSNN search algorithm, and write S ( j ) = ( u ( j ) 0 , u ( j ) 1 , . . . , u ( j ) T j ) . We define a parent-tag sequence as follows. For each index t , let q ( j ) t denote the vertex from which u ( j ) t is discovered by the RSNN search algorithm along a true edge of G ; when u ( j ) t is chosen as an initial vertex of a search, we set q ( j ) t := u ( j ) t . We then define

In particular, the discovery vertex q ( j ) t need not equal the previous vertex u ( j ) t -1 in the recorded sequence: S ( j ) may contain discontinuities where consecutive vertices u ( j ) t -1 and u ( j ) t are not adjacent in G . The parent tag π ( j ) t always records, in anonymous coordinates given by τ , the vertex from which the search actually arrived at u ( j ) t along a true edge of G . Thus, the collection of identity and parent tags ( τ, ( π ( j ) ) j ) encodes all edges traversed by the RSNN searches, even across such discontinuities.

Theorem A.15 (Universal Approximation by RSNNs on Sparse Graphs with Bounded Degree) . Let ϵ > 0 and let f : G → R d be any continuous graph-level function, where G is the space of sparse, unweighted graphs with | E | = O ( | V | ) , maximum degree at most d max , and size at most | V | ≤ n max . Assume f RSNN ( G ) uses (i) a universal set encoder f agg , (ii) a universal sequence encoder f seq , and (iii) anonymous identity and parent tags. Assume m satisfies Lemma A.13, so that full coverage is achieved with probability at least 1 -δ . Then, with probability at least 1 -δ there exists an RSNN configuration such that

Proof. Let S FC ( G ) be the set of search sets of size m on G . Define a target on search sets by

This ˜ f is well-defined (for any given input search set, there is a single unique output), and is permutation-invariant in the multiset argument. Because the input space (bounded-length sequences over a finite alphabet, aggregated into multisets of bounded size) is finite, the assumed universal sequence encoder and universal set aggregator can uniformly approximate ˜ f to error < ε across ⋃ G ∈G ≤ n max S FC ( G ) . Therefore, with those parameters, for any G and any random S ( G ) , and hence unconditionally Pr( ∥ f RSNN ( S ) -F ( G ) ∥ < ε ) ≥ 1 -δ .

A.3 Random Search Neural Network Invariance (Theorem 4.3, Corollary 4.4)

We next study invariance properties of RSNNs. Because RSNNs are randomized graph functions, we adopt a probabilistic notion of isomorphism invariance: if two graphs are isomorphic, the distributions of RSNN outputs coincide. As a consequence, the expected predictor Φ( G ) = E [ f RSNN ( G )] is an isomorphism-invariant graph function. Moreover, RSNNs learn this invariance via stochastic training: sampling a fresh search per step yields an unbiased gradient of the invariant risk, and under standard SGD conditions the parameters converge to a stationary point of the invariant objective. In practice, this justifies sampling with a small number of searches (e.g., m =1 ) in limited budget regimes.

Theorem A.16 (Isomorphism-Invariance of RSNN) . A randomized search procedure on a graph G produces a sequence S G = ( s G 0 , . . . , s G | V ( G ) | ) of visited vertices. We say the procedure is probabilistically invariant to graph isomorphisms if,

The randomized DFS procedure used in RSNNs satisfies the above definition. Hence, RSNNs satisfy probabilistic invariance: for all G ∼ = H , f RSNN ( G ) d = f RSNN ( H ) , and the averaged predictor Φ( G ) := E [ f RSNN ( G ) ] is an invariant function on graphs: Φ( G ) = Φ( H ) for all G ∼ = H .

Proof. Write X DFS ( G ) = ( s 0 , . . . , s | V |-1 ) for the vertex sequence produced by the randomized DFS on G , and let H = π · G for an isomorphism π : V ( G ) → V ( H ) . The randomness comes from: (i) the root s 0 ∼ Unif( V ( G )) and (ii) an independent random order of neighbors at each vertex.

We prove by induction on t that the next state has the same pushforward conditional law under any isomorphism π :

π ( X DFS ( G )[ t ] ∣ ∣ x ) d = X DFS ( H )[ t ] ∣ ∣ π x , (10) for every valid DFS prefix x = ( s 0 , . . . , s t -1 ) on G (and its image π x on H ). Averaging over prefixes then yields πX DFS ( G )[ t ] d = X DFS ( H )[ t ] for each t , and thus πX DFS ( G ) d = X DFS ( H ) .

State, admissible set, and frontier. For a prefix x valid on G , let V vis ( G ; x ) = { s 0 , . . . , s t -1 } be the visited set and let top( G ; x ) be the current DFS stack top (the vertex whose adjacency list is being explored). Define the admissible neighbor set

̸

If A ( G ; x ) = ∅ , the rule 'pick the unvisited neighbor at random' makes the next vertex s t uniform on A ( G ; x ) . If A ( G ; x ) = ∅ , the next move is the (deterministic) backtrack to the parent of top( G ; x ) in the current DFS tree. Under an isomorphism π : G ∼ = H , relabeling preserves these invariants:

Base case ( t = 0 ). s 0 ∼ Unif( V ( G )) and πs 0 ∼ Unif( V ( H )) , so

Induction step. Assume πX DFS ( G )[: t ] d = X DFS ( H )[: t ] . Fix any realization x of the prefix on G . There are two cases.

̸

(i) Expansion step: A ( G ; x ) = ∅ . Conditioned on x , X DFS ( G )[ t ] is uniform on A ( G ; x ) . Conditioned on π x , X DFS ( H )[ t ] is uniform on A ( H ; π x ) = πA ( G ; x ) . Pushing the uniform measure on A ( G ; x ) forward by π yields the uniform measure on πA ( G ; x ) , hence

π ( X DFS ( G )[ t ] ∣ ∣ x ) = X DFS ( H )[ t ] ∣ ∣ π x , In both cases, the conditional laws match after applying π . Taking expectations over the distributions gives πX DFS ( G )[ t ] d = X DFS ( H )[ t ] for each t , which completes the induction and yields

(ii) Backtrack step: A ( G ; x ) = ∅ . The next state is the parent of top( G ; x ) in the DFS tree determined by x ; thus it is deterministic given x . Relabeling preserves parent/child relations in the explored DFS tree, so

This proves probabilistic invariance of the randomized DFS. Since the RSNN output f RSNN is a deterministic function of the search sequence, it follows that f RSNN ( G ) d = f RSNN ( H ) , and the averaged predictor Φ( G ) = E [ f RSNN ( G )] is an invariant graph function.

Corollary A.17 (Stochastic training converges to the invariant objective) . Let ℓ ( · , y ) be a differentiable loss. Consider the expected risk

At each SGD step t , sample ( G t , y t ) ∼ D and one search draw S t ∼ S DFS ( G t ) , and update

Then E [ ∇ W ℓ ( f RSNN ( G t , S t ; W t ) , y t ) ] = ∇ W L ( W t ) , i.e., the single-sample gradient is an unbiased estimator of the invariant objective's gradient. Under standard SGD conditions, W t converges almost surely to the optimal W ⋆ of the invariant objective.

Proof sketch. This follows directly from the proof of Proposition A.1 in Murphy et al. [29], replacing permutations by RSNN searches: since the search randomness S ∼S DFS ( G ) is sampled independently of W and ℓ is differentiable with integrable gradient, we can exchange ∇ W and the expectations to get ∇ W L ( W ) = E ( G,y ) ∼D E S ∼S DFS ( G ) [ ∇ W ℓ ( f RSNN ( G,S ; W ) , y ) ] , so the single-sample stochastic gradient is unbiased; standard Robbins-Monro/Polyak supermartingale arguments then yield a.s. convergence of SGD to a stationary point (and to W ⋆ under convexity).

B Additional Model Details: Positional Encodings and Sampling Algorithms

In this section, we provide additional details on the positional encoding scheme and sampling algorithms used in both RSNN and RWNN models. These components are essential not only for implementation but also for theoretical expressivity. We also present detailed descriptions of the sampling procedures for both random walks and random searches. For RWNNs, we outline the walk generation algorithm, including initialization, neighbor selection, and PE encoding. For RSNNs, we describe the random depth-first search strategy, including how spanning trees are constructed and how node visitation is handled.

B.1 Positional Encodings

Identity and Adjacency Encodings. Tönshoff et al. [5] and Chen et al. [6] augment each walk with two binary feature matrices that inject explicit structural context. For a walk W = ( w 0 , . . . , w ℓ ) on graph G , the identity encoding id s W ∈ { 0 , 1 } ( ℓ +1) × s marks node repetitions within a sliding window of size s : for indices 0 ≤ i ≤ ℓ and 0 ≤ j ≤ s -1 we set

and 0 otherwise. Thus column j signals whether the current node re-appeared exactly j steps earlier, explicitly encoding cycles of length j +1 . Second, the adjacency encoding adj s W ∈ { 0 , 1 } ( ℓ +1) × ( s -1) records edges among already-visited nodes that the walk does not traverse. We define

and 0 otherwise. Here, E ( · ) denotes the edge set of the input. Consequently, for every pair of nodes that appears within the window, the encoding reveals whether they are adjacent in the underlying graph. The two blocks are concatenated to form a positional-encoding matrix h PE = [ id s W ∥ adj s W ] ∈ R ( ℓ +1) × d pe with d pe = 2 s -1 . Appending h PE to the raw node embeddings ensures that, once full node- and edge-coverage is achieved, the sequence model receives enough information to reconstruct the entire subgraph induced by the walk.

Anonymous Encodings. As an alternative to the identity-adjacency scheme, anonymous encodings have been proposed to capture graph structure by Wang and Cho [3] and Kim et al. [7]. For a walk W we create an integer vector ω anon ( W ) ∈ { 1 , . . . , ℓ +1 } ℓ +1 defined recursively:

In words, the first time a node appears in the walk it is assigned the next unused label 1 , 2 , 3 , . . . ; every subsequent visit to that same node reuses the original label. Hence ω anon is invariant to the specific node IDs yet records the order in which unique vertices are discovered, providing topological context without relying on absolute labels.

Role in Expressivity. These positional encodings play a critical role in the expressive power of both RWNNs and RSNNs. They serve as the main mechanism by which the walk or search encodes structural information from the underlying graph. In particular, the identity and anonymous encodings, when combined with walks that achieve full edge coverage, allow for exact reconstruction of the input graph. Meanwhile, adjacency encodings with sufficient window size and identity/parent tags introduced in Definition A.14 enable full reconstruction even with only node coverage, as they record structural edges not explicitly traversed in the sequence. In our RSNN implementation, we rely on adjacency encodings. These are especially important for preserving expressivity in RSNNs: depth-first searches introduce disconnections in the sequence, where jumps between non-adjacent nodes may obscure structure. Consider for example nodes w i and w i +1 traversed adjacent to one another in a search sequence, but disconnected in the graph. With an appropriate window size, the adjacency encoding first signals the disconnection setting adj s W [ i + 1 , 1] = 0 , then identifies the connecting edge when it appeared in the sequence setting adj s W [ i +1 , j ] = 1 for ( w i , w i -j ) ∈ E ( G ) . This ensures that, once full edge coverage is achieved across searches, the sequence model receives all structural information necessary to reconstruct the graph. Thus, positional encodings are central to the theoretical guarantees of RSNN expressivity.

B.2 Sampling Algorithms

Random Walk Sampling. We adopt a standard uniform random walk procedure to extract sequences from a graph (Algorithm 1). The algorithm begins by uniformly sampling a starting node from the vertex set. At each step, it selects the next node uniformly at random from the current node's neighbors. As the walk progresses, we maintain a sliding window of fixed size s to compute identity and adjacency encodings for each step. The algorithm takes as input the graph G , walk length l , and window size s , and returns both the walk and the corresponding structural encodings.

Random Search Sampling. We implement random searches in RSNNs using a randomized depthfirst search (DFS) traversal (Algorithm 2). The algorithm begins by sampling a starting node uniformly at random from the vertex set. From there, we perform a standard DFS, visiting each neighbor in a random order to introduce stochasticity. As nodes are visited, they are recorded sequentially in the walk W , and only the adjacency-based positional encoding adj s W is computed using a sliding window of size s . Since DFS visits each node exactly once, identity encodings are unnecessary. The resulting walk and adjacency encoding together define the structural input for RSNNs.

C Extended Results

We present two additional experiments to complement our main findings. First, we conduct an ablation study evaluating the impact of the sequence model architecture on performance by comparing CRAWL, the best performing RWNN, and RSNNs equipped with GRUs, LSTMs, and Transformers on molecular benchmarks. This experiment helps assess whether the RSNN framework is sensitive to the choice of sequence model. Second, we report runtime comparisons between RSNNs and RWNNs to evaluate computational efficiency. Specifically, we compare training times across varying sample sizes to understand how the two approaches scale under realistic computational budgets.

C.1 Sequence Model Ablations

We evaluate the impact of sequence model architecture by comparing RSNNs and CRAWL equipped with GRUs, LSTMs, and Transformers (Table 1). Across all configurations, the trends from the main paper hold: RSNNs consistently outperform RWNNs at low sample sizes ( m = 1 ), regardless of sequence model. Notably, RSNNs with m = 1 often match or exceed the performance of RWNNs with m = 16 , reaffirming the sample efficiency advantages of random search. When m = 16 on the BACE dataset, CRAWL-LSTM and CRAWL-GRU slightly outperform their RSNN counterparts, however in the remaining comparisons RSNN always outperforms CRAWL across all m . Overall, GRUs and LSTMs perform comparably within both RSNN and RWNN variants, indicating that RSNN improvements are robust to the choice of sequence model, provided it has adequate recurrence-based inductive bias. In contrast, Transformers underperform relative to GRUs and LSTMs across most benchmarks and sample sizes. One possible explanation is that Transformers lack the hard-coded recurrence structure present in GRUs and LSTMs, relying instead on global attention mechanisms

that may require more data to model sequential dependencies effectively, especially in low-sample regimes. These results suggest that recurrent sequence models are better suited for graph-based walk or search processing under constrained sampling budgets.

(a) SIDER

(b) BBBP

(c) TOXCAST

(seconds)

RSNN

(seconds)

400

RSNN

Runtime(seconds)

300

RSNN

400

CRAWL

CRAWL

CRAWL

Runtime

Runtime

300

200

200

200

100

100

1

4

8

16

1

4

8

16

1

4

8

16

Number ofWalks/Searches(m)

Number ofWalks/Searches(m)

NumberofWalks/Searches(m)

C.2 Runtime Comparisons

Experimental Setup. To ensure a fair comparison between RSNNs and CRAWL, we fix all model components except for the sampling strategy. Both models use a GRU sequence model with hidden dimension 64 and are composed of 2 layers. We set the positional encoding window size to 8 and batch size to 64. For each graph, the random walk length is set to l = | V | , equal to the number of nodes, so that the number of sequence steps is identical between random walks and searches. As a result, RSNNs and CRAWL have equivalent asymptotic runtimes per sample. We measure training runtime over 25 epochs on three molecular benchmarks, SIDER , BBBP , and TOXCAST , across varying sample sizes m ∈ { 1 , 4 , 8 , 16 } . For each forward pass, a fresh set of m walks or searches is sampled per graph. All experiments are run on a single NVIDIA GeForce GTX 1080 Ti GPU, with sampling parallelized across 4 CPU workers to reflect practical deployment conditions.

Results. Empirically we observe that RSNN searches are never more expensive than CRAWL walks for any tested number of walks m , and that for larger m the RSNN implementation can even become faster (Figure 4). Although, each routine shares the same asymptotic cost, O ( | V | ) on our sparse graphs, they differ by constant factors that affect runtime comparisons in practice:

Random Walks Revisit Nodes with Larger Degrees. A DFS visits each vertex exactly once, while a random walk visits nodes randomly, potentially revisiting many vertices with higher degrees. Consequently, searches and random walks visit different sets of nodes. This affects runtime since operations per node depend on their degrees (e.g., shuffling neighbors, random choices on neighbors, identity/adjacency checks), incurring more computation per-step and increasing runtimes for random walks.

Per-step work. The DFS runs one for s loop that updates a single adjacency-encoding tensor. The RWNN walk performs an identical for s loop, but each iteration evaluates two conditions (identity & adjacency encoding) and writes to two tensors, effectively doubling the cost of that inner loop per step.

Neighbor handling. DFS shuffles the neighbor list once per new vertex, whereas random walks rebuilds a Python list and calls random.choice at every step, and if non-backtracking is enabled, creates an additional filtered list. These repeated list allocations and Python-level random picks inflate wall time.

Together, these constant-factor differences explain why the asymptotically identical O ( | V | ) algorithms show distinct wall-time profiles: RSNN remains competitive for all m , while CRAWL exhibit longer runtimes at larger m .

D Experimental Details and Code

Training and Hyperparameter Selection. All models are trained by minimizing the binary crossentropy loss on molecular benchmarks and the negative log-likelihood loss on protein benchmarks. Training is performed for a maximum of 200 epochs with early stopping patience set to 25 epochs based on validation performance. The best-performing model on the validation set is selected for evaluation on the test set. We perform a grid search over the following hyperparameters for all RWNN and RSNN models:

Number of layers: {1, 2, 3}

Learning rate: {0.05, 0.01, 0.005, 0.001}

Batch size: {32, 64, 128}

Hidden dimension: {32, 64, 128}

Global pooling: { mean , sum , max }

Sequence model: { GRU , LSTM , Transformer }

Number of samples m : {1, 4, 8, 16}

We fix the window size s = 8 for both CRAWL and RSNN models. All models are optimized using the Adam optimizer [42].

E Extended Discussion

Background on WL and its Variants. The Weisfeiler-Lehman (WL) hierarchy has become a standard lens for characterizing graph model expressivity. Xu et al. [15] first established the equivalence between 1-dimensional WL and MPNNs, while Morris et al. [39], Azizian and Lelarge [21] generalized this perspective to higher-order GNNs via higher-order WL variants. Beyond MPNNs, recent work has aligned graph transformers with WL, clarifying their expressivity within the same hierarchy [43, 44]. In parallel, random walk kernels and path GNNs have been connected to WLas sequence-based representations [41, 45].

Our Walk Weisfeiler-Lehman (WWL) refinement builds directly on this line: we introduce a walkbased color refinement and show that, under full coverage, its distinguishing power coincides with 1-WL. In doing so, we place RWNNs firmly within the WL-centered expressivity landscape alongside MPNNs, graph transformers, and path-based GNNs, advancing a unified view of diverse graph learning architectures through the WL hierarchy.

Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper introduces Random Search Neural Networks (RSNNs), a novel graph learning framework designed to overcome the expressive limitations of existing Random Walk Neural Networks (RWNNs). The authors argue that under realistic sampling constraints, RWNNs often fail to achieve complete node and edge coverage of a graph, thereby limiting their ability to capture global structure. To address this, RSNNs replace the random walk sampling strategy with a random search procedure, specifically depth-first search (DFS). \n\nThe core contributions of the paper are both theoretical and empirical:\n\n1. Theoretical Analysis: The article theoretically proves that the RSNNs are more expressive and have tighter generalization.\n\n2. Empirical Validation: Through experiments on molecular and protein graph classification benchmarks, RSNNs are shown to consistently and significantly outperform various RWNN baselines.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n\nImpressive Sample Efficiency: The empirical results demonstrate the practical value of RSNNs. The finding that an RSNN with a single search (m=1) can match or outperform the best RWNN with 16 walks (m=16) on several benchmarks is a standout result that underscores the method's efficiency.\n\nStrong Theoretical Foundation: The proof that partially-covered RWNNs are less expressive than subgraph GNNs (Theorem 3.1) provides a motivation for the work. The subsequent theorems establishing the efficient coverage (Lemma 4.1), universal approximation capability (Theorem 4.2), and tighter generalization bounds (Theorem 4.3) of RSNNs provide a solid theoretical underpinning for the proposed method.\n\nWeaknesses:\n\nScalability to Dense Graphs: The primary weakness is the reliance on a full depth-first search. While efficient on the sparse graphs targeted in the paper, the O(|V|+|E|) complexity of DFS would be prohibitive for very large or dense graphs. This limits the immediate applicability of the current RSNN framework to a subset of all possible graph structures.\n\nQUESTIONS:\n1. For the experiments, the RWNN walk length $I$ was set to |V| to ensure equivalent asymptotic runtimes with DFS. However, RWNNs are often used with much shorter walks [1]. So, RWNN walk length set to |V| may increase time complexity, but brings a slight performance improvement. Could the authors provide a comparison against RWNNs in this more conventional, computationally cheaper setting? This would offer a more direct view of the trade-off between computational cost, coverage, and performance.\n\n2. The article theoretically proves Tighter Learning Guarantees in RSNNs. Why not provide experimental evidence to further confirm this?\n\n3. RWNN has poor expressiveness due to limited coverage, so you compare it with subgraph GNN in Section 3.1. I want to know why the subgraph GNN is not used as the baseline model in the experiment.\n\nReference:\n\n[1] Nikolentzos, Giannis, and Michalis Vazirgiannis. \"Random walk graph neural networks.\" Advances in Neural Information Processing Systems 33 (2020): 16211-16222.\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 2, "CLARITY": 2, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-06-30"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes RSNN, a graph representation learning method based on depth-first search (DFS). To address the limited expressivity of RWNNs under realistic sampling constraints—due to insufficient node and edge coverage—RSNN generates spanning trees via DFS to achieve full graph coverage with significantly lower sampling cost. Theoretical and empirical results demonstrate that RSNN offers stronger expressivity and better performance, especially on sparse graphs such as molecular and protein datasets.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n\n1. Novel analytical perspective: The paper analyzes model expressivity from the perspective of node and edge coverage, providing a new angle for understanding the limitations of RWNNs.\n\n2. Theoretical support: It offers solid theoretical results, showing how full coverage improves RWNN performance and establishing universal approximation and generalization guarantees for RSNNs.\n\n3. The paper is clearly written and well-structured, making the motivation, methodology, and analysis easy to follow.\n\n4. Experiments on molecular and protein benchmarks demonstrate consistent and meaningful improvements over RWNNs, using fewer sampled sequences.\n\nWeaknesses:\n\n1. The experimental evaluation focuses mainly on small and relatively sparse datasets (molecular and protein graphs), leaving it unclear how well the proposed method generalizes to larger or denser graphs.\n\n2. Some technical details of the method are presented at a high level, and certain design choices (e.g., how DFS is implemented or integrated with the sequence model) are not clearly explained, making it harder to assess reproducibility and implementation complexity.\n\nQUESTIONS:\n1. The experiments focus on sparse molecular and protein graphs. How does RSNN perform on larger or denser graphs? Additional results or discussion would clarify its generality.\n\n2. Since subgraph GNNs are discussed theoretically, can the authors add empirical comparisons with such methods to better position RSNN?\n\n3. The paper lacks detailed explanation of methods, such as the DFS sampling and spanning tree encoding. Please clarify or provide pseudo-code to improve reproducibility.\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-01"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper presents random search neural networks (RSNNs), which operate on random searches to provide a more efficient, expressive, and generalizable model compared to random walk neural networks (RWNNs). The authors provide a series of experimental results and theoretical justifications on the advantages of RSNNs.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n- The paper is well written and easy to follow\n- Experimental results support the claim that RSNNs improve performance across several different benchmarking tasks. Inclusion of traditional molecular fingerprinting methods provide a holistic comparison for molecular models.\n- The authors provide clear theoretical analysis for RSNN - Theorem 4.2 in particular provides an interesting result.\n\nWeaknesses:\n- Although the authors present a compelling argument that RSNNs are more efficient in achieving full coverage of a graph, the following results linking higher coverage to increased expressivity seem somewhat trivial. Especially since most of the results are related to graph-level tasks, it is obvious that seeing more of the input graph would lead to better performance.\n  - Theorem 3.1 also seems somewhat trivial given a recent result in Bao et al., 2025 (Theorem 4.6) which already proves that motif/homomorphism encodings (which are directly related to subgraphs) are more expressive than random walk encodings. Could the authors please address how their result meaningfully builds upon this?\n\nReferences:\n\nBao et al. Homomorphism Counts as Structural Encodings for Graph Learning. In ICLR, 2025.\n\nQUESTIONS:\nSee weaknesses. Additionally:\n- Tönshoff et al., 2023 state that GNNs intuitively have an advantage over RWNN methods such as CRaWl since GNNs effectively perform a breadth-first search. Given this and the fact that a final permutation-invariant function that is applied at the end, do the authors have some intuition as to if using BFS rather than DFS would meaningfully change the performance of RSNNs?\n\nReferences:\n\nTönshoff et al. Walking out of the weisfeiler leman hierarchy: Graph learning beyond message passing. In _TMLR_, 2023.\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-07-01"}

comments: SUMMARY:
The paper compares several methods for obtaining data about graphs via neural networks: in one approach (subgraphs GNN), the neural network is based on subgraphs, in the other (RWNN), it is based on random walks, and in the third (RSNN), it is based on random depth-first trees. The paper presents several theorems, as well as empirical results, suggesting that RSNN is superior to RWNN, which in turn is superior to subgraphs GNN. I suggest accepting this paper.

STRENGTHS AND WEAKNESSES:
The results are strong, but it is very hard for the reader to understand what they really are, because too many notions are not well-defined. For example, the notion of expressivity is presented many times, but never rigorously defined. In line 146, there is a list of papers evaluating expressivity, but no hint as to what this expressivity is or how it is evaluated. In Theorem 3.1, it is unclear what "equipped with maximally expressive components" means, and so on. It would help a lot to give a definition, or at least a heuristic explanation. 

From Proposition 2.1, I guess expressivity has to do with distinguishing between nonisomorphic graphs, and that "appropriate graph metric" means a metric that somehow measures how graphs are close to being isomorphic. But I would be very happy with a few words affirming or refuting my guess.

QUESTIONS:
.

LIMITATIONS:
.
TITLE: Official Review
CLARITY: 3
IMPACT: 4
ORIGINALITY: 4
DATE: 2025-07-22