SlowGuess/NeurIPS_2023_OCR huggingface dataset:
How can this sampling procedure capture higher-order patterns? As discussed in Appendix C, SETWALKs on  $\mathcal{G}$  can be interpreted as a random walk on the dual hypergraph of  $\mathcal{G}$ ,  $\tilde{\mathcal{G}}$ . However, a simple (or hyperedge-independent) random walk on the dual hypergraph is equivalent to the walk on the CE of the dual hypergraph [40, 41], missing the higher-order dependencies of hyperedges. Inspired by Chitra and Raphael [40], we use hyperedge-dependent weights  $\Gamma: \mathcal{V} \times \mathcal{E} \to \mathbb{R}^{\geq 0}$  and sample hyperedges with a probability proportional to  $\exp\left(\sum_{u \in e \cap e_p} \Gamma(u, e) \Gamma(u, e')\right)$ , where  $e_p$  is the previously sampled hyperedge. In the dual hypergraph  $\tilde{\mathcal{G}} = (\mathcal{E}, \mathcal{V})$ , we assign a score  $\tilde{\Gamma}: \mathcal{E} \times \mathcal{V} \to \mathbb{R}^{\geq 0}$  to each pair of  $(e, u)$  as  $\tilde{\Gamma}(e, u) = \Gamma(u, e)$ . Now, a SETWALK with this sampling procedure is equivalent to the edge-dependent hypergraph walk on the dual hypergraph of  $\mathcal{G}$  with edge-dependent weight  $\tilde{\Gamma}(.)$ . Chitra and Raphael [40] show that an edge-dependent hypergraph random walk can capture some information about higher-order interactions and is not equivalent to a simple walk on the weighted CE of the hypergraph. Accordingly, even on the dual hypergraph, SETWALK with this sampling procedure can capture higher-order dependencies of hyperedges and is not equivalent to a simple walk on the CE of the dual hypergraph  $\tilde{\mathcal{G}}$ . We conclude that, unlike existing random walks on hypergraphs [37, 38, 41, 79], SETWALK can capture both higher-order interactions of nodes, and, based on its sampling procedure, higher-order dependencies of hyperedges.
E Theoretical Results
Proof. In this proof, we focus on the hypergraph random walk and simple random walk on the CE. The proof for the SE graph is the same and also it has been proven that the SE graph and the CE of a hypergraph have close (or equal in uniform hypergraphs) Laplacian and have the same expressiveness power in the representation of hypergraphs [121-123].
First, note that each SETWALK can be approximately decomposed to a set of either hypergraph walks, simple random walks, or walk on the SE. Moreover, each of these walks can be mapped to
a corresponding SETWALK (but not a bijective mapping), by sampling hyperpedges corresponding to each consecutive pair of nodes in these walks. Accordingly, SETWALKs includes the information provided by these walks and so its expressiveness is not less than these methods. To this end, next, we discuss two examples in two different tasks for which SETWALKs are successful while other walks fail.
① In the first task, we want to see if there exists a pair of hypergraphs with different semantics that SETWALKs can distinguish, but other walks cannot. We construct such hypergraphs. Let  $\mathcal{G} = (\mathcal{V},\mathcal{E})$  be a hypergraph with  $\mathcal{V} = \{u_1,u_2,\ldots ,u_N\}$  and  $\mathcal{E} = \{(e,t_i)\}_{i = 1}^T$ , where  $e = \{u_{1},u_{2},\dots ,u_{N}\}$  and  $t_1 < t_2 < \dots < t_T$ . Also, let  $\mathcal{A}$  be an edge-independent hypergraph random walk (or random walk on the CE) sampling algorithm. Chitra and Raphael [40] show that each of these walks is equivalent to a random walk on the weighted CE. Assume that  $\xi (.)$  is a function that assigns weights to edges in  $\mathcal{G}^{*} = (\mathcal{V},\mathcal{E}^{*})$ , the weighted CE of the  $\mathcal{G}$ , such that a hypergraph random walk on  $\mathcal{G}$  is equivalent to a walk on this weighted CE graph. Next, we construct a weighted hypergraph  $\mathcal{G}' = (\mathcal{V},\mathcal{E}')$  with the same set of vertices but with  $\mathcal{E}' = \bigcup_{k = 1}^{T}\{((u_i,u_j),t_k)\}_{u_i,u_j\in \mathcal{V}}$ , such that each edge  $e_{i,j} = (u_i,u_j)$  is associated with a weight  $\xi (e_{i,j})$ . Clearly, sampling procedure  $\mathcal{A}$  on  $\mathcal{G}$  and  $\mathcal{G}'$  are the same, while they have different semantics. For example, assume that both are collaboration networks. In  $\mathcal{G}$ , all vertices have published a single paper together, while in  $\mathcal{G}'$ , each pair of vertices have published a separate paper together. The proof for the hypergraph random walk with hyperedge-dependent weights is the same, while we construct weights of the hypergraph  $\mathcal{G}'$  based on the sampling probability of hyperedges in the hypergraph random walk procedure.
② Next, in the second task, we investigate the expressiveness of these walks for reconstructing hyperedges. That is, we want to see that given a perfect classifier, can these walks provide enough information to detect higher-order patterns in the network. To this end, we show that for a finite number of samples of each walk,  $\mathsf{SETWALK}$  is more expressive than all of these walks in detecting higher-order patterns. Let  $M$  be the maximum number of samples and  $L$  be the maximum length of walks, we show that for any  $M \geq 2$  and  $L \geq 2$  there exists a pair of hypergraphs  $\mathcal{G}$ , with higher-order interactions, and  $\mathcal{G}'$ , with pairwise interactions, such that  $\mathsf{SETWALK}$  can distinguish them, while they are indistinguishable by any of these walks. We construct a temporal hypergraph  $\mathcal{G} = (\mathcal{V}, \mathcal{E})$  as a hypergraph with  $\mathcal{V} = \{u_1, u_2, \ldots, u_{M(L+1)+1}\}$  and  $\mathcal{E} = \{(e, t_i)\}_{i=1}^L$ , where  $e = \{u_1, u_2, \ldots, u_{M(L+1)+1}\}$  and  $t_1 < t_2 < \dots < t_L$ . We further construct  $\mathcal{G}' = (\mathcal{V}, \mathcal{E}')$  with the same set of vertices but with  $\mathcal{E}' = \bigcup_{k=1}^L \{((u_i, u_j), t_k)\}_{u_i, u_j \in \mathcal{V}}$ . Figure 6 illustrates  $\mathcal{G}$  and its projected graphs at a given timestamp  $t \in \{t_1, t_2, \ldots, t_L\}$ .
$\mathsf{SETWALK}$  with only one sample, Sw, can distinguish interactions in these two hypergraphs. That is, let  $\mathbf{Sw}:(e,t_L)\to (e,t_{L - 1})\to \dots \to (e,t_1)$  be the sample SETWALK from  $\mathcal{G}$  (note that masking the time, this is the only SETWALK on  $\mathcal{G}$ , so in any case the sampled SETWALK is Sw). Since all interactions in  $\mathcal{G}'$  are pairwise, any sampled SETWALK on  $\mathcal{G}'$ , Sw', includes only pairwise interactions, so Sw  $\neq$  Sw', in any case. Accordingly, SETWALK can distinguish interactions in these two hypergraphs.
Since the output of hypergraph random walks, simple walks on the CE, and walks on the SE include only pairwise interactions, it seems that they are unable to detect higher-order patterns, so are unable to distinguish these two hypergraphs. However, one might argue that by having a large number of sampled walks and using a perfect classifier, which learned the distribution of sampled random walks and can detect whether a set of sampled walks is from a higher-order interaction, we might be able to detect higher-order interactions. To this end, we next assume that we have a perfect classifier  $C(.)$  that can detect whether a set of sampled hypergraph walks, simple walks on the CE, or walks on the SE are sampled from a higher-order structure or pair-wise patterns. Next, we show that hypergraph random walks cannot provide enough information about every vertex for  $C(.)$  to detect whether all vertices in  $\mathcal{V}$  shape a hyperedge. To this end, assume that we sample  $S = \{W_1,W_2,\dots ,W_M\}$  walks from hypergraph  $G$  and  $S^{\prime} = \{W_{1}^{\prime},W_{2}^{\prime},\ldots ,W_{M}^{\prime}\}$  walks from hypergraph  $G^{\prime}$ . In the best case scenario, since  $C(.)$  is a perfect classifier, it can detect that  $G^{\prime}$  includes only pair-wise interactions based on sampled walk  $S^{\prime}$ . To distinguish these two hypergraphs, we need  $C(.)$  to detect sampled walks from  $G$  (i.e.,  $S$ ) that come from a higher-order pattern. For any  $M$  sampled walks with length  $L$  from  $G$ , we observe at most  $M\times (L + 1)$  vertices, so we have information about at most  $M\times (L + 1)$  vertices, unable to capture any information about the neighborhood of at least one vertex. Due to the symmetry of vertices, without loss of generality, we can assume that this vertex is  $u_{1}$ . This means that with these  $M$  sampled hypergraph random walks with length  $L$ , we are not able to provide any information about node  $u_{1}$  at any timestamp for  $C(.)$ . Therefore, even a perfect classifier  $C(.)$  cannot verify whether  $u_{1}$
is a part of higher-order interaction or pair-wise interaction, which completes the proof. Note that the proof for the simple random walk is completely the same.
Remark 1. Note that while the first task investigates the expressiveness of these methods with respect to their sampling procedure, the second tasks discuss the limitation and difference in their outputs.
Remark 2. Note that in reality, we can have neither an unlimited number of samples nor an unlimited walk length. Also, the upper bound for the number of samples or walk length depends on the RAM of the machine on which the model is being trained. In our experiments, we observe that usually, we cannot sample more than 125 walks with a batch size of 32.
Before discussing the proof of Theorem 2 we first formally define what missing information means in this context.
Definition 7 (Missing Information). We say a pooling strategy like  $\Psi(.)$  misses information if there is a model  $\mathcal{M}$  such that using  $\Psi(.)$  on top of the  $\mathcal{M}$  (call it  $\tilde{\mathcal{M}}$ ) decreases the expressive power of  $\mathcal{M}$ . That is,  $\tilde{\mathcal{M}}$  has less expressive power than  $\mathcal{M}$ .
Theorem 2. Given an arbitrary positive integer  $k \in \mathbb{Z}^+$ , let  $\Psi(.)$  be a pooling function such that for any set  $S = \{w_1, \ldots, w_d\}$ :
$$
\Psi (S) = \sum_ { \begin{array}{c} S ^ {\prime} \subseteq S \\ | S ^ {\prime} | = k \end{array} } f \left(S ^ {\prime}\right), \tag {13}
$$
where  $f$  is some function. Then the pooling function can cause missing information, limiting the expressiveness of the method to applying to the projected (hyper)graph of the hypergraph.
Proof. The main intuition of this theorem is that a pooling function needs to capture higher-order dependencies of its input's elements and if it can be decomposed to a summation of functions that capture lower-order dependencies, it misses information. We show that, in the general case for a given  $k \in \mathbb{Z}^{+}$ , the pooling function  $\Psi(\cdot)$  when applied to a hypergraph  $\mathcal{G}$  is at most as expressive as  $\Psi(\cdot)$  when applied to the  $k$ -projected hypergraph of  $\mathcal{G}$  (Definition 6). Let  $\mathcal{G} = (\mathcal{V}, \mathcal{E})$  be a hypergraph with  $\mathcal{V} = \{u_{1}, u_{2}, \ldots, u_{k+1}\}$  and  $\mathcal{E} = \{(\mathcal{V}, t)\} = \{(\{u_{1}, u_{2}, \ldots, u_{k+1}\}, t)\}$  for a given time  $t$ , and  $\hat{\mathcal{G}} = (\mathcal{V}, \hat{\mathcal{E}})$  be its  $k$ -projected graph, i.e.,  $\hat{\mathcal{E}} = \{(e_{1}, t), \ldots, (e_{\binom{k+1}{k}}, t)\}$ , where  $e_{i} \subset \{u_{1}, u_{2}, \ldots, u_{k+1}\}$  such that  $|e_{i}| = k$ . Applying pooling function  $\Psi(\cdot)$  on the hypergraph  $\mathcal{G}$  is equivalent to applying  $\Psi(\cdot)$  to the hyperedge  $(\mathcal{V}, t) \in \mathcal{E}$ , which provides  $\Psi(\mathcal{V}) = \sum_{i=1}^{k+1} f(e_{i})$ . On the other hand, applying  $\Psi(\cdot)$  on projected graph  $\hat{\mathcal{G}}$  means applying it on each hyperedge  $e_{i} \in \hat{\mathcal{E}}$ . Accordingly, since for each hyperedge  $e_{i} \in \hat{\mathcal{E}}$  we have  $\Psi(e_{i}) = f(e_{i})$ , all captured information by pooling function  $\Psi(\cdot)$  on  $\hat{\mathcal{G}}$  is the set of  $S = \{f(e_{i})\}_{i=1}^{k+1}$ . It is clear that  $\Psi(\mathcal{V}) = \sum_{i=1}^{k+1} f(e_{i})$  is less informative than  $S = \{f(e_{i})\}_{i=1}^{k+1}$  as it is the summation of elements in  $S$  (in fact,  $\Psi(\mathcal{V})$  cannot capture the non-linear combinations of positional encodings of vertices, while  $S$  can). Accordingly, the provided information by applying  $\Psi(\cdot)$  on  $\mathcal{G}$  cannot be more informative than applying  $\Psi(\cdot)$  on the  $G$ 's  $k$ -projected hypergraph.
Remark 3. Note that the pooling function  $\Psi (.)$  is defined on a (hyper)graph and gets only (hyper)edges as input.
Remark 4. Although  $\Psi(.)=\mathrm{MEAN}(.)$  cannot be written as Equation 13, we can simply see that the above proof works for this pooling function as well.
E.3 Proof of Theorem 3
Proof. Let  $\pi(S)$  be a given permutation of set  $S$ , we aim to show that  $\Psi(S) = \Psi(\pi(S))$ . We first recall the SETMIXER and its two phases: Let  $S = \{\mathbf{v}_1, \dots, \mathbf{v}_d\}$ , where  $\mathbf{v}_i \in \mathbb{R}^{d_1}$ , be the input set and  $\mathbf{V} = [\mathbf{v}_1, \dots, \mathbf{v}_d]^T \in \mathbb{R}^{d \times d_1}$  be its matrix representation:
$$
\Psi (\mathbf {V}) = \operatorname {M E A N} \left(\mathbf {H} _ {\text {t o k e n}} + \sigma \left(\text {L a y e r N o r m} \left(\mathbf {H} _ {\text {t o k e n}}\right) \mathbf {W} _ {s} ^ {(1)}\right) \mathbf {W} _ {s} ^ {(2)}\right), \quad \left(\text {C h a n n e l M i x e r}\right)
$$
$$
\mathbf {H} _ {\text {t o k e n}} = \mathbf {V} + \sigma \left(\operatorname {S o f t m a x} \left(\operatorname {L a y e r N o r m} (\mathbf {V}) ^ {T}\right)\right) ^ {T}. \quad (T o k e n M i x e r)
$$
Let  $\pi(\mathbf{V}) = [\mathbf{v}_{\pi(1)}, \dots, \mathbf{v}_{\pi(d)}]^T$  be a permutation of the input matrix  $\mathbf{V}$ . In the token mixer phase, None of LayerNorm, Softmax, and activation function  $\sigma(.)$  can affect the order of elements (note that Softmax is applied row-wise). Accordingly, we can see the output of the token mixer is permuted by  $\pi(.)$ :
$$
\begin{array}{l} \mathbf {H} _ {\mathrm {t o k e n}} (\pi (\mathbf {V})) = \pi (\mathbf {V}) + \sigma \left(\operatorname {S o f t m a x} \left(\operatorname {L a y e r N o r m} (\pi (\mathbf {V})) ^ {T}\right)\right) ^ {T} \\ = \pi (\mathbf {V}) + \pi \left(\sigma \left(\operatorname {S o f t m a x} \left(\operatorname {L a y e r N o r m} (\mathbf {V}) ^ {T}\right)\right) ^ {T}\right) \\ = \pi \left(\mathbf {V} + \sigma \left(\operatorname {S o f t m a x} \left(\operatorname {L a y e r N o r m} (\mathbf {V}) ^ {T}\right)\right) ^ {T}\right) \\ = \pi \left(\mathbf {H} _ {\text {t o k e n}} (\mathbf {V})\right). \tag {14} \\ \end{array}
$$
Next, in the channel mixer, by using Equation 14 we have:
$$
\begin{array}{l} \Psi (\pi (\mathbf {V})) = \operatorname {M E A N} \left(\pi (\mathbf {H} _ {\text {t o k e n}}) + \sigma \left(\text {L a y e r N o r m} (\pi (\mathbf {H} _ {\text {t o k e n}})) \mathbf {W} _ {s} ^ {(1)}\right) \mathbf {W} _ {s} ^ {(2)}\right) \\ = \mathbf {M} _ {\text {E A N}} \left(\pi \left(\mathbf {H} _ {\text {t o k e n}}\right) + \pi \left(\sigma \left(\text {L a y e r N o r m} \left(\mathbf {H} _ {\text {t o k e n}}\right) \mathbf {W} _ {s} ^ {(1)}\right)\right) \mathbf {W} _ {s} ^ {(2)}\right) \\ = \operatorname {M E A N} \left(\pi \left(\mathbf {H} _ {\text {t o k e n}}\right) + \pi \left(\sigma \left(\text {L a y e r N o r m} \left(\mathbf {H} _ {\text {t o k e n}}\right) \mathbf {W} _ {s} ^ {(1)}\right) \mathbf {W} _ {s} ^ {(2)}\right)\right) \\ = \operatorname {M E A N} \left(\pi \left(\mathbf {H} _ {\text {t o k e n}} + \mathbf {W} _ {s} ^ {(2)} \sigma \left(\text {L a y e r N o r m} \left(\mathbf {H} _ {\text {t o k e n}}\right) \mathbf {W} _ {s} ^ {(1)}\right)\right)\right) \\ = \Psi (\mathbf {V}). \tag {15} \\ \end{array}
$$
In the last step, we use the fact that  $\mathrm{MEAN}(.)$  is permutation invariant. Based on Equation 15 we can see that SETMIXER is permutation invariant.
Since the token mixer is just normalization it is inevitable and cannot affect the expressive power of SETMIXER. Also, channel mixer is a 2-layer MLP, which is the universal approximator of any function. Therefore, SETMIXER is a universal approximator.
E.4 Proof of Theorem 4
Theorem 4. The set-based anonymization method is more expressive than any existing anonymization strategies on the CE of the hypergraph. More precisely, there exists a pair of hypergraphs  $\mathcal{G}_1 = (\mathcal{V}_1,\mathcal{E}_1)$  and  $\mathcal{G}_2 = (\mathcal{V}_2,\mathcal{E}_2)$  with different structures (i.e.,  $\mathcal{G}_1\neq \mathcal{G}_2$ ) that are distinguishable by our anonymization process and are not distinguishable by the CE-based methods.
Proof. To the best of our knowledge, there exist two anonymization processes for random walks by Wang et al. [33] and Micali and Zhu [45]. Both of these methods are designed for graphs and to adapt them to hypergraphs we need to apply them to the (weighted) CE. Here, we focus on the process designed by Wang et al. [33], which is more informative than the other. The proof for the Micali and Zhu [45] process is the same. Note that the goal of this theorem is to investigate whether a method can distinguish a hypergraph from its CE. Accordingly, this theorem does not provide any information about the expressivity of these methods in terms of the isomorphism test.
The proposed 2-step anonymization process can be seen as a positional encoding for both vertices and hyperedges. Accordingly, it is expected to assign different positional encodings to vertices and hyperedges of two non-isomorphism hypergraphs. To this end, we construct the same hypergraphs as in the proof of Theorem 1. Let  $M$  be the number of sampled SETWALKs with length  $L$ . We construct a temporal hypergraph  $\mathcal{G} = (\mathcal{V}, \mathcal{E})$  as a hypergraph with  $\mathcal{V} = \{u_1, u_2, \ldots, u_{M(L+1)+1}\}$  and  $\mathcal{E} = \{(e, t_i)\}_{i=1}^L$ , where  $e = \{u_1, u_2, \ldots, u_{M(L+1)+1}\}$  and  $t_1 < t_2 < \dots < t_L$ . We further construct  $\mathcal{G}' = (\mathcal{V}, \mathcal{E}')$  with the same set of vertices but with  $\mathcal{E}' = \bigcup_{k=1}^{L} \{((u_i, u_j), t_k)\}_{u_i, u_j \in \mathcal{V}}$ . As we have seen in Theorem 1, random walks on the CE of the hypergraph cannot distinguish these two hypergraphs. Since CAW [33] also uses simple random walks, it cannot distinguish these two hypergraphs. Accordingly, after its anonymization process, it again cannot distinguish these two hypergraphs.
The main part of the proof is to show that in our method, the assigned positional encodings are different in these hypergraphs. The first step is to assign each node a positional encoding. Masking
the timestamps, there is only one  $\mathsf{SETWALK}$  in the  $\mathcal{G}$ . Accordingly, the positional encodings of nodes in  $\mathcal{G}$  are the same and non-zero. Given a  $\mathsf{SETWALK}$  with length  $L$  we might see at most  $L \times (d_{\max} - 1) + 1$  nodes, where  $d_{\max}$  is the maximum size of hyperedges in the hypergraph. Accordingly, with  $M$  samples on  $\mathcal{G}'$ , which  $d_{\max} = 2$ , we can see at most  $M \times (L + 1)$  vertices. Therefore, in any case, we assign a zero vector to at least one vertex. This proves that the positional encodings by  $\mathsf{SETWALKs}$  are different in these two hypergraphs, and if the assigned hidden identities to counterpart nodes are different, clearly, feeding them to the  $\mathsf{SETMIXER}$  results in different hyperedge encodings.
Note that each SETWALK can be decomposed into a set of causal anonymous walks [33]. Accordingly, it includes the information provided by these walks, so its expressiveness is not less than the CAW method on hypergraphs, which completes the proof of the theorem.
Although the above statement completes the proof, next we discuss that even given the same positional encodings for vertices in these two hypergraphs, SETMIXER can capture higher-order interactions by capturing the size of the hyperedge. Recall token mixer phase in SETMIXER:
$$
\mathbf {H} _ {\text {t o k e n}} = \mathbf {V} + \sigma \left(\operatorname {S o f t m a x} \left(\operatorname {L a y e r N o r m} (\mathbf {V}) ^ {T}\right)\right) ^ {T},
$$
where  $\mathbf{V} = [\mathbf{v}_1, \ldots, \mathbf{v}_{M(L + 1) + 1}]^T \in \mathbb{R}^{(M(L + 1) + 1) \times d_1}$  and  $\mathbf{v}_i \neq 0_{1 \times d_1}$  represents the positional encoding of  $u_i$  in  $\mathcal{G}$ . We assumed that the positional encoding of  $u_i$  in  $\mathcal{G}'$  is the same. The input of the token mixer phase on  $\mathcal{G}$  is  $\mathcal{V}$  as all of them are connected by a hyperedge. Then we have:
$$
\left(\mathbf {H} _ {\text {t o k e n}}\right) _ {i, j} = \mathbf {v} _ {i, j} + \sigma \left(\frac {\exp \left(\mathbf {v} _ {i , j}\right)}{\sum_ {k = 1} ^ {M (L + 1) + 1} \exp \left(\mathbf {v} _ {k , j}\right)}\right). \tag {16}
$$
On the other hand, when applied to hypergraph  $\mathcal{G}'$  and  $(u_{k_1}, u_{k_2})$ . We have:
$$
\left(\mathbf {H} _ {\text {t o k e n}} ^ {\prime}\right) _ {i, j} = \mathbf {v} _ {i, j} + \sigma \left(\frac {\exp \left(\mathbf {v} _ {i , j}\right)}{\exp \left(\mathbf {v} _ {k _ {1} , j}\right) + \exp \left(\mathbf {v} _ {k _ {2} , j}\right)}\right), \quad i \in \{k _ {1}, k _ {2} \}. \tag {17}
$$
Since we use zero padding, for any  $i \geq 3$ ,  $(\mathbf{H}_{\mathrm{token}})_{i,j} \neq 0$  and  $(\mathbf{H}_{\mathrm{token}}')_{i,j} = 0$ . These zero rows, which capture the size of the hyperedge, result in different encodings for each connection.
Remark 5. To the best of our knowledge, the only anonymization process that is used on hypergraphs is by Liu et al. [78], which uses simple walks on the CE and is the same as Wang et al. [33]. Accordingly, it also suffers from the above limitation. Also, note that this theorem shows the limitation of these anonymization procedures when simply adopted to hypergraphs.
E.5 Proof of Proposition 2
Proposition 2. Edge-independent random walks on hypergraphs [37], edge-dependent random walks on hypergraphs [40], and simple random walks on the CE of hypergraphs are all special cases of  $r$ -SETWALK, when applied to the 2-projected graph, UP graph, and 2-projected graph, respectively. Furthermore, all the above methods are less expressive than  $r$ -SETWALKs.
Proof. For the first part, we discuss each walk separately:
① Simple random walks on the CE of the hypergraphs: We perform 2-SETWALKs on the (weighted) 2-projected hypergraph with  $\Gamma(.) = 1$ . Accordingly, for every two adjacent edges in the 2-Projected graph like  $e$  and  $e'$ , we have  $\varphi(e, e') = 1$ . Therefore, it is equivalent to a simple random walk on the CE (2-projected graph).
② Edge-independent random walks on hypergraphs: As is shown by Chitra and Raphael [40], each edge-independent random walk on hypergraphs is equivalent to a simple random walk on the (weighted) CE of the hypergraph. Therefore, as discussed in ①, these walks are a special case of  $r$ -SETWALKs, when  $r = 2$  and applied to (weighted) 2-Projected hypergraph.
③ Edge-dependent random walks on hypergraphs: Let  $\Gamma'(e, u)$  be an edge-dependent weight function used in the hypergraph random walk sampling. For each node  $u$  in the UP hypergraph, we store the set of  $\Gamma'(e, u)$  that  $e$  is a maximal hyperedge that  $u$  belongs to. Note that there might be several maximal hyperedges that  $u$  belongs to. Now, we perform 2-SETWALK sampling on the UP hypergraph with these weights and in each step, we sample each hyperedge with weight  $\Gamma(u, e) = \Gamma'(e, u)$ . It is
straightforward to show that given this procedure, the sampling probability of a hyperedge is the same in both cases. Therefore, edge-dependent random walks on hypergraphs are equivalent to 2-SETWALKs when applied to the UP hypergraph.
As discussed above, all these walks are special cases of  $r$ -SETWALKs and cannot be more expressive than  $r$ -SETWALKs. Also, as discussed in Theorem 1, all these walks are less expressive than SETWALKs, which are also special cases of  $r$ -SETWALKs, when  $r = \infty$ . Accordingly, all these methods are less expressive than  $r$ -SETWALKs.
F Experimental Setup Details
F.1 Datasets
We use 10 publicly available<sup>3</sup> benchmark datasets, whose descriptions are as follows: