SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Proof. This is also straightforward to verify, since the basis in Theorem C.9 still holds for shared edges among cycles. However, the two edge flows in the cycle definitely have opposite directions in the same edge, as we assign anticlockwise as the positive direction for the flows. Consequently, the flow in  $e(j)$  is the difference of two edge flows. Moreover, for the projection without operation of taking element-wise absolute value in the projection matrix, the projection of two cycles in the  $j$ -th element of  $\mathbf{e}$  will cancel out.
There are also some additional interesting conclusions, for example, if  $e(j)$  is in cycle  $c$  and in a 2-simplex, then the edge flow in  $e(j)$  will be  $\frac{2}{3}\gamma$ , where  $\gamma$  is the original edge flow in the cycle, and the edge flow in the other two edges of the 2-simplicial will be  $\frac{1}{3}\gamma$ . Moreover, the eigenvalue 1 of  $\mathbf{L}_1$  will mark out the following substructure: node  $a, c$  are both only connected to  $b$ , while  $b$  is also connected to a fourth node  $d$ . All these interesting results can be verified through simple algebra, and we will not list all of them due to the limited space.
Finally, despite the satisfying theoretical properties of Hodge1Lap if we use the projection method (Hodge1Lapproj in the main text), we experimentally find that a naive summation over the absolute values of eigenvectors (referred as Hodge1Lapabs) works fine for a number of real-world tasks, although this method is sign-invariant but not basis-invariant.
We also provide more discussion on the cycle detecting ability of Hodge1Lap. As explained above, in Hodge1Lapproj, projecting the constant vector onto the kernel space of  $\mathbf{L}_1$  will result in the element  $j$  equal to 1 if the edge  $j$  is in a cycle, and 0 otherwise. However, it is unable to distinguish different cycles or to indicate which cycle an edge belongs to. In comparison, Hodge1Lapabs can distinguish and detect different cycles, which we explain as follows. In the main text, this implementation is described as  $\sum_{i}\phi (|\mathbf{v}_{i}|)$ , where  $||$  indicates taking element-wise absolute values, and  $\mathbf{v}_i$  refers to the interested eigenvectors (e.g. orthogonal basis of kernel space of  $\mathbf{L}_1$ , in case that we are discussing detecting cycles). Note that here  $\mathbf{v}_i$  is different from  $\mathbf{u}_i$  described in Theorem C.9; instead, they are arbitrary random eigenvectors, and we aim to use randomness to distinguish different cycles. For simplicity, we first do not consider overlapping or 2-simplexes; then every eigenvector  $\mathbf{v}_i$  corresponding to zero eigenvalues has the following properties: (i) the elements corresponding to edges that are not in any cycles are zero; (ii) the elements corresponding to edges from the same cycles have identical absolute values, while those of edges from different cycles (almost surely) have
different absolute values. These properties can be easily verified, since any  $\mathbf{v}_i$  can be represented as a linear combination of  $\mathbf{u}_i$  described in Theorem C.9. Therefore,  $\sum_{i} MLP(|\mathbf{v}_i|)$  obtains different values for edges from different cycles, which is why we distinguish and detect them. In practical implementations, we further apply a random projection technique to make Hodge1Lap more robust to the choice of basis  $\mathbf{v}_i$ :  $\sum_{j} MLP(\sum_{i} \alpha_{ij} | \mathbf{v}_i|)$ , where  $\alpha_{ij} \sim \mathcal{N}(0, 1)$  are random variables, and  $j = 1, \ldots, J$  where  $J$  is the maximum number of projections. Through this design, the model can explicitly learn different cycles and implicitly maintain the basis invariance.
C.3 Random walk on higher order and inter-order simplices
C.3.1 Properties of random walk on higher order and inter-order simplices
Analogously to 0-simplices and 1-simplices, we can use random walk on any  $k$ -simplices to build expressive  $k$ -RWSE in order to facilitate graph and simplicial learning. Meanwhile, Hodge  $k$ -Laplacians are closely connected with random walk of  $k$  order, whose spectra, including their eigenvalues and eigenvectors, are applicable in methods such as PE or spectral convolution learning.
Here we prove some crucial properties of Hodge Laplacians  $\mathbf{L}_k$ .
Lemma C.12.
$$
\mathbf {L} _ {k} ^ {r} = \left(\mathbf {L} _ {k, u p} + \mathbf {L} _ {k, d o w n}\right) ^ {r} = \mathbf {L} _ {k, u p} ^ {r} + \mathbf {L} _ {k, d o w n} ^ {r} \tag {60}
$$
Proof. We prove this by induction. For  $r = 1$ , the equation obviously holds.
Suppose  $\mathbf{L}_k^r = \mathbf{L}_{k,up}^r +\mathbf{L}_{k,down}^r$  , then for its  $r + 1$  -th power, we have
$$
\begin{array}{l} \mathbf {L} _ {k} ^ {r + 1} = \left(\mathbf {L} _ {k, u p} ^ {r} + \mathbf {L} _ {k, d o w n} ^ {r}\right) \left(\mathbf {L} _ {k, u p} + \mathbf {L} _ {k, d o w n}\right) (61) \\ = \mathbf {L} _ {k, u p} ^ {r + 1} + \mathbf {L} _ {k, d o w n} ^ {r + 1} + \mathbf {L} _ {k, u p} ^ {r} \cdot \mathbf {L} _ {k, d o w n} + \mathbf {L} _ {k, d o w n} ^ {r} \cdot \mathbf {L} _ {k, u p} (62) \\ = \mathbf {L} _ {k, u p} ^ {r + 1} + \mathbf {L} _ {k, d o w n} ^ {r + 1} + \left(\delta_ {k} ^ {*} \delta_ {k}\right) ^ {r} \delta_ {k - 1} \delta_ {k - 1} ^ {*} + \left(\delta_ {k - 1} \delta_ {k - 1} ^ {*}\right) ^ {r} \delta_ {k} ^ {*} \delta_ {k} (63) \\ = \mathbf {L} _ {k, u p} ^ {r + 1} + \mathbf {L} _ {k, d o w n} ^ {r + 1} (64) \\ \end{array}
$$
The last equation holds because  $\delta_k\delta_{k - 1} = 0$  and  $\delta_{k - 1}^{*}\delta_{k}^{*} = 0$  always hold for all  $k\geq 1$
Using the above property, we can prove that the power of the inter-order adjacent matrix (defined in Section 6.2 in the main text) satisfies the following property.
Theorem C.13. (Equation 7 in main text).
$$
\mathcal {A} _ {K} ^ {r} = \left[ \begin{array}{c c c c c} p _ {r} (\mathbf {L} _ {0}) & q _ {r - 1} (\mathbf {L} _ {0, u p}) \delta_ {0} ^ {*} & & & \\ q _ {r - 1} (\mathbf {L} _ {1, d o w n}) \delta_ {0} & p _ {r} (\mathbf {L} _ {1}) & q _ {r - 1} (\mathbf {L} _ {1, u p}) \delta_ {1} ^ {*} & & \\ & \dots & \dots & \dots & \\ & & \dots & \dots & \dots \\ & & & q _ {r - 1} (\mathbf {L} _ {K, d o w n}) \delta_ {K - 1} & p _ {r} ^ {\prime} (\mathbf {L} _ {K}, \mathbf {L} _ {K, d o w n}) \end{array} \right] \tag {65}
$$
where  $p_r(\cdot)$  and  $q_r(\cdot)$  are polynomials with maximum order  $r$ , except the last row is incomplete without  $\mathbf{L}_{K,up}$ . Note that we replace  $\mathbf{B}_{k+1}^*$  in main text with coboundary operators  $\delta_k$  for universality.
Proof. We still prove by induction. For  $r = 1$ , the equation obviously holds with  $p_1(x) = x, q_0 = 1$ . Suppose the equation holds for  $r$ , then the  $r + 1$  power satisfies the following.
(1) The block at first row and first column concerning  $\mathbf{L}_0$ , note that  $\mathbf{L}_0 = \mathbf{L}_{0,up} = \delta_0^*\delta_0$ , we have
$$
\mathcal {A} _ {K} ^ {r + 1} [ 1, 1 ] = p _ {r} (\mathbf {L} _ {0}) \mathbf {L} _ {0} + q _ {r - 1} (\mathbf {L} _ {0, u p}) \delta_ {0} ^ {*} \delta_ {0} = \left(p _ {r} (\mathbf {L} _ {0}) + q _ {r - 1} (\mathbf {L} _ {0})\right) \mathbf {L} _ {0} = p _ {r + 1} (\mathbf {L} _ {0}) \tag {66}
$$
(2) The  $k$ th diagonal block  $(1 < k < K)$ :
$$
\begin{array}{l} \mathcal {A} _ {K} ^ {r + 1} [ k, k ] = q _ {r - 1} \left(\mathbf {L} _ {k, d o w n}\right) \delta_ {k - 1} \delta_ {k - 1} ^ {*} + p _ {r} \left(\mathbf {L} _ {k}\right) \mathbf {L} _ {k} + q _ {r - 1} \left(\mathbf {L} _ {k, u p}\right) \delta_ {k} ^ {*} \delta_ {k} (67) \\ = \left(p _ {r} \left(\mathbf {L} _ {k}\right) + q _ {r - 1} \left(\mathbf {L} _ {k}\right)\right) \mathbf {L} _ {k} (68) \\ = p _ {r + 1} \left(\mathbf {L} _ {k}\right) (69) \\ \end{array}
$$
where we use  $q_{r-1}(\mathbf{L}_{k,up}) + q_{r-1}(\mathbf{L}_{k,down}) = q_{r-1}(\mathbf{L}_k)$  according to Lemma C.12. Further, we have  $p_{r+1}(x) = (p_r(x) + q_{r-1}(x)) \cdot x$ .
(3) The block at  $k$ -th row and  $k + 1$ -th column:
$$
\begin{array}{l} \mathcal {A} _ {K} ^ {r + 1} [ k, k + 1 ] = 0 + p _ {r} (\mathbf {L} _ {k}) \delta_ {k} ^ {*} + q _ {r - 1} (\mathbf {L} _ {k, u p}) \delta_ {k} ^ {*} (\delta_ {k} \delta_ {k} ^ {*} + \delta_ {k + 1} ^ {*} \delta_ {k + 1}) + 0 (70) \\ = p _ {r} \left(\delta_ {k - 1} \delta_ {k - 1} ^ {*}\right) \delta_ {k} ^ {*} + p _ {r} \left(\delta_ {k} ^ {*} \delta_ {k}\right) \delta_ {k} ^ {*} + q _ {r - 1} \left(\mathbf {L} _ {k, u p}\right) \mathbf {L} _ {k, u p} (71) \\ = \left(p _ {r} \left(\mathbf {L} _ {k, u p}\right) + q _ {r - 1} \left(\mathbf {L} _ {k, u p}\right) \mathbf {L} _ {k, u p}\right) \delta_ {k} ^ {*} (72) \\ = q _ {r} \left(\mathbf {L} _ {k, u p}\right) \delta_ {k} ^ {*} (73) \\ \end{array}
$$
Hence we have  $q_{r}(x) = p_{r}(x) + q_{r - 1}(x)\cdot x$
Further, the block at  $k + 1$ -th and  $k$ -th column is the adjoint of that:
$$
\mathcal {A} _ {K} ^ {r + 1} [ k + 1, k ] = \left(q _ {r} \left(\mathbf {L} _ {k, u p}\right) \delta_ {k} ^ {*}\right) ^ {*} = \delta_ {k} \cdot q _ {r} \left(\delta_ {k} ^ {*} \delta_ {k}\right) = q _ {r} \left(\mathbf {L} _ {k + 1, d o w n}\right) \delta_ {k} \tag {74}
$$
(4) The block at  $k$ -th row and  $k + 2$ -th column:
$$
\mathcal {A} _ {K} ^ {r + 1} [ k, k + 1 ] = 0 + 0 + q _ {r - 1} \left(\mathbf {L} _ {k, u p}\right) \delta_ {k} ^ {*} \delta_ {k + 1} ^ {*} = 0 \tag {75}
$$
Therefore, the block in  $k + 2$ -th row and  $k$ -th column is also 0.
(5) The other blocks are obviously zero.
Combining all these pieces together, we prove the theorem.
The above equation states that simplices with difference of order larger than one cannot directly exchange information even after infinite rounds, but they can affect each other through the coefficients in  $p_r$  and  $q_{r-1}$  in the blocks on the offset ±1-diagonal blocks.
It's noticeable that a number of previous works such as [8] can be reformatted and unified by  $\mathcal{A}_K$ . Additionally, we can make use of  $\mathcal{A}_K^r$  to build random walk based positional encoding for all simplices in  $K$ -dimensional simplicial complex that contains more information than random walks within the same order simplices.
In addition, analogously to [7], we can introduce any form of discrete topological structures, e.g., cellular complex, as expanded complex cells. Our methods can also be naturally extended to discrete domains without orientations, e.g. hypergraphs. In particular, we can define random walk on these discrete topological structures, which can greatly facilitate graph learning by incorporating structures of higher order other than simplicial complexes. Our methods can be easily generalized to these structures, for example, we provide the results of random walk on cellular complexes in Table 1, which treats cycles as 2-cellular complexes and greatly improve the performance of the base model. More implementation details can be found in Appendix E. However, the theoretical analysis including expressive power in distinguishing non-isomorphic graphs of these new methods are nontrivial due to the flexibility in definition of complex cells, which is a future direction worth exploring.
C.3.2 Hodge Laplacians spectra and graph isomorphism
In the spectral domain, a necessary but insufficient condition for two graphs to be isomorphic is that their Hodge  $k$ -Laplacians are isospectral (having the same eigenvalues) for all  $k \geq 0$ . As we have discussed before, the Hodge 0-isospectra is incomparable to 1-FWL and is not more powerful than 2-FWL. Hodge 1-isospectra is incomparable with 1-FWL and 2-FWL. Note that Hodge  $k + 1$ -isospectra is not necessarily more powerful than Hodge  $k$ -isospectra, except that for  $k = 0$  the conclusion holds since  $\mathbf{L}_{0,\text{down}} = 0$ . However, we can always get a more powerful algorithm by increasing the highest order of Hodge Laplacians while maintaining all the Hodge Laplacians of lower orders.
In addition to making use of eigenvalues, we can build more powerful GNNs based on the spectra of Hodge  $k$ -Laplacians. We address that we can universally approximate permutation equivariant and basis invariant functions (moreover,  $k$ -form) defined on  $k$ -simplicial complexes if we use Expressive-BasisNets [32]. Universality is guaranteed by the following decomposition theorem.
Theorem C.14. (Theorem 4 in [32].) Theorem 4 (Decomposition Theorem). Let  $\mathcal{X}_1, \ldots, \mathcal{X}_k$  be topological spaces and let  $G_i$  be a topological group that continuously acts on  $\mathcal{X}_i$  for each  $i$ . We assume that the mild topological conditions on  $\mathcal{X}_i$  and  $G_i$  hold. Assume that there is a topological embedding  $\psi_i: \mathcal{X}_i / G_i \to \mathbb{R}^{a_i}$  of each quotient space into a Euclidean space  $\mathbb{R}^{a_i}$  for some dimension  $a_i$ . Then, for any continuous function  $f: \mathcal{X} = \mathcal{X}_1 \times \dots \times \mathcal{X}_k \to \mathbb{R}^{d_{out}}$  that is invariant to the action of  $G = G_1 \times \dots \times G_k$ , there exist continuous functions  $\phi_i: \mathcal{X}_i \to \mathbb{R}^{a_i}$  and a continuous function  $\rho: \mathcal{Z} \subseteq \mathbb{R}^a \to \mathbb{R}^{d_{out}}$ , where  $a = \sum_i a_i$  such that  $f(v_1, \ldots, v_k) = \rho(\phi_1(v_1), \ldots, \phi_k(v_k))$ .
The proof of Theorem C.14 is completed in [32]. For mild topological conditions, we only need  $G_{i}$  to be a topological group that continuously acts on  $\mathcal{X}_i$  for each  $i$ , and to know that there exists a topological embedding of each quotient space into some Euclidean space. As a matter of fact, the continuous group action is a very mild assumption which holds for any finite or compact matrix group. For a finite simplicial complex, the eigenvalues and eigenvectors of Hodge-Laplacians are finite; thus, the compactness naturally holds. Furthermore, using Hodge-Laplacians normalization in [24], the maximum eigenvalue of  $\mathbf{L}_k$  satisfies  $\lambda_{max} \leq k + 2$ . Finally, letting  $\mathcal{X}_i$  be the cochain  $\mathcal{C}^i$  in the above theorem, we see that the permutation equivariant and basis-invariant function as well as the  $k$ -form defined on the simplicial complex can be universally approximated, for instance, by Expressive-BasisNet.
Although we can recover the spectra of  $\mathbf{L}_k, k = 0,\dots ,K$  (or spectra convolution on simplicial complexes) via Expressive-BasisNet, it is actually impractically expensive. On the other hand, we can use spectral information, such as the eigenvalues and eigenvectors of  $\mathbf{L}_k$  to facilitate learning via simple networks.
D Random walk message passing
In our main text, we mainly discuss how to facilitate graph and simplicial learning through designing PE and SE based on random walk on simplicial complexes. However, it is remarkable that random walk on simplicial complexes can give different insights and inspirations in designing powerful GNNs and simplicial networks in addition to PE and SE. In this section, we propose a novel random walk message passing (RWMP) mechanism, which introduces node distance metrics and simulates a weighted random walk at node level by dropping edges according to distances between neighboring nodes. As revealed in the experiments shown in the main text, RWMP is able to improve the performance of the base models and achieves SOTA performance in the Zinc dataset [17].
D.1 Connections between random walk, subgraph sampling and edge dropping
There have been a great number of relevant works on improving GNNs with subgraph sampling and edge dropping. Subgraph GNNs are a huge family of GNN variations that encode a set of subgraphs instead of the original graph. Some subgraph GNNs are more powerful than MPNN or 1-WL such as [5], while Frasca et al. [22] upper bound node-based subgraph GNNs by 3-WL. Despite the improvement in expressivity and performances, a number of subgraph GNNs suffer from extensive computation complexity due to the exponential growth of the subgraph amount. Therefore, various subgraph sampling methods have been proposed to improve the scalability of subgraph GNNs. For example,  $K$ -hop GNN [21] extracts a  $K$ -hop subgraph for each root node, and Zeng et al. [49], Toenshoff et al. [43] propose to sample subgraphs via a random walk started from a root node. Due to its internal association with the diffusion process, random walk is a powerful tool to design subgraph sampling methods. Meanwhile, these subgraph GNNs based on sampling in a random walk fashion can be interpreted as implicitly encoding structure information obtained by random walk via the following procedure: sampling subgraphs (sampling according to probability transition matrix), running GNN on the subgraphs (encoding structure captured by a random walk), and aggregating information among subgraphs (taking expectation).
Dropping edge is another family of methods that introduce randomness into graph learning. DropEdge [40] is a widely adopted technique that randomly drops edges during training to improve the generalizability of GNNs. Unlike DropEdge, DropGNN [37] randomly drops edges in both the training and inference stages. DropGNN is able to partly improve expressive power, but it has to gather several rounds of inference results to obtain the final representation. In the context of representation learning for linear features, the representations obtained by randomly dropping edges layer-wise are unbiased estimators of full message passing if we consider normalization. In comparison, the
representations obtained by subgraph sampling are biased estimators of full message passing. The layer-wise drop edge can be viewed as an intermediate product between full message passing and subgraph sampling GNNs, which (1) obtains a representation in every forward pass like original message passing and does not have to run several forward passes on subgraphs in parallel; (2) has randomness like subgraph sampling methods and has the probability to encode subgraph structures. We leave further discussion and more theoretical analysis for future work.
D.2 Random walk message passing: weighted random walk based on node-level distance metric
As we discussed above, both the subgraph sampling and edge-dropping methods can be associated with the simple node-level random walk on unweighted graphs. Through introducing randomness, they can better encode structure information and improve expressive power of GNNs. However, these methods use uniform sampling and uniform dropping strategies associated with unweighted random walk, and hence they are only able to learn pure structure information but no feature information.
To address this limitation, we propose the Random Walk Message Passing mechanism (RWMP), which can jointly learn feature and structure information by introducing distance metrics of nodes (or node features). Briefly speaking, RWMP drops edges in a layerwise fashion, and the probability of dropping edge  $e = (u,v)$  is a function of the distance between nodes  $u$  and  $v$ :  $P_{\mathrm{drop}}(e) = f(d(u,v))$ , where  $f$  is the designed function, and  $d(u,v)$  is the (feature) distance between  $u$  and  $v$  in the defined metric. For example, if we use cosine similarity between node features as the distance metric, a possible drop probability is:
$$
P _ {\text {d r o p}} (e) = \frac {1}{2} \left(1 - \frac {| | \mathbf {u} \cdot \mathbf {v} | |}{| | \mathbf {u} | | \cdot | | \mathbf {v} | |}\right) \tag {76}
$$
where  $\mathbf{u}$  and  $\mathbf{v}$  denote the node feature vector of  $u$  and  $v$ . The above metric states that the probability of message passing between two nodes is linear with their cosine similarity in node features. The more they are different, the less likely they are to exchange information with each other. Intuitively, this can partly address the problem of oversmoothing, since it can reserve heterogeneity by reducing message passing times within dissimilar nodes. RWMP also encourages message passing within similar nodes, which is beneficial for discovering local clusters (it is equivalent to sample the subgraphs containing local structures of similar nodes with larger probabilities).
RWMP is a novel framework that can jointly learn structure and feature information by simulating a weighted random walk based on a distance metric. Appropriate designs of distance metrics are able to greatly improve the performance of RWMP, which is applicable to any message-passing based GNN variations. RWMP can be regarded as an intermediate scheme between deterministic message passing variations like GAT [45] and subgraph sampling-based methods. However, on the one hand, RWMP is different from GAT in the way that RWMP explicitly introduces randomness via dropping edges instead of taking expectation through the attention weights, and the stochastic process forces RWMP to explore more about local structures while relieving oversmoothing. On the other hand, RWMP greatly reduces the computation costs on subgraph GNNs - the latter often fail to run on large graph benchmarks, while preserving the ability to explore more on certain structures.
Combining RWMP with edge-level PE/SE, a GINE [25] model is able to achieve highly competitive performance in the Zinc dataset. More experimental details are listed in Appendix E.
E.1 Datasets description
In Section 7 in the main text and Appendix E.2, we conduct extensive experiments on various datasets, confirming the effectiveness of our methods. Among these datasets, Zinc, MNIST, and CIFAR10 are from Benchmarking GNN [17], ogbg-molhiv and ogbg-molpcba are from Open Graph Benchmark [26], while PCQM-Contact, Peptides-function and Peptides-struct are from Long-range Graph Benchmark [19].
Zinc. Zinc-12k is a subset of Zinc-250k, which consists of 12000 molecular graphs from the ZINC database of commercially available chemical compounds. The task is to perform graph-level molecular property regression (on constrained solubility logP). These molecular graphs are between 9 and 37 nodes large. We follow the common predefined 10K/1K/1K train/validation/test split.
MNIST and CIFAR10. These two datasets both contain directed graphs, and are derived from like-named image classification datasets. Both of them are 10-class classification tasks, and we follow the standard dataset splits as the original image classification datasets, i.e.,  $55\mathrm{K} / 5\mathrm{K} / 10\mathrm{K}$  for MNIST and  $45\mathrm{K} / 5\mathrm{K} / 10\mathrm{K}$  for CIFAR10 of train/validation/test graphs, respectively.
ogbg-molhiv and ogbg-molpcba. They are both molecular property prediction datasets, using a common node (atom) and edge (bond) featurization to represent chemophysical properties. In detail, the prediction task of ogbg-molhiv is a binary classification of the fitness of a molecule to inhibit HIV replication, which is measured by AUROC. The task of ogbg-molpcba is a 128-task binary classification, evaluated by average precision.
PCQM-Contact. It is a dataset derived from PCQM4Mv2 and the corresponding 3D molecular structures. The task is a binary link prediction which is evaluated by the Mean Reciprocal Rank (MRR).
Peptides-func and Peptides-struct. These two datasets both consist of atomic graphs of peptides. The task for Peptides-func is a multi-label graph classification into 10 nonexclusive peptide functional classes measured by average precision. The task for Peptides-struct is graph regression of 11 3D-structural properties of the peptides measured by MAE.
E.2 Full experiment results
While SOTA and highly competitive results on Zinc, ogbg-molhiv and ogbg-molpcba are shown in the main text, we further discuss the performance of our methods on other datasets here, including synthetic datasets and real-world datasets. For real-world datasets including BenchmarkingGNN [17] and Long Range Graph Benchmark (LRGB) [19], we follow the experimental settings of [38] and
do not perform a hyperparameter search, since our main goal is to verify that our Hodge1Lap and EdgeRWSE can benefit the arbitrary base model.