SlowGuess/NeurIPS_2023_OCR huggingface dataset:
We first quantify what a relational neural network can compute, using the formalization developed by Luo et al. [2022]; see also Morris et al. [2019], Barceló et al. [2020]. Let depth  $D$  be the number of layers in a RelNN, and breadth  $B$  be the maximum arity of the relations (hyperedges) in the network. For example, to represent a vector embedding for each tuple of size 5,  $f(x_{1},x_{2},\dots ,x_{5})$  ,where  $x_{1},\dots ,x_{5}\in \mathcal{U}$  are entities in the planning problem, we need a relational neural network with breadth 5. We will only consider networks with a constant breadth. We denote the family of relational neural networks with depth  $D$  and breadth  $B$  as RelNN[D,B]. We will not be modeling the actual "hidden dimension" of the neural network layers (i.e., the number of neurons inside each layer), but we will assume that it is bounded (for
Figure 2: The input and output of a relational neural network (RelNN) policy.
example, as a function of  $B$  and the number of predicates in the domain). Under such assumptions (most practical graph neural networks, Transformers, and hypergraph neural networks do follow these assumptions), we have the following lemma.
Lemma 4.1 (Logical expressiveness of relational neural networks [Luo et al., 2022, Cai et al., 1992]). Let  $FOC_{B}$  denote a fragment of first-order logic with at most  $B$  variables, extended with counting quantifiers of the form  $\exists^{\geq n}\phi$ , which state that there are at least  $n$  nodes satisfying formula  $\phi$ .
- (Upper Bound) Any function  $f$  in  $FOC_{B}$  can be realized by RelNN[D, B] for some  $D$ .
- (Lower Bound) There exists a function  $f$  in  $FOC_{B}$  such that for all  $D, f$  cannot be realized by RelNN $[D, B - 1]$ .
Here, we sketch a constructive proof for using RelNNs to realize FOL formulas. The breadth  $B$  is analogous to the number of variables in FOL for encoding the value of the expression; the depth  $D$  is the number of "nested quantifiers." For example, the formula  $\exists x.\forall y.\forall z.p(x,y,z)$  needs 3 layers, one for each quantifier. Furthermore, we call a RelNN finite breadth (depth) if  $B(D)$  is independent of the number of objects. Otherwise, we call it unbounded breadth (depth).
4.2 Compilation of BWD and S-GRS
Lemma 4.1 states an "equivalence" between the expressive power of relational neural networks and first-order logic formulas. In the following, we take advantage of this equivalence to compile search algorithms into relational neural networks. First, we have the following theorem.
Theorem 4.1 (Compilation of BWD). Given a planning problem  $P$ , let  $T$  be the length of the optimal trajectory (the planning horizon),  $k_{BWD}$  be the maximum number of atoms in the goal set in BWD, and  $\beta$  be the maximum arity of atoms in the domain. The backward search algorithm BWD can be compiled into a relational neural network in RelNN[ O(T), \beta \cdot k_{BWD} ].
Proof sketch. We provide a construction in which the RelNN computes a set of subgoals (i.e., a set of sets of ground atoms)  $Goal^d$  at each layer  $d$ . Initially,  $Goal^0 = \{\{g\}\}$ . Then,  $sg \in Goal^d$  if there is a path of length  $d$  from any state  $s$  that satisfies  $sg \subseteq s$  to  $g$ . See Appendix B.1 for the full proof.
Although this construction is general and powerful, it is unrealistic for large problems because the depth can be exponential in the number of objects, and the number of atoms in the subgoal conjunctive formulas can be exponential in the depth. Therefore, in the following, we will leverage the idea of serialized goal regression to make a more efficient construction.
Theorem 4.2 (Compilation of S-GRS). Given a planning problem  $P$  of SOS width  $k$ , let  $T$  be the length of the optimal trajectory, and  $\beta$  be the maximum arity of atoms in the domain. The serialized goal regression search  $S$ -GRS can be compiled into a RelNN[O(T), (k + 1) \cdot \beta], where  $k + 1 \leq k_{BWD}$ . See Appendix B.2 for the full proof using a similar technique as in Theorem 4.1.
(b) Compilation of rule selectors into First-Order Logic formulas.
Figure 3: State-dependent regression rule selector in the Blocks World domain. For brevity, we have omitted atoms in the constraint set. All rules listed above are applicable under any constraints.
Remark. The compilation in Theorem 4.2 can generate finite-breadth, unbounded-depth RelNN circuits for more problems than the compilation in Theorem 4.1. For example, in Blocks World, the number of atoms in the goal sets in BWD is unbounded. However, since the problem is of constant SOS width, it can be compiled into a finite-breadth circuit. When the search horizon  $T$  is finite, both the SOS width and  $k_{\mathrm{BWD}}$  will be finite because there are only a constant number of actions that can be applied to update the goal set / constraint set. So a problem can be compiled into a finite-depth, finite-breadth RelNN circuit with BWD compilation if and only if it can be compiled with S-GRS compilation, although S-GRS compilation may generate smaller circuits.
4.3 Compilation of S-GRS with a Regression Rule Selector
Unfortunately, both constructions in the previous section require a depth- $O(T)$  RelNN circuit, which is impractical for most real-world problems, as  $T$  can be exponential in the number of objects in a domain. In order to compile the goal regression search algorithm into a smaller circuit, we consider scenarios where the rule used to construct the (optimal) plan can be computed without doing a full search. Here the idea is to bring in human priors or machine learning to learn a low-complexity regression rule selector that directly predicts which goal regression rule to use. There exists such a rule selector for some problems (e.g., Blocks World) but not others (e.g., Sokoban).
Formally, we define a state-dependent regression rule selector (RRS) select  $(s,g,\mathrm{cons})$ , which returns an ordered list of preconditions and a ground action  $a$ . Its function body can be written in the following form:  $g^{cons} \Leftrightarrow$  if  $\rho(s,\mathrm{cons}): p_1^{cons}, p_2^{cons \cup \{p_1\}}, \dots, p_m^{cons \cup \{p_1, \dots, p_{m-1}\}} \parallel a$ , which reads: in order to achieve  $g$  under constraint cons at state  $s$ , if  $\rho(s,\mathrm{cons})$  is true, we can apply the regression rule  $p_1, p_2, \dots, p_m \parallel a$ . Formally, select is composed of a collection of rules. Each rule is a tuple of  $\langle args,\rho,pre,a,g,\mathrm{cons}\rangle$ , where args is a set of variables,  $\rho$  is a FOL formula that can be evaluated at any given state  $s$ , pre is an ordered list of preconditions,  $a$  is a ground action,  $g$  is the goal atom, and cons is the constraint.  $\rho,pre,a,g$ , and cons may contain variables in args. Such a rule selector can be implemented by hand (as a set of rules, as illustrated in Fig. 3a for the simple BlocksWorld domain), or learned by a RelNN model.
The main computational advantage is that now, for a given tuple  $(s, g, cons)$ , there is a single ordered list of preconditions and final action. Therefore, given the function select, we can simply construct a sequence of actions that achieves the goal by recursively applying the rule selector. In this section, we will first discuss scenarios where such rule selectors can be computed with shallow RelNN circuits.
Circuit complexity of regression rule selectors. We first discuss scenarios where there exist finite-depth, finite-breadth circuits for computing the regression rule selector. Note that there are three "branching" factors in choosing the regression rule: 1) the operator to use, 2) the order of the preconditions, and 3) the binding of "free" variables that are not mentioned in the goal atom.
First, in many cases, a simple condition on the state determines which operator is appropriate to use, and the preconditions can be achieved in a fixed order. Second, in many cases, the unbound variables in a rule are either determined by early preconditions or can be bound arbitrarily. When these values must be chosen very carefully (e.g., when there are resource constraints), then a simple regression rule selector may not exist. For example, in Blocks World, to achieve clear(B), then the optimal way is to perform  $\text{unstack}(A, B)$ , where  $A$  is the object on  $B$ . Furthermore, in this case, the precondition order is fixed: clear(B)  $\iff \text{on}(A, B)$ , clear(A), handsfree()  $\| \text{unstack}(A, B)$  for arbitrary pairs of objects  $A, B$ . In the following, we consider cases where the regression rule selector can be computed with a first-order logic formula, therefore a finite-depth and finite-breadth RelNN circuit.
Serializability of RRS. Given a regression rule selector select, we can obtain a single trajectory (if  $g$  is reachable from  $s$  under constraints cons). We define  $tr^{select}(s, g, cons)$  as the trajectory returned by recursive application of select. It returns  $\bot$  if no plan can be found.
Definition 4.1 (RRS Serializability). A regression rule selector is serializable if and only if the following condition holds for any state  $s$ , any goal  $g$ , and any constraint set. Consider the regression rule returned by select  $(s, g, cons) \Leftrightarrow p_1, p_2, \dots, p_k \parallel a$ . If  $g$  is achievable from  $s$ , then it can be achieved via the concatenation of the following trajectories:  $\overline{a}_1 = tr^{\text{regress}}(s, p_1, \text{constraint}), \overline{a}_2 = tr^{\text{regress}}(\mathcal{T}(s, \overline{a}_1), p_2, \text{constraint} \cup \{p_1\}), \dots, \overline{a}_k$ , and  $\{a\}$ .
Compiling policy neural networks. If there exists a regression rule selector that is computable with a finite-depth, finite-breadth RelNN circuit, it will be possible to construct a policy for the original problem with another RelNN circuit. This construction can be dramatically more efficient than the general constructions in Section 4.2. Let  $k$  denote the "width" of the regression rule selector, i.e., the maximum size of the constraints we need to keep track of when applying select.
Theorem 4.3 (Compilation of S-GRS with a regression rule selector). Given a regression rule selector select (i.e., the state-constraint condition function  $\rho$ ) that can be computed by a relational neural network in  $RelNN[D_r, B_r]$ , for any planning problem  $P$ , if  $tr^{select}(s_0, g, \emptyset) \neq \bot$ , letting  $d$  be the depth of the regression tree, then there is a  $RelNN$  circuit for the problem in  $RelNN[O(d \cdot D_r), \max(B_r, \beta)]$ , where  $\beta$  is the maximum arity of predicates.
Proof. For any step  $i$ , let  $pre_{\leq i}$  be the length  $i$  prefix of the precondition list. We define  $Goal^d$  to be the single goal atom and set of constraints we need to satisfy at depth  $d$ . We have the following rules:  $\forall d.\forall i < |pre|.(Goal^d = (g, cons)) \land \rho(s, cons) \land pre_{\leq i} \land \neg pre_{i+1} \Rightarrow (Goal^{d+1} = (pre_{i+1}, cons \cup pre_{\leq i})$
$$
\forall d. \left(G o a l ^ {d} = (g, c o n s)\right) \wedge \rho (s, c o n s) \wedge p r e \Rightarrow a
$$
Here the evaluations of preconditions  $pre$  are based on the current state (input to the policy network) and they can be evaluated in parallel. We illustrate this construction in Fig. 3b. The first set of rules computes the next subgoal (and constraint set) to achieve, while the second set of rules outputs the next action when all preconditions of the current rule have been satisfied (which will be the output of the entire policy). Since at each layer  $d$ , we only need to keep track of one tuple of  $(g, cons)$ , the breadth of the circuit is  $\max(B_r, \beta)$ , where  $\beta$  is required to store the input state.
This construction yields a significant reduction in breadth (from tracking all conjunctions up to  $k_{BWD}$  atoms to only one tuple of  $(g, cons)$ ), as well as a reduction of the number of layers for the RelNN policy, depending on the domain. In particular, if the average number of preconditions that need to be recursively solved in each rule is  $b$  (roughly equal to the number of preconditions in each rule), the depth of the regression tree is  $d = O(\log_b T)$ , where  $T$  is the planning horizon.
Finally, for some problems, even if we have a regression rule selector, to compute a plan, we may still need an unbounded depth of search steps (i.e.,  $d$  depends on the number of objects in the domain). For example, in Blocks World, to achieve clear(A), the number of steps needed is proportional to the number of objects on top of A. There are two ways to build a relational neural network policy in this case. First, we can allow the network to have a variable number of iterations by applying the same layer recurrently several times. Second, there might be "shortcuts" that we can learn. We discuss both solutions in Appendix B.3. These methods allow us to construct finite-depth RelNN circuits for a variety of domains, including Blocks World, and path-finding in any acyclic maze.
5 Problem Analysis and Results
We first analyze the regression width and circuit complexities for familiar AI planning problems. Table 1 summarizes the results on a simplified setting where the goal of the planning problem is a
Problem	Constant Breadth?	Depth
BlocksWorld	✓ (k=1)	Unbounded
Logistics	✓ (k=0)	Unbounded
Gripper	✓ (k=0)	Constant
Rover	✓ (k=0)	Unbounded
Elevator	✓ (k=0)	Unbounded
Sokoban	✗	Unbounded
Table 1: Width and circuit complexity analysis for 6 problems widely used in AI planning communities. For problems with a constant circuit breadth, we also annotate their regression width  $k$ . "Unbounded" depth means that the depth depends on the number of objects.
Task	Model	n=10	n=30	n=50
As3	RelNN[1, 2]	0.2±0.05	0.02±0.01	0.0±0.0
As3	RelNN[2, 2]	1.0±0.0	1.0±0.0	1.0±0.0
Log.	RelNN[3, 2]	1.0±0.0	0.30±0.03	0.23±0.05
Log.	RelNN[f(n), 2]	1.0±0.0	1.0±0.0	1.0±0.0
BW	RelNN[3, 2]	1.0 / 2.9±0.4	1.0 / 10.2±0.45	1.0 / 15.7±2.5
BW	RelNN[f(n), 2]	1.0 / 2.5±0.4	1.0 / 3.1±0.5	1.0 / 3.5±0.4
Table 2: Success rate of learned policies in different environments. For Assembly3 (As3) and Logistics (Log.), we show the success rate. For Blocks World (BW), we show the success rate / average solution length. We choose  $f(n) \stackrel{\mathrm{def}}{=} n / 5 + 1$  for Logistics and  $f(n) \stackrel{\mathrm{def}}{=} n / 10 + 3$  for BlocksWorld. In the notation of RelNN[D,B],  $D$  is the number of layers and  $B$  is the maximum arity of edges.
single atom. In summary, most of these problems have a constant breadth (i.e., the regression width of the corresponding problem is constant) except for Sokoban. Most problems have an unbounded depth: that is, the depth of the circuit will depend on the number of objects (e.g., the size of the graph or the number of blocks). For problems in this list, when there are multiple goals, usually the goals are not serializable (in the optimal planning case). If we only care about satisficing plans, for Logistics, Gripper, Rover, and Elevator, there exists a simple serialization for any conjunction of goal atoms (basically achieving one goal at a time). See Appendix C for more detailed proofs and additional discussions of generalization to multiple goals and suboptimal plans.
Next, given this analytical understanding of the relationship between planning problems and their policy circuit complexity, we perform some simple experiments to see whether that relationship is borne out in practice. Relational neural networks have been demonstrated to be effective in solving some planning problems [Dong et al., 2019, Jiang and Luo, 2019, Li et al., 2020], but their complexity has not been systematically explored. We consider two families of problems: one predicted to require finite depth and one predicted to require unbounded depth. For all tasks, we use Neural Logic Machines [Dong et al., 2019] as the model; we set the number of hidden neurons in the MLP layers to be sufficiently large (64 in all experiments) so that it is not the bottleneck for network expressiveness. We use the same encoding style for all problems (the graph-like relationship). Therefore, whether different problems have bounded or unbounded width primarily depends on the available goal regression rules, how these rules can be serialized, and the width of the rules. We show the average performance across 3 random seeds, with standard errors.
Assembly3: finite depth circuits. The domain Assembly3 contains  $n$  objects. Each object has a category, chosen from the set  $\{A, B, C\}$ . The goal of the task is to select three objects  $o_A, o_B,$  and  $o_C$  sequentially, one for each category, while satisfying a matching constraint: match  $(o_A, o_B)$  and match  $(o_B, o_C)$ . Therefore, to select the first object (e.g., an object of type  $A$ ), the policy needs to perform two layers of goal regression (first find the set of possible  $B$ -typed objects that match the object, and then find another  $C$ -typed object). We trained two models. Both models have breadth 2, but the first model has only 1 layer (theoretically not capable of representing the two-step goal regression), while the second model has 2 layers. Shown in Table 2, we train models on environments with 10 objects and test them on environments with 10, 30, and 50 objects. The first model is not able to learn a policy for the given task, while the second exhibits perfect generalization.
Logistics: unbounded depth circuits. We also construct a simple Logistics domain with only cities and trucks (no airplanes), in which the graphs are not strongly connected (by first sampling a directed tree and then adding forward-connecting edges). We train the policy network on problems with less than  $n = 10$  cities and test it on  $n = 30$  and  $n = 50$ . Here, we trained two policies. The first policy has a constant depth of 3, and the second policy has a depth of  $D = n / 5 + 1$ , which is a manually chosen function such that it is larger than the diameter of the graph. Shown in Table 2, the first model fails to generalize to larger graphs due to its limited circuit depth. Intuitively, to find a path in a graph by recursively applying the regression rule, we need a circuit of an adaptive depth (proportional to the length of the path). By contrast, the second model with adaptive depths generalizes perfectly.
BlocksWorld-Clear: unbounded depth circuits. Based on the goal regression analysis, in BlocksWorld, in order to achieve the goal atom clear(A) for a specific block  $A$ , we would need
a circuit that is as deep as the number of objects on  $A$  in the initial state. We follow the same training and evaluation setup as in Logistics. Our analysis focuses on the length of the generated plan. Shown in Table 2, although both policy networks accomplish the task with a 1.0 success rate (because there is a trivial policy that randomly drops clear blocks onto the table), the policy whose depth depends on the number of blocks successfully finds a plan that is significantly shorter than fixed-depth policy. This suggests the importance of using RelNNs with a non-constant depth for certain problems.
6 Related Work and Conclusion
Most of the existing work on planning complexity considers the NP-Completeness or PSPACE-Completeness of particular problems, such as the traveling salesman problem [Karp, 1972], Sokoban [Culberson, 1997], and Blocks World [Gupta and Nau, 1992]. In general, the decision problem of plan existence is PSPACE-Complete for atomic STRIPS planning problems [Bylander, 1994]. Seminal work on fine-grained planning problem complexity introduces planning width [Chen and Giménez, 2007] and the IW algorithm [Lipovetzky and Geffner, 2012, Drexler et al., 2022]. Our work is greatly inspired by these ideas and strives to connect them with circuit complexity.
The concept of serialized goal regression is not completely new. These rules are particularly related to the methods in hierarchical task networks [Erol et al., 1994] and can also be seen as special cases of hierarchical goal networks [Alford et al., 2016], derived by dropping preconditions [Sacerdoti, 1974]. In addition, others, including Korf [1987] and Barrett and Weld [1993], have characterized different degrees of serializability of subgoals, but their analysis is not as fine-grained as this.
Our circuit complexity analysis builds on existing work on relational neural network expressiveness, in particular, GNNs and their variants. Xu et al. [2019] provides an illuminating characterization of GNN expressiveness in terms of the WL graph isomorphism test. Azizian and Lelarge [2021] analyze the expressiveness of higher-order Folklore GNNs by connecting them with high-dimensional WL-tests. Barceló et al. [2020] and Luo et al. [2022] reviewed GNNs from the logical perspective and rigorously refined their logical expressiveness with respect to fragments of first-order logic. Our work extends their results, asking the question of what planning problems a RelNN can solve.
Our work is also related to how GNNs may generalize to larger graphs (in our case, planning problems with an arbitrary number of objects). Xu et al. [2020, 2021] have studied the notion of algorithmic alignment to quantify such structural generalization. Dong et al. [2019] provided empirical results showing that NLMs generalize in Blocks World and many other domains. Buffelli et al. [2022] introduced a regularization technique to improve GNNs' generalization and demonstrated its effectiveness empirically. Xu et al. [2021] also showed empirically on certain algorithmic reasoning problems (e.g., max-Degree, shortest path, and the n-body problem). In this paper, we focus on constructing policies that generalize instead of algorithms, using a similar idea of compiling specific (search) algorithms into neural networks.
Conclusion. To summarize, we have illustrated a connection between classical planning width, regression width, search complexity, and policy circuit complexity. We derive upper bounds for search and circuit complexities as a function of the regression width of problems and planning horizon. These results provide an explanation of the success of relational neural networks learning generalizable policies for many object-centric domains. The compilation algorithms highlight that when there are resource constraints such as free space, agents' inventory size, etc., a small policy circuit may not exist. This includes cases such as Sokoban and general task and motion planning. Furthermore, our idea of serialization can be generalized to hierarchical decomposition of problems; it can be potentially extended to other domains such as optimization and planning under uncertainty.
Although all of the analyses in this paper have been done for fully discrete domains by analyzing first-order logic formulas, understanding the implications in relational neural network learning is important because, ultimately, RelNNs can be integrated with perception and continuous action policy networks to solve realistic robotic manipulation problems (e.g., physical Blocks World [Li et al., 2020]). It is challenging to directly generalize our proof to scenarios where states and actions are continuous values because in these cases, there will be an infinite number of possible actions (e.g., the choice of grasping poses, trajectories, etc.). We believe that it is an interesting question to investigate similar "width"-like definitions in sampling and discretization-based continuous planning algorithms. So far, we have only analyzed the circuit complexity of policies for individual planning problems; an interesting future direction is to extend these analyses to the learning and inference of "general" policies such as large language models that can generate plans for many different planning domains.