SlowGuess/NeurIPS_2023_OCR huggingface dataset:
In addition to the realizability of the  $IW(k + 1)$  algorithm, we need to additionally prove that, if we follow the policy derived by simulating  $IW(k + 1)$  for the first step, we can recurrently apply this policy. To see that, consider the path from  $s_0$  to the goal state in  $IW(k + 1)$  in the tuple graph. Since we are moving forward by one step in this tuple path, doing a search from the second tuple in the path (i.e., applying the same RelNN policy at the second world state) will yield the same trajectory.
Similar to the construction in Theorem 4.1, all the rules are lifted FOL rules and can be implemented by a RelNN circuit. It is also possible to use the same trick to compile the original S-GRS algorithm: essentially, for each  $(g, cons)$  pair, keep track of the optimal trajectory for achieving it. However, the construction will be more complicated and is omitted here.
B.3 More Discussion
For some problems, even if we have a regression rule selector, to compute a plan, we may still need an unbounded depth of search steps (i.e.,  $d$  depends on the number of objects in the domain). For example, in Blocks World, to achieve clear(A), the number of steps needed is proportional to the number of objects on top of A. There are two ways to build a relational neural network policy in this case. First, we can allow the network to have a variable number of iterations by applying the same layer recurrently several times. We have already illustrated this in the experiment.
Second, there might be "shortcuts" that we can learn. An intuitive example is that in BlocksWorld, the goal regression for clear(A) will recur until we have found the topmost block stacked on top of A. If the domain has a new predicate above(x,y) indicating whether the block  $x$  is above another block  $y$ , then we can use just one single rule to predict the topmost block on top of A: the topmost block B on A satisfies above(B,A) ∧ clear(B).
The same technique also applies to the problem of path-finding in any acyclic maze. Recall that a classical solution to any acyclic maze is that we only make right turns. This can be cast as a linearization of the entire maze by doing a depth-first search from the start or the goal location. In particular, starting from the goal location if we only do right turns, we will get a sequence of the edges of the maze such that the start edge is one of the elements (if there is a path from the start to the goal). Therefore, doing goal regression under this linearized maze is trivial: the entire "regression tree" is just a chain of edges and there is a simple shortcut for predicting the next move.
C Analysis of AI Planning Problems
Here, we list six popular problems that have been widely used in AI planning communities and discuss their circuit depth and breadth. We summarize our results in the Table 3.
Problem	Breadth	Regression Width	Depth
BlocksWorld	Constant	1	Unbounded
Logistics	Constant	0	Unbounded
Gripper	Constant	0	Constant
Rover	Constant	0	Unbounded
Elevator	Constant	0	Unbounded
Sokoban	Unbounded	N/A (not serializable)	Unbounded
Table 3: Width and circuit complexity analysis for 6 problems widely used in AI planning communities. This table contains the same content as Table 1; we include it here for easy reference.
Note that here, we are limiting our discussion to the case where the goal of the planning problem is a single atom. In summary, most of these problems have a constant breadth (i.e., the regression width of the corresponding problem is constant) except for Sokoban. Most problems have an unbounded depth: that is, the depth of the circuit will depend on the number of objects (e.g., the size of the graph or the number of blocks. The problem gripper has a constant depth under a single-atom goal because it assumes the agent can move directly between any two rooms; therefore, no pathfinding is required. Sokoban has an unbounded breadth even for single-atom goals because if there are multiple boxes blocking the way to a designated position, the order to move the boxes can not be determined by simple rules. The constant breadth results can be proved by construction: list all regression rules in the domain. In the following Appendix C.1, we first discuss the SOS width of two representative problems, Blocks World and Logistics. All other problems can be analyzed in a similar way.
Before we delve into concrete complexity proofs, we would like to add the note that for problems in this list, when there are multiple goals, usually the goals are not serializable (in the optimal planning case). This can be possibly addressed by introducing "super predicates" that combine two literals. For example, if two goal atoms  $p(x)$  and  $q(x)$  are not serializable, we can introduce a new predicate  $p$  and  $q(x)$ , and rewrite all operators with this new super predicate. Appendix A.3 provides more discussion and examples. This will make the problem serializable but at the cost of significantly enlarging the set of predicates (exponentially with respect to the number of goal atoms).
If we only care about satisficing plans, for Logistics, Gripper, Rover, and Elevator, there exists a simple serialization for any conjunction of goal atoms (basically achieving one goal at a time).
These theoretical analyses can be used in determining the size of the policy circuit needed for each problem. For example, one should use a RelNN of breadth  $(k + 1)\cdot \beta$ , where  $k$  is the regression width and  $\beta$  is the max arity of predicates. For problems have unbounded depth, usually the depth of the circuit grows in  $O(N)$ , where  $N$  is the number of objects in the environment (e.g., in Elevator, the number of floors).
C.1 SOS Width Analysis
To prove that a problem can be solved optimally using S-GRS, we only need to show that there exists a goal regression rule selector. Therefore, we will be using the syntax from Section 4 to write down the set of goal regression rule selectors.
Finite SOS width for Blocks World.
$$
\forall c o n s. \forall x. \forall y. o n (y, x) \in s _ {0} \quad c l e a r (x) ^ {c o n s} \leftrightarrow o n (y, x), c l e a r (y), h a n d s f r e e () \| u n s t a c k (y, x)
$$
$$
\forall \operatorname {c o n s}. \forall x. \operatorname {h o l d i n g} (x) \in s _ {0} \operatorname {c l e a r} (x) ^ {\text {c o n s}} \Leftrightarrow \operatorname {h o l d i n g} (x) \| \text {p l a c e - t a b l e} (x)
$$
$$
\forall \operatorname {c o n s}. \forall x. \forall y. \operatorname {o n} (x, y) \in s _ {0} h o d i n g (x) ^ {\text {c o n s}} \Leftrightarrow \operatorname {o n} (x, y), \operatorname {c l e a r} (x), \operatorname {h a n d s f r e e} () \| \operatorname {u n s t a c k} (x, y)
$$
$$
\forall c o n s. \forall x. o n - t a b l e (x) \in s _ {0} h o d i n g (x) ^ {c o n s} \Leftrightarrow o n - t a b l e (x), c l e a r (x) \| p i c k - t a b l e (x)
$$
$$
\forall c o n s. \forall x. o n t - t a b l e (x) ^ {c o n s} \Leftrightarrow h o l d i n g (x) \| p l a c e - t a b l e (x)
$$
$$
\forall c o n s. \forall x. \forall y. o n (x, y) ^ {c o n s} \Leftrightarrow c l e a r (y), h o l i n g (x) \| \text {s t a c k} (x, y)
$$
$$
\forall \text {c o n s .} \forall x. \text {h o l d i n g} (x) \in s _ {0} \text {h a n d s f r e e}) ^ {\text {c o n s}} \Leftrightarrow \text {h o l d i n g} (x) \| \text {p l a c e - t a b l e} (x)
$$
Finite SOS width for Logistics.
$$
\forall \text {c o n s .} \forall v. \forall o. \forall \ell . a t (o, \ell) ^ {\text {c o n s}} \leftrightarrow i n (o, v), a t (v, \ell) \| \text {u n l o a d} (y, x)
$$
$$
\forall \operatorname {c o n s}. \forall v. \forall o. \forall \ell . i n (o, v) ^ {\text {c o n s}} \Leftrightarrow a t (o, \ell), a t (v, \ell) \| l o a d (y, x)
$$
$$
\forall c o n s. \forall v. \forall \ell_ {1}. \forall \ell_ {2}. \forall c. a t (v, \ell_ {2}) ^ {c o n s} \Leftrightarrow l o c (\ell_ {1}, c), l o c (\ell_ {2}, c), a t (o, \ell_ {1}) \| d r i v e (v, \ell_ {1}, \ell_ {2}, c)
$$
$$
\forall c o n s. \forall v. \forall a _ {1}. \forall a _ {2}. a t (v, a _ {2}) ^ {c o n s} \Leftrightarrow a t (v, a _ {1}) \| f l y (v, a _ {1}, a _ {2})
$$
Here, we have omitted static properties for type checking (such as  $a_1$  and  $a_2$  need to be airports).
Environment setup. For the Assembly3 task, we first uniformly sample the set of type-A, type-B, and type-C object. Then, we randomly add match relationships so that there will be exactly only one tuple of  $(a,b,c)$  that satisfies the matching condition.
For the BlocksWorld-Clear task, we use the same random sampler for initial configurations as Neural Logic Machines [Dong et al., 2019]. It iteratively adds blocks to an empty initial state. At each step, it randomly selects a clear object (or the table), and places a new object on top of it. After generating the initial state, we randomly sample an object that is not clear in the initial state as the target block.
For the Logistics task, we first randomly generate a tree rooted at the start node. This is done by iteratively attaching new nodes to a leaf node. Next, we randomly add a small number of edges to the tree. Finally, we randomly pick a node as the target node. Since we are only a small number of additional edges added to the tree, with high probability, the starting node and the end node are not in the same strongly connected component.
Additional hyperparameters. The breadth, width, and hidden dimension parameters for neural networks have been specified in the experiment section and they vary across tasks. For all experiments, we use the Adam optimizer to train the neural network, with a learning rate of  $\epsilon = 0.001$ . Additionally, we set  $\beta_{1} = 0.9$  and  $\beta_{2} = 0.999$ .
Compute. We trained with 1 NVIDIA Titan RTX per experiment for all datasets, from an internal cluster.
Code. We release the code for reproducing all our experimental results in https://github.com/concepts-ai/goal-regression-width.