Title: What Planning Problems Can A Relational Neural Network Solve?
Venue: NeurIPS 2023
Accepted: yes
Keywords: Planning

Relational Neural Network

Circuit Complexity
Abstract: Goal-conditioned policies are generally understood to be "feed-forward" circuits, in the form of neural networks that map from the current state and the goal specification to the next action to take. However, under what circumstances such a policy can be learned and how efficient the policy will be are not well understood. In this paper, we present a circuit complexity analysis for relational neural networks (such as graph neural networks and transformers) representing policies for planning problems, by drawing connections with serialized goal regression search (S-GRS). We show that there are three general classes of planning problems, in terms of the growth of circuit width and depth as a function of the number of objects and planning horizon, providing constructive proofs. We also illustrate the utility of this analysis for designing neural networks for policy learning.

Jiayuan Mao

Abstract

Goal-conditioned policies are generally understood to be 'feed-forward' circuits, in the form of neural networks that map from the current state and the goal specification to the next action to take. However, under what circumstances such a policy can be learned and how e ffi cient the policy will be are not well understood. In this paper, we present a circuit complexity analysis for relational neural networks (such as graph neural networks and transformers) representing policies for planning problems, by drawing connections with serialized goal regression search (S-GRS). We show that there are three general classes of planning problems, in terms of the growth of circuit width and depth as a function of the number of objects and planning horizon, providing constructive proofs. We also illustrate the utility of this analysis for designing neural networks for policy learning.

1 Introduction

Goal-conditioned policies are generally understood to be 'feed-forward' circuits, in the form of neural networks such as multi-layer perceptrons (MLPs) or transformers [Vaswani et al., 2017]. They take a representation of the current world state and the goal as input, and generate an action as output. Previous work has proposed methods for learning such policies for particular problems via imitation or reinforcement learning [Wang et al., 2018, Dong et al., 2019, Li et al., 2020]; recently, others have tried probing current large-language models to understand the extent to which they already embody policies for a wide variety of problems [Liang et al., 2022, Carta et al., 2023].

However, a major theoretical challenge remains. In general, we understand that planning problems are PSPACE-hard with respect to the size of the state space [Bylander, 1994], but there seem to exist e ffi cient (possibly suboptimal) policies for many problems such as block stacking [Dong et al., 2019] that generalize to arbitrarily sized problem instances. In this paper, we seek to understand and clarify the circuit complexity of goal-conditioned policies for classes of 'classical' discrete planning problems: under what circumstances can a polynomial-sized policy circuit be constructed, and what is its size? Specifically, we highlight a relationship between a problem hardness measure ( regression width , related to the notion of 'width' in the forward planning literature [Chen and Giménez, 2007]) and circuit complexity. We concretely provide upper bounds on the policy circuit complexity as a function of the problem's regression width, using constructive proofs that yield algorithms for directly compiling a planning problem description into a goal-conditioned policy. We further show that such policies can be learned with conventional policy gradient methods from environment interactions only, with the theoretical results predicting the necessary size of the network.

There are several useful implications of these results. First, by analyzing concrete planning domains, we show that for many domains, there do exist simple policies that will generalize to problem instances with an arbitrary number of objects. Second, our theory predicts the circuit complexity of policies for these problems. Finally, our analysis suggests insights into why certain planning

//

concepts-ai.com

/

p

/

goal-regression-width

/

(a) Blocks World Visualization and its Representation

(b) Action schemas in Blocks World

pick-table

(x):

pre:

on-table(x),

clear(x),

handsfree

eff+:

holding(x)

eff-:

on-table(x),

clear(x),

handsfree

place-table

(x):

pre:

holding(x)

eff+:

on-table(x),

clear(x),

handsfree

eff-:

holding(x)

unstack

(x,

y):

#

x

on

y

pre:

on(x,

y),

clear(x),

handsfree

eff+:

holding(x),

clear(y)

eff-:

on(x,

y),

clear(x),

handsfree

stack

(x,

y):

#

x

on

y

pre:

holding(x),

clear(y)

eff+:

clear(y),

holding(x)

eff-:

holding(x),

clear(y)

clear(B)

unstack(C,

B)

{on(C,

B),

clear(C),

handsfree}

{on(C,

B),

clear(C),

holding(D),

clear(E)}

stack(D,

E)

{on(C,

B),

clear(E),

on(D,

C),

clear(D),

handsfree}

unstack(D,

C)

unstack(D,

B)

place-table

(B)

......

......

clear(B)

unstack(C,

B)

on(C,

B),

clear(C),

handsfree

unstack(D,

B)

place-table

(B)

on(D,

C),

clear(D),

handsfree

place-table

(D)

holding(D)

unstack(D,

C)

......

......

(d) Goal Regression Search with Serialization

Next Action

(e) Compiled RelNN Policy

{

on(C,

B)

}

{

on(C,

B), clear(C)

}

𝓤 =

{

A, B, C, D, E

}

𝓟 =

{

handsfree,

on-table(A),

on-table(B),

on-table(E),

on(C,

B),

on(D,

C),

clear(A), clear(D),

clear(E)

}

Tensor Representation

handsfree

on

on-goal

...

unstack(D,

C)

Relational Neural Network

(e.g., GNNs, Transformers)

problems, such as Sokoban and general task and motion planning (TAMP) problems, are hard, and likely cannot be solved in general by fixed-sized MLP or transformer-based policies [Culberson, 1997, Vega-Brown and Roy, 2020]. In the rest of the paper, Section 2 provides problem definitions, Sections 3 and 4 provides complexity definitions, theoretical results, and the policy compilation algorithm; Section 5 discusses the practical implications of these results.

2 Preliminaries: Planning Domain and Problem

Throughout this paper, we focus on analyzing the search complexity and policy circuit complexity of classical planning problems. Importantly, these planning problems have an object-centric representation and sparsity in the transition models: the state of the world is represented as a set of entities and their properties and relationships, while each action only changes a sparse set of properties and relations of a few objects. These features will contribute to search e ffi ciency.

Formally, we consider the problem of planning and policy learning for a space S of world states. A planning problem is a tuple ⟨S , s 0 , G , A , T⟩ , where s 0 ∈ S is the initial state, G ⊆ S is a goal specification, A is a set of actions that the agent can execute, and T is a (possibly partial) environmental transition model T : S × A → S . The task of planning is to output a sequence of actions ¯ a in which the terminal state sT induced by applying ai sequentially following T satisfies sT ∈ G . The task of policy learning is to find a function π ( s , G ) that maps from the state and a goal specification to the next action so that applying π recurrently on s 0 eventually yields a state sT ∈ G .

We use atomic STRIPS [Fikes and Nilsson, 1971, Lifschitz, 1987] to formalize the object-centric factorization of our planning problems. Specifically, as illustrated in Fig. 1a, the environmental state can be represented as a tuple ( U s , P s ), where U s is the set of objects in state s , denoted by arbitrary unique names (e.g., A, B). The second component, P s , is a set of atoms : handfree () , on-table (A) , on (C , B), etc. Each atom contains a predicate name (e.g., on ) and a list of arguments (e.g., C, B). All atoms in P s are true at state s , while any atoms not in P s are false. Since we do not consider object creation and deletion, we simply use P 0 to denote the set of all possible atoms for the object universe U s 0 * .

Illustrated in Fig. 1b, a (possibly partial) transition model T is specified in terms of a set of objectparameterized action schemas ⟨ name , args , precond , e ff ect ⟩ , where name is the name of the action, args is a list of symbols, precond and e ff ect are descriptions of the action's e ff ects. In the basic STRIPS formulation, both precond and e ff ect are sets of atoms with variables in args , and e ff ect is further decomposed into e ff + and e ff -, denoting 'add' e ff ects and 'delete' e ff ects. An action schema (e.g., unstack ) can be grounded into a concrete action a (e.g., unstack (D, C)) by binding all variables in args .

The formal semantics of STRIPS actions is: ∀ s . ∀ a . pre ( a ) ⊆ s = ⇒ GLYPH<0> T ( s , a ) = s ∪ e ff + ( a ) \ e ff -( a ) GLYPH<1> . That is, for any state s and any action a , if the preconditions of a are satisfied in s , the state resulting from applying a to s will be s ∪ e ff + ( a ) \ e ff -( a ). Note that T may not be defined for all ( s , a ) pairs. Furthermore, we will only consider cases where the goal specification is a single atom, although more complicated conjunctive goals or goals that involve existential quantifiers can be easily supported by introducing an additional 'goal-achieve' action that lists all atoms in the original goal as its precondition. From now on, we will use g to denote the single goal atom of the problem.

3 Goal Regression Search and Width

Asimple and e ff ective way to solve STRIPS problems is backward search † . Illustrated in Algorithm 1 ‡ , we start from the goal { g } , and search for the last action a that can be applied to achieve the goal (i.e., g ∈ e ff + ( a )). Then, we add all the preconditions of action a to our search target and recurse. To avoid infinite loops, we also keep track of a ' goal stack ' variable composed of all goal_set 's that have appeared in the depth-first search stack. The run-time of this algorithm has critical dependencies on (1) the number of steps in the resulting plan and (2) the number of atoms in the intermediate goal sets in search. In general, it has a worst-case time complexity exponential in the number of atoms.

3.1 Serialized Goal Regression Search

One possible way to speed up backward search is to serialize the subgoals in the goal set: that is, to commit to achieving them in a particular order. In the following sections, we study a variant of the plain backward search algorithm, namely serialized goal regression search (S-GRS, Algorithm 2), which uses a sequential decomposition of the preconditions: given the goal atom g , it searches for the last action a and also an order in which the preconditions of a should be accomplished. Of course, this method cannot change the worst-case complexity of the problem (actually, it may slow down the algorithm in cases where we need to search through many orders of the preconditions), but there are useful subclasses of planning problems for which it can achieve a substantial speed-up.

A serialized goal regression rule in R , informally, takes the following form: g f p 1 , p 2 , · · · , pk ∥ a . It reads: in order to achieve g , which is a single atom, we need to achieve atoms p 1 , · · · , pk sequentially, and then execute action a . To formally define the notion of achieving atoms p 1 , · · · , pk ' sequentially ,' we must include constraints in goal regression rules. Formally, we consider the application of a rule under constraint c to have the following definition, g c f p c 1 , p c ∪{ p 1 } 2 , · · · , p c ∪{ p 1 , ··· pk -1 } k ∥ a which reads: in order to achieve g without deleting atoms in set c , we can first achieve p 1 while maintaining c , then achieve p 2 while maintaining c and p 1 , . . . , until we have achieved pk . Finally, we perform a to achieve g . For example, in Blocks World, we have the rule: clear (B) ∅ f on (A , B) , clear (A) { on (A , B) } , handsfree () { on (A , B) , clear (A) } ∥ unstack (A , B). Note that many rules might be infeasible for a given state, because it is simply not possible to achieve pi while

maintaining c and { p 1 , · · · , pi -1 } . In this case, achieving handsfree () before clear (A) is infeasible because handsfree () may be falsified while achieving clear (A), if we need to move other blocks that are currently on A. Similarly, on (A , B) must be achieved before clear (A) because moving A onto B will break clear (A) (because we need to pick up A).

The set of all possible goal regression rules can be instantiated based on all ground actions A in a planning problem. Specifically, for a given atom g (e.g., holding (A)) and a set of constraints c , and for each action a ∈ A (e.g., pick-table (A)), if g ∈ e ff + ( a ) while c ∩ e ff -( a ) = ∅ , then for any permutation of pre ( a ), there will be a goal regression rule: g c f p 1 , · · · , pk ∥ a , where p 1 , · · · , pk is a permutation of pre ( a ). We call this regression rule set R 0 .

Given a set of regression rules (e.g., R 0), we can apply the serialized goal-regression search (S-GRS) algorithm, shown in Algorithm 2. S-GRS returns a shortest path from state to achieve goal while maintaining cons . Unfortunately, this algorithm is not optimal nor even complete (e.g., a sequence of preconditions may not be 'serializable,' which we will define formally later), in the general case. To make it complete, it is necessary to backtrack through di ff erent action sequences to achieve each subgoal, which increases time complexity. We include a discussion in Appendix A.1.

3.2 Serialization of Goal Regression Rules

Although Algorithm 2 is not complete in general, it provides insights about goal regression. In the following, we will introduce two properties of planning problems such that S-GRS becomes optimal, complete, and e ffi cient. For brevity, we define OptSearch ( state , goal , cons ) as the set of optimal trajectories that achieve goal , from state , while maintaining cons . Here, goal can be a conjunction.

Definition 3.1 (Optimal serializability) . A goal regression rule g cons f p 1 , · · · , pk ∥ a is optimally serializable w.r.t. a state s if and only if, for all steps i, if ∀ π< i ∈ OptSearch ( s , p < i ∧· · ·∧ pi -1 , cons ) and ∀ π i ∈ OptSearch ( T ( s , π 1 ) , pi , cons ∪{ p 1 , · · · , pi -1 } ) then π< i ⊕ π i ∈ OptSearch ( s , p 1 ∧·· · ∧ pi , cons ) . Furthermore, ∀ π ∈ OptSearch ( s , p 1 ∧ · · · ∧ pk , cons ) , π ⊕ a ∈ OptSearch ( s , g , cons ) § .

Intuitively, a rule is optimally serializable if any optimal plan for the length i -1 prefix of its preconditions can be extended into an optimal plan for achieving the length i prefix. For example, in Blocks World, the rule holding (A) f on (A , B) , clear (A) , handsfree () ∥ unstack (A , B) is optimally serializable when on (A , B) is true for s . We define the set of optimally serializable rules for a state s as OSR ( s ), and the set of single-literal goals that can be solved with OSR ( · ) from s 0 as OSG ( s 0 ) ¶ .

Theorem 3.1. For any goal g ∈ OSG ( s 0 ) , S-GRS is optimal and complete. See proof in Appendix A.2.

3.3 Width of Search Problems

Chen and Giménez [2007] introduced the notion of the width of a planning problem and showed that the forward-search complexity is exponential in the width of a problem. We extend this notion to regression-based searches and introduce a generalized version of regression rules in which not all previously achieved preconditions must be explicitly maintained.

Definition 3.2 (Generalized regression rules) . A generalized, optimally-serializable regression rule is g c f p c ∪ c 1 1 , p c ∪ c 2 2 , · · · , p c ∪ ck k ∥ a, where ci ⊆ { p 1 , p 2 , · · · , pi -1 } . Its width is | c | + max {| c 1 | , · · · , | ck |} .

For example, for any initial state s 0 that satisfies on (A , B), the following generalized rule is optimally serializable: clear (B) ∅ f on (A , B) ∅ , clear (A) ∅ , handsfree () { clear (A) } ∥ unstack (A , B). When we plan for the second precondition clear (A), we can ignore the first condition on (A , B), because the optimal plan for clear (A) will not change on (A , B). Using such generalized rules to replace the default rule set R 0 improves search e ffi ciency by reducing the number of possible subgoals. To define existence conditions for highly e ffi cient search algorithms and policies, we need a stronger notion:

Definition 3.3 (Strong optimally-serializable (SOS) width of regression rules) . A generalized regression rule has strong optimally-serializable width k w.r.t. state s if (a) it is optimally serializable, (b) its width is k, and, (c) ∀ cOptSearch ( s , pi ∪ ci ∪ c , ∅ ) ⊆ OptSearch ( s , { p 1 , · · · , pi } ∪ c , ∅ ) .

In the Blocks World domain, given a particular state s , if on (A , B) ∈ s , the generalized goal regression rule clear (B) f on (A , B) , clear (A) , handsfree () { clear (A) } ∥ unstack (A , B) is strong optimallyserializable, because the optimal trajectory to achieve clear (A) will not move A.

Definition 3.4 (SOS width of problems) . A planning problem P = ⟨S , s 0 , g , A , T⟩ has strong optimally-serializable width k if there exists a set of strong optimally-serializable width k rules R w.r.t. s 0 , such that S-GRS can solve P using only rules in R .

Theorem 3.2. If a problem P is of SOS width k, it can be solved in time O ( N k + 1 ) with s -grs , where N is the number of atoms. Here we have omitted polynomials related to enumerating all possible actions and their serializations (which are polynomial w.r.t. the number of objects in U ).

Proof idea. We start by assuming knowing R : the proof can be simply done by analyzing function calls: there are at most O ( N k + 1 ) possible argument combinations of ( goal , cons ). Next, since the enumeration of all possible widthk rules can be done in polynomial time. Therefore, the algorithm runs in polynomial time and it does not need to know R a priori. Note that the SOS condition cannot be removed because even if regression rules have a small width, there will be a possibly exponential branching factor caused by 'free variables' in regression rules that do not appear in the goal atom. See the full proof in Appendix A.4. We also describe the connections with classical planning width and other related concepts, as well as how to generalize to ∀ -quantified preconditions in Appendix A.5.

Importantly, planning problems such as Blocks World and Logistics have constant SOS widths, independent of the number of objects. Therefore, there exist polynomial-time algorithms for finding their solutions. Our analysis is inspired by and closely related to classical (forward) planning width Chen and Giménez [2007] and Lipovetzky and Ge ff ner [2012]. Indeed, we have:

Theorem 3.3. Any planning problem that has SOS-width k has a forward width of at most k + 1 and, hence, can be solved by the algorithm IW ( k + 1) . See the full proof in Appendix A.6.

Remark. Unfortunately, SOS width and forward width are not exactly equivalent to each other. There exist problems whose forward width is smaller than k + 1, where k is the SOS width. Appendix A.6 presents a concrete example. However, there are two advantages of our SOS width analysis over the forward width analysis. In particular, proofs for (forward) widths of planning problems were mostly done by analyzing solution structures. By contrast, our constructions form a new perspective: a problem has a large width if the number of constraints to track during a goal regression search is large . This view is helpful because our analyses can now focus on concrete regression rules for each individual operator in the domain - how its preconditions can be serialized. Second and more importantly, our 'backward' view allows us to reason about circuit complexities of policies.

4 Policy Realization

We have studied sequential backward search algorithms for planning; now, we address the questions of how best to 'compile' or 'learn' them as feed-forward circuits representing goal-conditioned policies, and how the size of these circuits depends on properties of the problem. Here, we will focus on understanding the complexity of a certain problem class ; that is, a set of problems in the same domain (e.g., Blocks World) with a similar goal predicate (e.g., having one particular block clear). In particular, we will make use of relational neural networks ( RelNN , such as graph neural networks [Morris et al., 2019] and Transformers [Vaswani et al., 2017]). They accept inputs of arbitrary size, which is critical for building solutions to problems with arbitrary-sized universes. Another critical fact is that these networks have 'parameter-tying' in the sense that there is a constant-sized set of parameters that is re-used according to a given pattern to realize a network.

In particular, as illustrated in Fig. 2, we will show how to take the set of operator descriptions for a planning domain and construct a relational neural network that represents a goal-conditioned policy: it takes a state and a goal (encoded as a special predicate, e.g., on-goal ) and outputs a ground action. Here, the state is represented as a graph (objects are nodes, and relations are edges; possibly there will be hyperedges for high-arity relations). For example, the input contains nullary features (a single vector for the entire environment), unary features (a vector for each object), binary features (a vector for each pair of objects), etc. The goal predicate can be represented similarly. These inputs are usually represented as tensors. The complexity of this neural network will depend on the SOS width of problems in the problem class of interest.

We will develop two compilation strategies. First, in Section 4.2, we directly compile the BWD and the S-GRS algorithms into RelNN circuits. For problems with constant SOS width that is independent of the size of the universe, there will exist finite-breadth circuits (but the depth may depend on the size of the universe). In Section 4.3, we discuss a new property of planning problems-the existence of regression rule selector circuits-which results in policy circuits that are smaller than those generated

by previous strategies, and they are potentially of finite depth. The construction of the regression rule selector is not automatic and will generally require human priors or machine learning.

4.1 Relational Neural Network Background

We first quantify what a relational neural network can compute, using the formalization developed by Luo et al. [2022]; see also Morris et al. [2019], Barceló et al. [2020]. Let depth D be the number of layers in a RelNN, and breadth B be the maximum arity of the relations (hyperedges) in the network. For example, to represent a vector embedding for each tuple of size 5, f ( x 1 , x 2 , · · · , x 5), where x 1 , · · · , x 5 ∈ U are entities in the planning problem, we need a relational neural network with breadth 5. We will only consider networks with a constant breadth. We denote the family of relational neural networks with depth D and breadth B as RelNN [ D , B ]. We will not be modeling the actual 'hidden dimension' of the neural network layers (i.e., the number of neurons inside each layer), but we will assume that it is bounded (for

Nullary Predicates

Unary Predicates

handsfree

on-table

holding

clear

Binary Predicates

on

on-goal

Nullary Actions

Unary Actions

Binary Actions

stack

unstack

RelNN

Environment

State

Action

example, as a function of B and the number of predicates in the domain). Under such assumptions (most practical graph neural networks, Transformers, and hypergraph neural networks do follow these assumptions), we have the following lemma.

Lemma 4.1 (Logical expressiveness of relational neural networks [Luo et al., 2022, Cai et al., 1992]) . Let FOCB denote a fragment of first-order logic with at most B variables, extended with counting quantifiers of the form ∃ ≥ n ϕ , which state that there are at least n nodes satisfying formula ϕ .

(Upper Bound) Any function f in FOCB can be realized by RelNN [ D , B ] for some D.

(Lower Bound) There exists a function f in FOCB such that for all D, f cannot be realized by RelNN [ D , B -1] .

Here, we sketch a constructive proof for using RelNNs to realize FOL formulas. The breadth B is analogous to the number of variables in FOL for encoding the value of the expression; the depth D is the number of 'nested quantifiers.' For example, the formula ∃ x . ∀ y . ∀ z . p ( x , y , z ) needs 3 layers, one for each quantifier. Furthermore, we call a RelNN finite breadth (depth) if B ( D ) is independent of the number of objects. Otherwise, we call it unbounded breadth (depth).

4.2 Compilation of BWD and S-GRS

Lemma 4.1 states an 'equivalence' between the expressive power of relational neural networks and first-order logic formulas. In the following, we take advantage of this equivalence to compile search algorithms into relational neural networks. First, we have the following theorem.

Theorem 4.1 (Compilation of BWD) . Given a planning problem P, let T be the length of the optimal trajectory (the planning horizon), kBWD be the maximum number of atoms in the goal set in bwd , and β be the maximum arity of atoms in the domain. The backward search algorithm bwd can be compiled into a relational neural network in RelNN [ O ( T ) , β · kBWD ] .

Proof sketch. We provide a construction in which the RelNN computes a set of subgoals (i.e., a set of sets of ground atoms) Goal d at each layer d . Initially, Goal 0 = {{ g }} . Then, sg ∈ Goal d if there is a path of length d from any state s that satisfies sg ⊆ s to g . See Appendix B.1 for the full proof.

Although this construction is general and powerful, it is unrealistic for large problems because the depth can be exponential in the number of objects, and the number of atoms in the subgoal conjunctive formulas can be exponential in the depth. Therefore, in the following, we will leverage the idea of serialized goal regression to make a more e ffi cient construction.

Theorem 4.2 (Compilation of S-GRS) . Given a planning problem P of SOS width k, let T be the length of the optimal trajectory, and β be the maximum arity of atoms in the domain. The serialized goal regression search S-GRS can be compiled into a RelNN [ O ( T ) , ( k + 1) · β ] , where k + 1 ≤ kBWD. See Appendix B.2 for the full proof using a similar technique as in Theorem 4.1.

(b) Compilation of rule selectors into First-Order Logic formulas.

Remark. The compilation in Theorem 4.2 can generate finite-breadth, unbounded-depth RelNN circuits for more problems than the compilation in Theorem 4.1. For example, in Blocks World, the number of atoms in the goal sets in bwd is unbounded. However, since the problem is of constant SOS width, it can be compiled into a finite-breadth circuit. When the search horizon T is finite, both the SOS width and k BWD will be finite because there are only a constant number of actions that can be applied to update the goal set / constraint set. So a problem can be compiled into a finite-depth, finite-breadth RelNN circuit with BWD compilation if and only if it can be compiled with S-GRS compilation, although S-GRS compilation may generate smaller circuits.

4.3 Compilation of S-GRS with a Regression Rule Selector

Unfortunately, both constructions in the previous section require a depthO ( T ) RelNN circuit, which is impractical for most real-world problems, as T can be exponential in the number of objects in a domain. In order to compile the goal regression search algorithm into a smaller circuit, we consider scenarios where the rule used to construct the (optimal) plan can be computed without doing a full search. Here the idea is to bring in human priors or machine learning to learn a low-complexity regression rule selector that directly predicts which goal regression rule to use. There exists such a rule selector for some problems (e.g., Blocks World) but not others (e.g., Sokoban).

Formally, we define a state-dependent regression rule selector (RRS) select ( s , g , cons ), which returns an ordered list of preconditions and a ground action a . Its function body can be written in the following form: g cons f if ρ ( s , cons ) : p cons 1 , p cons ∪{ p 1 } 2 , · · · , p cons ∪{ p 1 , ··· , pm -1 } m ∥ a , which reads: in order to achieve g under constraint cons at state s , if ρ ( s , cons ) is true, we can apply the regression rule p 1 , p 2 , · · · , pm ∥ a . Formally, select is composed of a collection of rules. Each rule is a tuple of ⟨ args , ρ, pre , a , g , cons ⟩ , where args is a set of variables, ρ is a FOL formula that can be evaluated at any given state s , pre is an ordered list of preconditions, a is a ground action, g is the goal atom, and cons is the constraint. ρ, pre , a , g , and cons may contain variables in args . Such a rule selector can be implemented by hand (as a set of rules, as illustrated in Fig. 3a for the simple BlocksWorld domain), or learned by a RelNN model.

The main computational advantage is that now, for a given tuple ( s , g , cons ), there is a single ordered list of preconditions and final action. Therefore, given the function select , we can simply construct a sequence of actions that achieves the goal by recursively applying the rule selector. In this section, we will first discuss scenarios where such rule selectors can be computed with shallow RelNN circuits.

Circuit complexity of regression rule selectors. We first discuss scenarios where there exist finite-depth, finite-breadth circuits for computing the regression rule selector. Note that there are three 'branching' factors in choosing the regression rule: 1) the operator to use, 2) the order of the preconditions, and 3) the binding of 'free' variables that are not mentioned in the goal atom.

First, in many cases, a simple condition on the state determines which operator is appropriate to use, and the preconditions can be achieved in a fixed order. Second, in many cases, the unbound variables in a rule are either determined by early preconditions or can be bound arbitrarily. When these values must be chosen very carefully (e.g.,when there are resource constraints), then a simple regression rule selector may not exist. For example, in Blocks World, to achieve clear (B), then the optimal way is to perform unstack ( A , B ), where A is the object on B. Furthermore, in this case, the precondition order is fixed: clear (B) f on (A , B) , clear (A) , handsfree () ∥ unstack (A , B) for arbitrary pairs of objects A , B. In the following, we consider cases where the regression rule selector can be computed with a first-order logic formula, therefore a finite-depth and finite-breadth RelNN circuit.

Serializability of RRS. Given a regression rule selector select , we can obtain a single trajectory (if g is reachable from s under constraints cons ). We define tr select ( s , g , cons ) as the trajectory returned by recursive application of select . It returns ⊥ if no plan can be found.

Definition 4.1 (RRS Serializability) . A regression rule selector is serializable if and only if the following condition holds for any state s, any goal g, and any constraint set. Consider the regression rule returned by select ( s , g , cons ) f p 1 , p 2 , · · · , pk ∥ a. If g is achievable from s, then it can be achieved via the concatenation of the following trajectories: a 1 = tr regress ( s , p 1 , constraint ) , a 2 = tr regress ( T ( s , a 1 ) , p 2 , constraint ∪ { p 1 } ) , · · · , ak, and { a } .

Compiling policy neural networks. If there exists a regression rule selector that is computable with a finite-depth, finite-breadth RelNN circuit, it will be possible to construct a policy for the original problem with another RelNN circuit. This construction can be dramatically more e ffi cient than the general constructions in Section 4.2. Let k denote the 'width' of the regression rule selector, i.e., the maximum size of the constraints we need to keep track of when applying select .

Theorem 4.3 (Compilation of S-GRS with a regression rule selector) . Given a regression rule selector select (i.e., the state-constraint condition function ρ ) that can be computed by a relational neural network in RelNN [ Dr , Br ] , for any planning problem P, if tr select ( s 0 , g , ∅ ) , ⊥ , letting d be the depth of the regression tree, then there is a RelNN circuit for the problem in RelNN [ O ( d · Dr ) , max( Br , β )] , where β is the maximum arity of predicates.

Proof. For any step i , let pre ≤ i be the length i prefix of the precondition list. We define Goal d to be the single goal atom and set of constraints we need to satisfy at depth d . We have the following rules:

Here the evaluations of preconditions pre are based on the current state (input to the policy network) and they can be evaluated in parallel. We illustrate this construction in Fig. 3b. The first set of rules computes the next subgoal (and constraint set) to achieve, while the second set of rules outputs the next action when all preconditions of the current rule have been satisfied (which will be the output of the entire policy). Since at each layer d , we only need to keep track of one tuple of ( g , cons ), the breadth of the circuit is max( Br , β ), where β is required to store the input state. □

This construction yields a significant reduction in breadth (from tracking all conjunctions up to kBWD atoms to only one tuple of ( g , cons )), as well as a reduction of the number of layers for the RelNN policy, depending on the domain. In particular, if the average number of preconditions that need to be recursively solved in each rule is b (roughly equal to the number of preconditions in each rule), the depth of the regression tree is d = O (log b T ), where T is the planning horizon.

Finally, for some problems, even if we have a regression rule selector, to compute a plan, we may still need an unbounded depth of search steps (i.e., d depends on the number of objects in the domain). For example, in Blocks World, to achieve clear (A), the number of steps needed is proportional to the number of objects on top of A. There are two ways to build a relational neural network policy in this case. First, we can allow the network to have a variable number of iterations by applying the same layer recurrently several times. Second, there might be 'shortcuts' that we can learn. We discuss both solutions in Appendix B.3. These methods allow us to construct finite-depth RelNN circuits for a variety of domains, including Blocks World, and path-finding in any acyclic maze.

5 Problem Analysis and Results

We first analyze the regression width and circuit complexities for familiar AI planning problems. Table 1 summarizes the results on a simplified setting where the goal of the planning problem is a

Table 2: Success rate of learned policies in di ff erent environments. For Assembly3 (As3) and Logistics (Log.), we show the success rate. For Blocks World (BW), we show the success rate / average solution length. We choose f ( n ) def = n / 5 + 1 for Logistics and f ( n ) def = n / 10 + 3 for BlocksWorld. In the notation of RelNN [ D , B ], D is the number of layers and B is the maximum arity of edges.

single atom. In summary, most of these problems have a constant breadth (i.e., the regression width of the corresponding problem is constant) except for Sokoban. Most problems have an unbounded depth: that is, the depth of the circuit will depend on the number of objects (e.g., the size of the graph or the number of blocks). For problems in this list, when there are multiple goals, usually the goals are not serializable (in the optimal planning case). If we only care about satisficing plans, for Logistics, Gripper, Rover, and Elevator, there exists a simple serialization for any conjunction of goal atoms (basically achieving one goal at a time). See Appendix C for more detailed proofs and additional discussions of generalization to multiple goals and suboptimal plans.

Next, given this analytical understanding of the relationship between planning problems and their policy circuit complexity, we perform some simple experiments to see whether that relationship is borne out in practice. Relational neural networks have been demonstrated to be e ff ective in solving some planning problems [Dong et al., 2019, Jiang and Luo, 2019, Li et al., 2020], but their complexity has not been systematically explored. We consider two families of problems: one predicted to require finite depth and one predicted to require unbounded depth. For all tasks, we use Neural Logic Machines [Dong et al., 2019] as the model; we set the number of hidden neurons in the MLP layers to be su ffi ciently large (64 in all experiments) so that it is not the bottleneck for network expressiveness. We use the same encoding style for all problems (the graph-like relationship). Therefore, whether di ff erent problems have bounded or unbounded width primarily depends on the available goal regression rules, how these rules can be serialized, and the width of the rules. We show the average performance across 3 random seeds, with standard errors.

Assembly3: finite depth circuits. The domain Assembly3 contains n objects. Each object has a category, chosen from the set { A , B , C } . The goal of the task is to select three objects oA , oB , and oC sequentially, one for each category, while satisfying a matching constraint: match ( oA , oB ) and match ( oB , oC ). Therefore, to select the first object (e.g., an object of type A ), the policy needs to perform two layers of goal regression (first find the set of possible B -typed objects that match the object, and then find another C -typed object). We trained two models. Both models have breadth 2, but the first model has only 1 layer (theoretically not capable of representing the two-step goal regression), while the second model has 2 layers. Shown in Table 2, we train models on environments with 10 objects and test them on environments with 10, 30, and 50 objects. The first model is not able to learn a policy for the given task, while the second exhibits perfect generalization.

Logistics: unbounded depth circuits. We also construct a simple Logistics domain with only cities and trucks (no airplanes), in which the graphs are not strongly connected (by first sampling a directed tree and then adding forward-connecting edges). We train the policy network on problems with less than n = 10 cities and test it on n = 30 and n = 50. Here, we trained two policies. The first policy has a constant depth of 3, and the second policy has a depth of D = n / 5 + 1, which is a manually chosen function such that it is larger than the diameter of the graph. Shown in Table 2, the first model fails to generalize to larger graphs due to its limited circuit depth. Intuitively, to find a path in a graph by recursively applying the regression rule, we need a circuit of an adaptive depth (proportional to the length of the path). By contrast, the second model with adaptive depths generalizes perfectly.

BlocksWorld-Clear: unbounded depth circuits. Based on the goal regression analysis, in BlocksWorld, in order to achieve the goal atom clear (A) for a specific block A , we would need

a circuit that is as deep as the number of objects on A in the initial state. We follow the same training and evaluation setup as in Logistics. Our analysis focuses on the length of the generated plan. Shown in Table 2, although both policy networks accomplish the task with a 1.0 success rate (because there is a trivial policy that randomly drops clear blocks onto the table), the policy whose depth depends on the number of blocks successfully finds a plan that is significantly shorter than fixed-depth policy. This suggests the importance of using RelNNs with a non-constant depth for certain problems.

6 Related Work and Conclusion

Most of the existing work on planning complexity considers the NP-Completeness or PSPACECompleteness of particular problems, such as the traveling salesman problem [Karp, 1972], Sokoban [Culberson, 1997], and Blocks World [Gupta and Nau, 1992]. In general, the decision problem of plan existence is PSPACE-Complete for atomic STRIPS planning problems [Bylander, 1994]. Seminal work on fine-grained planning problem complexity introduces planning width [Chen and Giménez, 2007] and the IW algorithm [Lipovetzky and Ge ff ner, 2012, Drexler et al., 2022]. Our work is greatly inspired by these ideas and strives to connect them with circuit complexity.

The concept of serialized goal regression is not completely new. These rules are particularly related to the methods in hierarchical task networks [Erol et al., 1994] and can also be seen as special cases of hierarchical goal networks [Alford et al., 2016], derived by dropping preconditions [Sacerdoti, 1974]. In addition, others, including Korf [1987] and Barrett and Weld [1993], have characterized di ff erent degrees of serializability of subgoals, but their analysis is not as fine-grained as this.

Our circuit complexity analysis builds on existing work on relational neural network expressiveness, in particular, GNNs and their variants. Xu et al. [2019] provides an illuminating characterization of GNN expressiveness in terms of the WL graph isomorphism test. Azizian and Lelarge [2021] analyze the expressiveness of higher-order Folklore GNNs by connecting them with high-dimensional WL-tests. Barceló et al. [2020] and Luo et al. [2022] reviewed GNNs from the logical perspective and rigorously refined their logical expressiveness with respect to fragments of first-order logic. Our work extends their results, asking the question of what planning problems a RelNN can solve.

Our work is also related to how GNNs may generalize to larger graphs (in our case, planning problems with an arbitrary number of objects). Xu et al. [2020, 2021] have studied the notion of algorithmic alignment to quantify such structural generalization. Dong et al. [2019] provided empirical results showing that NLMs generalize in Blocks World and many other domains. Bu ff elli et al. [2022] introduced a regularization technique to improve GNNs' generalization and demonstrated its e ff ectiveness empirically. Xu et al. [2021] also showed empirically on certain algorithmic reasoning problems (e.g., max-Degree, shortest path, and the n-body problem). In this paper, we focus on constructing policies that generalize instead of algorithms, using a similar idea of compiling specific (search) algorithms into neural networks.

Conclusion. To summarize, we have illustrated a connection between classical planning width, regression width, search complexity, and policy circuit complexity. We derive upper bounds for search and circuit complexities as a function of the regression width of problems and planning horizon. These results provide an explanation of the success of relational neural networks learning generalizable policies for many object-centric domains. The compilation algorithms highlight that when there are resource constraints such as free space, agents' inventory size, etc., a small policy circuit may not exist. This includes cases such as Sokoban and general task and motion planning. Furthermore, our idea of serialization can be generalized to hierarchical decomposition of problems; it can be potentially extended to other domains such as optimization and planning under uncertainty.

Although all of the analyses in this paper have been done for fully discrete domains by analyzing first-order logic formulas, understanding the implications in relational neural network learning is important because, ultimately, RelNNs can be integrated with perception and continuous action policy networks to solve realistic robotic manipulation problems (e.g., physical Blocks World [Li et al., 2020]). It is challenging to directly generalize our proof to scenarios where states and actions are continuous values because in these cases, there will be an infinite number of possible actions (e.g., the choice of grasping poses, trajectories, etc.). We believe that it is an interesting question to investigate similar 'width'-like definitions in sampling and discretization-based continuous planning algorithms. So far, we have only analyzed the circuit complexity of policies for individual planning problems; an interesting future direction is to extend these analyses to the learning and inference of 'general' policies such as large language models that can generate plans for many di ff erent planning domains.

Acknowledgement. We thank anonymous reviewers for their valuable comments. We gratefully acknowledge support from ONR MURI grant N00014-16-1-2007; from the Center for Brain, Minds, and Machines (CBMM, funded by NSF STC award CCF-1231216); from NSF grant 2214177; from Air Force O ffi ce of Scientific Research (AFOSR) grant FA9550-22-1-0249; from ONR MURI grant N00014-22-1-2740; from ARO grant W911NF-23-1-0034; from the MIT-IBM Watson AI Lab; from the MIT Quest for Intelligence; and from the Boston Dynamics Artificial Intelligence Institute. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the authors and do not necessarily reflect the views of our sponsors.

References

Ron Alford, Vikas Shivashankar, Mark Roberts, Jeremy Frank, and David W Aha. Hierarchical Planning: Relating Task and Goal Decomposition with Task Sharing. In IJCAI , 2016. 10

Waiss Azizian and Marc Lelarge. Expressive Power of Invariant and Equivariant Graph Neural Networks. In ICLR , 2021. 10

Pablo Barceló, Egor V Kostylev, Mikael Monet, Jorge Pérez, Juan Reutter, and Juan Pablo Silva. The logical expressiveness of graph neural networks. In ICLR , 2020. 6, 10

Anthony Barrett and Daniel S Weld. Characterizing Subgoal Interactions for Planning. In IJCAI , 1993. 10

Davide Bu ff elli, Pietro Liò, and Fabio Vandin. SizeShiftReg: a Regularization Method for Improving SizeGeneralization in Graph Neural Networks. In NeurIPS , 2022. 10

Tom Bylander. The Computational Complexity of Propositional STRIPS Planning. Artif. Intell. , 69(1-2): 165-204, 1994. 1, 10

Jin-Yi Cai, Martin Fürer, and Neil Immerman. An Optimal Lower Bound on the Number of Variables for Graph Identification. Combinatorica , 12(4):389-410, 1992. 6

Thomas Carta, Clément Romac, Thomas Wolf, Sylvain Lamprier, Olivier Sigaud, and Pierre-Yves Oudeyer. Grounding Large Language Models in Interactive Environments with Online Reinforcement Learning. arXiv:2302.02662 , 2023. 1

Hubie Chen and Omer Giménez. Act Local, Think Global: Width Notions for Tractable Planning. In ICAPS , 2007. 1, 4, 5, 10, 15

Joseph Culberson. Sokoban is PSPACE-complete. Technical report, University of Alberta, 1997. 2, 10

Honghua Dong, Jiayuan Mao, Tian Lin, Chong Wang, Lihong Li, and Denny Zhou. Neural Logic Machines. In ICLR , 2019. 1, 9, 10, 21

Dominik Drexler, Jendrik Seipp, and Hector Ge ff ner. Learning Sketches for Decomposing Planning Problems into Subproblems of Bounded Width. In ICAPS , 2022. 10

Kutluhan Erol, James Hendler, and Dana S Nau. HTN planning: Complexity and Expressivity. In AAAI , 1994. 10

Richard E Fikes and Nils J Nilsson. STRIPS: A New Approach to the Application of Theorem Proving to Problem Solving. Artif. Intell. , 2(3-4):189-208, 1971. 2

Eugene C Freuder. A Su ffi cient Condition for Backtrack-Free Search. JACM , 29(1):24-32, 1982. 15

Naresh Gupta and Dana S Nau. On the Complexity of Blocks-World Planning. Artif. Intell. , 56(2-3):223-254, 1992. 10

Zhengyao Jiang and Shan Luo. Neural Logic Reinforcement Learning. In ICML , 2019. 9

Richard M Karp. Reducibility among Combinatorial Problems . Springer, 1972. 10

Richard E Korf. Planning as Search: A Quantitative Approach. Artif. Intell. , 33(1):65-88, 1987. 10

Richard Li, Allan Jabri, Trevor Darrell, and Pulkit Agrawal. Towards Practical Multi-Object Manipulation Using Relational Reinforcement Learning. In ICRA , 2020. 1, 9, 10

Jacky Liang, Wenlong Huang, Fei Xia, Peng Xu, Karol Hausman, Brian Ichter, Pete Florence, and Andy Zeng. Code as Policies: Language Model Programs for Embodied Control. arXiv preprint arXiv:2209.07753 , 2022. 1

Vladimir Lifschitz. On the Semantics of STRIPS. In Reasoning about Actions and Plans . Morgan Kaufmann, 1987. 2

Nir Lipovetzky and Hector Ge ff ner. Width and Serialization of Classical Planning Problems. In ECAI 2012 . IOS Press, 2012. 5, 10, 15

Zhezheng Luo, Jiayuan Mao, Joshua B Tenenbaum, and Leslie Pack Kaelbling. On the Expressiveness and Generalization of Hypergraph Neural Networks. In LoG , 2022. 6, 10

Christopher Morris, Martin Ritzert, Matthias Fey, William L Hamilton, Jan Eric Lenssen, Gaurav Rattan, and Martin Grohe. Weisfeiler and Leman Go Neural: Higher-Order Graph Neural Networks. In AAAI , 2019. 5, 6

Earl D Sacerdoti. Planning in a Hierarchy of Abstraction Spaces. Artif. Intell. , 5(2):115-135, 1974. 10

Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. Attention is All You Need. In NeurIPS , 2017. 1, 5

William Vega-Brown and Nicholas Roy. Task and Motion Planning is PSPACE-Complete. In AAAI , 2020. 2

Tingwu Wang, Renjie Liao, Jimmy Ba, and Sanja Fidler. Nervenet: Learning Structured Policy with Graph Neural Networks. In ICLR , 2018. 1

Keyulu Xu, Weihua Hu, Jure Leskovec, and Stefanie Jegelka. How Powerful are Graph Neural Networks? In ICLR , 2019. 10

Keyulu Xu, Jingling Li, Mozhi Zhang, Simon S Du, Ken-ichi Kawarabayashi, and Stefanie Jegelka. What can Neural Networks Reason About? In ICLR , 2020. 10

Keyulu Xu, Mozhi Zhang, Jingling Li, Simon S Du, Ken-ichi Kawarabayashi, and Stefanie Jegelka. How Neural Networks Extrapolate: From Feedforward to Graph Neural Networks. In ICLR , 2021. 10

Appendix

The appendix of the paper is organized as the following. In Appendix A, we provide the proof details and more discussion with related work on search complexity. In Appendix B, we provide the proof details and more discussion on policy circuit complexity. Next, in Appendix C, we include a detailed analysis of regression widths and circuit complexities for empirical planning problems. Finally, in Appendix D, we describe environmental setups and implementation details for the experiments.

A Proofs and Discussions for Search Complexity

A.1 Complete and Optimal Version of S-GRS

Algorithm 3 shows a complete and optimal version of the serialized goal regression search algorithm. The key di ff erence between this variant and the simplified one presented in the paper is that now we keep track of multiple possible trajectories that can achieve the specified goal. The algorithm is complete because for any set of precondition atoms, in the optimal trajectory, there will always be an order in which they have been achieved (note that some of the sub-trajectories can be empty, indicating that while planning for the prefix the next subgoal has already been achieved; this covers the 'parallel' precondition achievement case.). We show an example of this branching factor in Figure 4. Note that, to make sure that the algorithm eventually terminates, we need to make sure that all trajectories do not have loops.

However, this bare algorithm can be very slow: the worst time complexity of it is O (3 N ) with respect to N , the number of atoms in P 0 -in contrast to O (2 N ) for a simpler backward search algorithm ∥ . This is because the possible state set for di ff erent subgoals and di ff erent constraints may overlap, so it will 'waste' time during the search. Therefore, this algorithm may not be of practical use.

A.2 Proof for Theorem 3.1: S-GRS Optimality and Completeness

We define the set of optimally serializable rules for a state s as OSR ( s ), and the set of single-literal goals that can be solved with OSR ( · ) from s 0 as OSG ( s 0 ).

Proof. Completeness: the definition of OSG ( s 0) asserts that g can be solved by S-GRS.

Optimality: proof by induction. In the following, we prove that for any g ∈ OSG ( s 0), the trajectory returned by S-GRS is optimal.

Subgoal:

handsfree

Constraints:

on(C, B), clear(C)

(a) Current State

(b) 6 Different States that Satisfies the Subgoal and Constraints

D

L

D

A

E

A

B

A

B

E

D

A

B

E

A

B

A

B

B

E

A

B

D

First, we consider the set of all S-GRS calls in the regression tree that contributes to the returned trajectory. We do induction based on a distance metric. Consider any goal atoms g ∈ OSG ( s 0 ), let Distance ( s 0 , g , cons ) be the length of the path returned by S-GRS. First, for all argument tuples with distance 0 (i.e., g ∈ s 0 and cons ⊆ s 0), S-GRS will return the optimal trajectory.

Next, assuming that S-GRS will return optimal trajectories for all argument tuples with distances strictly smaller than d . Consider a new argument call grs ( s 0 , goal , cons ) with distance d . Consider now all optimally serializable and goal-achieving rules r ∈ OSR ( s 0) such that goal ( r ) = g and cons ∩ e ff -( action ( r )) = ∅ . Let rg be the rule that returns the plan during search, and p 1 · · · pk be the serialized precondition list. First, all sub-function calls to S-GRS will return optimal plans by induction. Second, by optimal serializability of the rule r , we know that concatenating the sub-plans will yield an optimal plan to achieve g while preserving g . Therefore, our induction holds. □

A.3 Making Non-SOS Problem Serializable

Here we discuss a general strategy to make non-SOS problems serializable by introducing super predicates. In particular, for a given operator, if two of the preconditions (without loss of generality, say, p ( x ) and q ( x )) are not serializable, we can introduce a new predicate p_and_q ( x ), and rewrite the precondition of the operator with this new super predicate.

The disadvantage of this encoding is that it will enlarge not only the predicate set but also the operator set. In particular, for any operator o such that p ( x ) ∈ e ff + ( o ) or p ( x ) ∈ e ff -( o ), we need to break the operator into two sub-operators. For example, if p ( x ) ∈ e ff + ( o ), we need to create a new operator o 1 that has ¬ q ( x ) in its precondition, and set e ff + ( o 1 ) = e ff + ( o ) and e ff -( o 1 ) = e ff + ( o ). For the second operator o 2, it should have q ( x ) in its precondition and e ff + ( o 1 ) = e ff + ( o ) ∪ { p_and_q ( x ) } and e ff -( o 1 ) = e ff -( o ). As the number of predicates in this super predicate increases, the number of additional operators grows exponentially.

This provides a characterization of 'what contributes to the (forward) width of a problem.' Specifically, we have identified two sources of forward widths: non-serializability of certain operators, and the need to track constraints while doing a regression search.

A.4 Proof for Theorem 3.2: Polynomial Hardness for SOS-k problems.

If a problem P is of SOS width k , it can be solved in time O ( N k + 1 ), where N is the number of atoms. Here we have omitted polynomials related to enumerating all possible actions and their serializations (which are polynomial w.r.t. the number of objects in U ).

Proof. First, if we are given the rule set R , then the proof can be simply done by analyzing function calls: it is important to observe that there are at most O ( N k + 1 ) possible argument combinations of ( goal , cons ) to function grs .

First, due to SOS condition, we only need to maintain one single optimal trajectory for each set of argument values ( s 0 , goal , cons ). Next, we consider we only maintain one single optimal trajectory for each pair of ( goal , cons ), ignoring the first argument s 0. Note that, even if the 'cached' trajectory does not include the input state s 0 we can still return the cached trajectory. This is because the optimal serializability suggests that any trajectories for achieving ( goal , cons ) can be extended to future preconditions.

Finally, we consider removing the assumption of knowing R . Note that the enumeration of all possible goal widthk regression rules can be done in O (2 k ), which is considered constant when k is small.

Therefore, overall the algorithm runs in polynomial time and the solver does not need to know the set of optimally serializable regression rules R a priori. □

First, the fact that the algorithm does not need to know R a priori is important because it suggests that, as long as there exists a set of optimally serializable rules for the problem we are trying to solve, the search algorithm will find a solution, without having any additional knowledge other than the operator definitions. Intuitively, this is because doing S-GRS with additional non-optimally serializable rules (e.g., using all rules in R 0) will not falsify the completeness and optimality of the algorithm.

Second, unfortunately, the SOS condition can not be relaxed. That is, if we only assume optimal serializability of generalized regression rules and all rules have width k , the proof will fail. This is because even if regression rules have a small width, there will be a possibly exponential branching factor caused by 'free variables' in regression rules that do not appear in the goal atom. Concretely, let us consider the following rule g ( x ) f p 1 ( x , y ) ∅ , p 2 ( x ) ∅ ∥ a 1 ( x , y ). This rule is of width 0; however, since there is no guarantee that any optimal trajectory for achieving p 2 ( x ) will also achieve p 1 ( x , y ) ∧ p 2 ( x ), during search, we can not use the cached trajectory for ( p 2 ( x ) , ∅ ) as the return value when we solve for p 2 ( x ) after achieving p 1 ( x , y ), which breaks the complexity analysis.

A.5 Additional Discussion

Deriving optimally-serializable, small-width rules. For a practical system, if we can know the set of regression rules needed, we can accelerate the search significantly. Here we discuss a strategy for deriving such rules by analyzing the domain definition itself.

If we ignore the optimality, a su ffi cient condition that a rule r is not strong optimal serializability for a given state s 0 is that, there exists a precondition pi such that there is no rule r ′ such that goal ( r ′ ) = pi , e ff -( action ( r ′ )) ∩ pi = ∅ and goal ( r ) < subgoals ( r ′ ). The last condition is the goal-stack checking: if achieving a goal g using rule r requires achieving goal g itself, then the rule is not serializable.

Putting this into practice, consider all rules for achieving clear (A). If there is another object B on A in s 0, then only the rule derived from unstack (B , A) will be SOS serializable. This is because, for any other rules unstack ( y , A), where y , B , achieving on (y , A) will involve achieving clear (A) itself. Note that this analysis can be done by static (lifted) analysis of the domain.

This analysis can be viewed as simulating a one-step goal regression search in a lifted way, and it can be extended to more steps. In some domains, this strategy will help rule out some serialization of the preconditions (such as the BlocksWorld example), but in general, whether a rule is strong optimally serializable is state-dependent and can be hard to compute.

Connection with other concepts in search. Another view of the regression width is the maximum number of constraints one needs to track while applying S-GRS recursively. This notion is analogous to the treewidth of a constraint graph in constraint satisfiability problems (CSPs): intuitively, the number of backward-pointing edges [Freuder, 1982].

As also briefly discussed in Lipovetzky and Ge ff ner [2012], delete-relaxation heuristics are also related. In particular, the hFF heuristic can be interpreted as applying backward search by considering 'parallelizability' in contrast to 'serializability.' Informally, for a given state s and a sequence of preconditions pi , we compute trajectories for each individual pi separately and concatenate them.

Generalization to ∀ -quantified preconditions. It would be possible to extend the definition to ∀ -quantified preconditions by simply allowing regression rules to have an unbounded number of subgoals (e.g., to handle goals such as ∀ xon-table ( x )). However, this is not helpful in general because the constraints will accumulate as the number of subgoals becomes larger, and enumerating subgoal serialization will now take O ( m !) where m is the number of subgoals. This notion will be more helpful when we have a 'goal regression rule selector.'

A.6 Connection with Width-Based Forward Search IW(k)

Wecall the width of a problem defined in Chen and Giménez [2007] and the Iterated Width algorithm IW ( k ) [Lipovetzky and Ge ff ner, 2012] as forward width. Then we have:

Theorem A.1 (Su ffi ciency condition for forward width) . Any planning problem that has SOS-width k has a forward width of at most k + 1 and, hence, can be solved by the algorithm IW ( k + 1) .

Proof. We prove that for any trajectory returned by the S-GRS algorithm for an SOS widthk problem P , all actions in the trajectory achieve a novel k + 1-sized atom set. If this is true, then the final goal will be reachable from s 0 in the size k + 1 tuple graph in IW search.

Consider any action a in the returned plan, it corresponds to a unique rule for achieving a subgoal g and a set of constraints cons . If the size k + 1 tuple { g } ∪ cons has already been achieved before a , then due to the SOS condition, we can make the returned plan shorter. This violates the optimality of the S-GRS algorithm. □

Unfortunately, these two definitions are not exactly equivalent to each other. For example, consider the Blocks World domain and a state of three stacked blocks A, B and C, the goal regression rule

is SOS. It has width 1 because while achieving handsfree (), we have to explicitly maintain clear (B); otherwise, it is possible that we directly put C back onto B instead of putting C onto the table (both plans are optimal for achieving handsfree ()) after achieving clear (B). By contrast, in IW, the Blocks World domain has width 1 instead of 2 if the goal predicate is clear , because the optimal plan for achieving handsfree () { clear (B) } happens to be the optimal plan for achieving on-table (C).

Proofs for (forward) widths of planning problems in IW were mostly done by analyzing solution structures. By contrast, the constructions in this paper form a new perspective to interpret planning problem width: they are subgoals and constraints that need to be satisfied during a goal regression search. This view is helpful because our analyses focus on analyzing concrete goal regression rules, and the derived results are compositional. Unfortunately, not all problems that can be solved by IW ( k ) have a regression width of k -1.

B Proofs and Discussions for Policy Circuit Complexity

B.1 Proof of Theorem 4.1: Compilation of BWD

Given a planning problem P , let T be the length of the optimal trajectory as (the planning horizon), k BWD be the maximum number of atoms in the goal set in bwd , and β be the maximum arity of atoms in the domain. The backward search algorithm bwd can be compiled into a relational neural network in RelNN [ O ( T ) , β · k BWD].

State: on, clear

Goal

0

{on(B, A)}

Residual

(Until

!

∧ 𝑠𝑔 ⊆ 𝑠

)

State: on, clear, ...

Goal

1

{clear(A)

∧

holding(B)}

RelNN

Layer

......

Next Action Prob.

unstack(D, C)

RelNN

Layer

Goal

i

is a

set of conjunctive formula

.

Proof. There is a proof by construction. As illustrated in Fig. 5, a relational neural network can use its intermediate representations to keep track of a set of subgoals (i.e., a set of sets of ground atoms) Goal d at each layer d . Initially, Goal 0 contains only the goal atom. Then, sg ∈ Goal d if there is a path of length d from any state s that satisfies sg ⊆ s to the final goal of the planning problem. For all layers d , for any subgoals sg , for all ground actions a ∈ A , sg ∈ Goal d ∧ GLYPH<0> e ff -( a ) ∩ sg = ∅ GLYPH<1> ⇒ GLYPH<0> sg ∪ pre ( a ) \ e ff + ( a ) GLYPH<1> ∈ Goal d + 1 . Since the maximum number of atoms in bwd is k BWD, we only need to keep track of ground atom conjunctions that involve at most k BWD · β distinct objects. Therefore, bwd search can be compiled into a relational neural network in RelNN [ O ( T ) , k BWD · β ].

A more detailed construction is the following. Let k ′ = k BWD · β . Let x 1 , · · · , xk ′ be variables. { qi ( x 1 , · · · , xk ′ ) } is the set all possible conjunctive expressions that involve variables { x 1 , · · · , xk ′ } . Let q ( d ) i ( x 1 , · · · , xk ′ ) be the value of qi ( x 1 , · · · , xk ′ ) at layer d , and expr ( qi ) be the conjunctive expression of qi . For all d , qi , qj , and for all actions a that are grounded on variables from { x 1 , · · · , xk ′ }, if ( e ff -( a ) ∩ expr ( qi ) = ∅ ) ∧ ( expr ( qj ) = expr ( qi ) ∪ pre ( a ) \ e ff + ( a )), there is a logical rule: ∀ x 1 , · · · , xk ′ . q ( d ) i ( x 1 , · · · , xk ′ ) ⇒ q ( d + 1) j ( x 1 , · · · , xk ′ ). Furthermore, for all d , qi , if ( e ff -( a ) ∩ expr ( qi ) = ∅ ) ∧ ( expr ( qi ) ∪ pre ( a ) \ e ff + ( a ) ⊂ s ), where s is the current state, we have the following rule that outputs the next action to take, ∀ x 1 , · · · , xk ′ . q ( d ) i ( x 1 , · · · , xk ′ ) ⇒ a ( d + 1) , where a ( d ) = 1 indicates that there exists a length d trajectory for achieving the goal. In practice, we will the shortest one. Note that all these rules are fully quantified by ∀ x 1 , · · · , xk ′ so they are lifted FOL rules and can be implemented by a RelNN circuit. □

B.2 Proof of Theorem 4.2: Compilation of S-GRS

Given a planning problem P of SOS width k , let T be the length of the optimal trajectory, and β be the maximum arity of atoms in the domain. The serialized goal regression search S-GRS can be compiled into a RelNN [ O ( T ) , ( k + 1) · β ], where k + 1 ≤ k BWD.

Proof. There is a proof by construction. The easiest construction is based on the su ffi cient condition between regression width k and IW ( k ). In particular, based on Theorem 3.3, we know that IW ( k + 1) can solve the planning problem P . Therefore, we can use the following construction to simulate IW ( k + 1).

The most important trick in the construction is that a relational neural network can use its intermediate representations to keep track of two sets of atoms. First, for each atom tuple t up to size k + 1, whether t is reachable from s 0 within d steps. Second, let st be the last state of the optimal trajectory associated with t ; for each atom tuple t up to size k + 1 and for a predicate p ( x 1 , x 2 , · · · , x β ), we

keep track of whether p ( x 1 , x 2 , · · · , x β ) is true in st . At each layer d , we consider all possible ground actions a ∈ A , if there exists such ( t 1 , t 2) such that t 2 is not reachable within d steps and t ⊆ T ( st 1 , a ), we set t 2 to be reachable within d + 1 and st 2 = T ( st 1 , a ).

In addition to the realizability of the IW ( k + 1) algorithm, we need to additionally prove that, if we follow the policy derived by simulating IW ( k + 1) for the first step, we can recurrently apply this policy. To see that, consider the path from s 0 to the goal state in IW ( k + 1) in the tuple graph. Since we are moving forward by one step in this tuple path, doing a search from the second tuple in the path (i.e., applying the same RelNN policy at the second world state) will yield the same trajectory.

Similar to the construction in Theorem 4.1, all the rules are lifted FOL rules and can be implemented by a RelNN circuit. It is also possible to use the same trick to compile the original S-GRS algorithm: essentially, for each ( g , cons ) pair, keep track of the optimal trajectory for achieving it. However, the construction will be more complicated and is omitted here. □

B.3 More Discussion

For some problems, even if we have a regression rule selector, to compute a plan, we may still need an unbounded depth of search steps (i.e., d depends on the number of objects in the domain). For example, in Blocks World, to achieve clear (A), the number of steps needed is proportional to the number of objects on top of A. There are two ways to build a relational neural network policy in this case. First, we can allow the network to have a variable number of iterations by applying the same layer recurrently several times. We have already illustrated this in the experiment.

Second, there might be 'shortcuts' that we can learn. An intuitive example is that in BlocksWorld, the goal regression for clear (A) will recur until we have found the topmost block stacked on top of A. If the domain has a new predicate above ( x , y ) indicating whether the block x is above another block y , then we can use just one single rule to predict the topmost block on top of A: the topmost block B on A satisfies above (B , A) ∧ clear (B).

The same technique also applies to the problem of path-finding in any acyclic maze. Recall that a classical solution to any acyclic maze is that we only make right turns. This can be cast as a linearization of the entire maze by doing a depth-first search from the start or the goal location. In particular, starting from the goal location if we only do right turns, we will get a sequence of the edges of the maze such that the start edge is one of the elements (if there is a path from the start to the goal). Therefore, doing goal regression under this linearized maze is trivial: the entire 'regression tree' is just a chain of edges and there is a simple shortcut for predicting the next move.

C Analysis of AI Planning Problems

Here, we list six popular problems that have been widely used in AI planning communities and discuss their circuit depth and breadth. We summarize our results in the Table 3.

Note that here, we are limiting our discussion to the case where the goal of the planning problem is a single atom. In summary, most of these problems have a constant breadth (i.e., the regression width of the corresponding problem is constant) except for Sokoban. Most problems have an unbounded depth: that is, the depth of the circuit will depend on the number of objects (e.g., the size of the graph or the number of blocks. The problem gripper has a constant depth under a single-atom goal because it assumes the agent can move directly between any two rooms; therefore, no pathfinding is required. Sokoban has an unbounded breadth even for single-atom goals because if there are multiple boxes blocking the way to a designated position, the order to move the boxes can not be determined by simple rules. The constant breadth results can be proved by construction: list all regression rules in the domain. In the following Appendix C.1, we first discuss the SOS width of two representative problems, Blocks World and Logistics. All other problems can be analyzed in a similar way.

Before we delve into concrete complexity proofs, we would like to add the note that for problems in this list, when there are multiple goals, usually the goals are not serializable (in the optimal planning case). This can be possibly addressed by introducing 'super predicates' that combine two literals. For example, if two goal atoms p ( x ) and q ( x ) are not serializable, we can introduce a new predicate p_and_q ( x ), and rewrite all operators with this new super predicate. Appendix A.3 provides more discussion and examples. This will make the problem serializable but at the cost of significantly enlarging the set of predicates (exponentially with respect to the number of goal atoms).

If we only care about satisficing plans, for Logistics, Gripper, Rover, and Elevator, there exists a simple serialization for any conjunction of goal atoms (basically achieving one goal at a time).

These theoretical analyses can be used in determining the size of the policy circuit needed for each problem. For example, one should use a RelNN of breadth ( k + 1) · β , where k is the regression width and β is the max arity of predicates. For problems have unbounded depth, usually the depth of the circuit grows in O ( N ), where N is the number of objects in the environment (e.g., in Elevator, the number of floors).

C.1 SOS Width Analysis

To prove that a problem can be solved optimally using S-GRS, we only need to show that there exists a goal regression rule selector. Therefore, we will be using the syntax from Section 4 to write down the set of goal regression rule selectors.

Finite SOS width for Blocks World.

Finite SOS width for Logistics.

Here, we have omitted static properties for type checking (such as a 1 and a 2 need to be airports).

D Experimental Details

D.1 Training details

Environment setup. For the Assembly3 task, we first uniformly sample the set of type-A, type-B, and type-C object. Then, we randomly add match relationships so that there will be exactly only one tuple of ( a , b , c ) that satisfies the matching condition.

For the BlocksWorld-Clear task, we use the same random sampler for initial configurations as Neural Logic Machines [Dong et al., 2019]. It iteratively adds blocks to an empty initial state. At each step, it randomly selects a clear object (or the table), and places a new object on top of it. After generating the initial state, we randomly sample an object that is not clear in the initial state as the target block.

For the Logistics task, we first randomly generate a tree rooted at the start node. This is done by iteratively attaching new nodes to a leaf node. Next, we randomly add a small number of edges to the tree. Finally, we randomly pick a node as the target node. Since we are only a small number of additional edges added to the tree, with high probability, the starting node and the end node are not in the same strongly connected component.

Additional hyperparameters. The breadth, width, and hidden dimension parameters for neural networks have been specified in the experiment section and they vary across tasks. For all experiments, we use the Adam optimizer to train the neural network, with a learning rate of ϵ = 0 . 001. Additionally, we set β 1 = 0 . 9 and β 2 = 0 . 999.

Compute. We trained with 1 NVIDIA Titan RTX per experiment for all datasets, from an internal cluster.

Code. We release the code for reproducing all our experimental results in https: // github.com / concepts-ai / goal-regression-width.

Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper focuses on formalizing what class of planning problems can be solved by a relational neural networks. It aims to bridge the gap between the expressivity of relational neural networks and the complexity of planning problems. To do this, a serialized goal regression rule and search algorithm is defined. Three classes of planning problems are defined based on the serialized goal regression search and show these classes can provide network parameters for relational neural network. Experiments with two planning domains show the empirical implications of the results.\n\nSTRENGTHS:\n* The paper provides a novel approach to quantifying the complexity of planning problems\n* As far as I am aware, this is the first paper to analyze what kind of planning problems can be solved by neural networks.\n* The formal analysis can aid in better evaluations of the neural approaches proposed for planning problems.\n\nWEAKNESSES:\n* Some of the notations are not clear (See questions for details).\n\nQUESTIONS:\n**Major**\n\n\n1. Algorithm 1 \n    1. Plain backward search should satisfy two conditions for selection of action: $g \\cup eff_{+}(a) \\neq  \\emptyset$ and $g \\cup eff_{-}(a) = \\emptyset$. The second condition is captured in the Algorithm. But the first is not.\n   1. Action should be mentioned as input in Algorithm 1. \n   1. `goal stack` is not defined\n1. Algorithm 2\n    1.  $R_o$ should be part of the input\n    1.  `goal stack` is not defined\n    1.  `goal_set` is not defined\n    1.  Is $\\pi$ a plan? If so, $\\pi[-1]$ is the last action. So is $s_i$ an intermittent state or action? \n    1.  If any of the $p_i$ is available in the goal stack, then that branch is not explored in the backward search? What is the benefit of doing that?\n1. Line 107 mentions that there will be a goal regression rule for any possible permutation of pre(a), but as explained in Line 100 all of these are not feasible. Will the regression set contain infeasible rules as well?\n1. How are the generalized goal regression rules identified? For example, consider the rule in Line 139. Can such a rule be identified automatically? \n1. Were the serialized goal regression rules used in defining the relational neural network? Can explicit knowledge of such rules enable efficient learning?\n1. Which relational network was used in the experiments? NLM, ReNN, or NLRL? \n1. This is not a question. Just a discussion point. The traditional relational learning approaches learn rules one predicate at a time. The serialized goal-regression defines a class of planning problems that can be solved by satisfying one predicate at a time. So can this aid in understanding what class of planning problems can be solved by traditional relational learning approaches?\n1. Can authors provide the classes for common IPC domains like Logistics, Gripper, Rover, Elevator, etc?\n\n**Minor**\n\n9. Regression rule set notation has discrepancies. In Algorithm 2, $R_0$ is used, whereas in Line 109 $R^0$ is defined.\n9. There is a discrepancy in the function name. Line 111 uses S-GRS but Algorithm 2 uses GRS. \n9. Is $P_0$ defined in Line 61 the same as $P_{s_0}$? The set of predicates in the initial state does not define the set of all atoms.\n9. In Definition 3.1, is $\\bar{a}$ a trajectory or action? For consistency, I recommend using the symbol $a$ for actions alone. \n9. The term 'generalized goal regression rule' is not formally introduced. In line 141 such a rule is referred to as the reduced goal regression rule. For consistency, I recommend sticking to a single term.\n9. I know the space is limited, but I would still recommend moving the connection to plan width from appendix to main paper.", "TITLE": "Official Review", "RECOMMENDATION": 7, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-06-30"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper explores the application of relational neural networks, seen as circuits, in representing policies for discrete planning problems. Specifically, it introduces a circuit complexity analysis that categorizes planning problems into three classes based on how circuit width and depth grow, by establishing connections with serialized goal regression search (S-GRS), which is a method for generating plans by working backwards from goals. In essence, the main contribution of the paper is that it combines previously introduced planning problem complexity related quantifications (width, depth) with circuit/policy complexity analysis to analyze the resource/representation requirements of policies. This analysis helps to better understand the capabilities of relational neural networks in solving planning problems, which in turn helps designing them for policy learning. By analyzing goal-conditioned policies for such planning problems, upper bounds on the circuit complexity of such policies are provided as a function of the problem's regression width. The experimental results support the theoretical formulations and demonstrate the practical applicability of the proposed approach for relatively small sized/simpler discrete planning problems.\n\nSTRENGTHS:\n- Originality: The paper proposes a new perspective for circuit complexity analysis for relational neural networks (GNNs, transformers), and thus what they can compute (i.e., their limits). \n\n- Quality: The proposed method is supported by theoretical analysis and proofs. A small set of experiments were conducted to validate the approach.\n\n- Clarity: The order of presentation is mostly nice, even though the terminology used throughout the paper sometimes makes it hard to follow. \n\n- Significance: Helpful for constructing relational neural networks for planning problems, and the proposed formulation can also help investigate solutions to planning problems for future works in this area. If the problems analyzed had a broader scope, it would have increased the overall impact of the paper.\n\nWEAKNESSES:\n- Lack of a convincing set of quantitative results: Proposed methods might be tested on different networks and problems (only two different problems and networks are presented). \n\n- A broader discussion of the applicability of the proposed analysis and formulation on different planning problems, especially to real-world (e.g., robotics) scenarios, would help convince the reader about the impact of the paper. \n\n- Related work section might be placed earlier (e.g., after introduction) for better placing this work w.r.t. the literature. \n\n- Supplementary material helps clarify some points, due to page limit it is hard to integrate this information but for clarity there might be more reference to it within the main text.\n\nQUESTIONS:\n- What are the main bottlenecks/issues to apply this formulation for continuous planning problems, e.g., such as the ones in robotics?\n\n- Would this analysis and construction be still useful/practical for larger/more complex problems, which require larger networks? If yes, how? \n\n- The problems investigated are solvable by classical search/planning methods, then what’s the advantage of using a RelNN that was constructed based on the provided analysis?\n\n- Serialization of goal regression rules seem to be highly related to mathematical optimization / dynamical programming. Can you elaborate on this connection?\n\nLIMITATIONS:\n- Limitations of the work have been barely touched upon on the paper. \n\n- The discussion on applying this formulation to continuous planning problems, e.g., in robotics, can be extended. \n\n- In general, an evaluation and discussion on the type of problems that the proposed formulation is not applicable to would be nice.", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-04"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper discusses the expressivity of classical relational architectures and its implication for solving planning problems. In particular, the authors identify characteristics of the environments that make planning hard. They prove a theorem that bounds the size of a network sufficient to solve the problem of planning perfectly, under some assumptions on the underlying domain. Finally, they show experimental results in two simple environments.\n\nSTRENGTHS:\nThe paper is very clear and easy to follow. It's very convenient that you provide intuitive explanations of the derived theorems and explain them in an example environment. The motivation is clearly stated in the introduction. The problem is interesting and deserves a study. The presented results are novel and provide a deeper understanding of the capabilities of widely used architectures.\n\nWEAKNESSES:\nIn my opinion, the greatest weakness of this paper is an insufficient discussion of the connection between the formulated theory and practical applications. I understand that the contribution of the work is mostly theoretical. Anyway, it would be nice to show how the presented theory should influence my design choices. For instance, I suggest providing a list of environments with a short explanation of their properties (finite/unbounded depth, serializability, width, theoretical bounds values, implications, etc.), even without experiments, even in the appendix.\n\nQUESTIONS:\nl.99 It's very convenient that you explain the discussed properties with an example. However, it would be even better if you briefly explained the meaning of actions. Although after reading I know what _clear_(A) means, explaining it at the beginning would be helpful. Also, generally, there is no precise description (even brief) of the BlocksWorld environment in the paper.\n\nl.114 Please refer to a specific appendix.\n\nl.124 I'm not sure if the provided intuition is correct. As far as I understand, it should be \"A rule is optimally serializable if any optimal trajectory extended with an optimal continuation forms an optimal plan.\" That is, the intuition described in the paper suggests just an existence of $a_2$. Is that correct?\n\nl.130 Does Theorem 3.1 have proof? Please, provide a reference.\n\nl. 317 It would be nice to show how the presented theory should influence design choices when using relational networks. For instance, I suggest providing a list of environments with a short explanation of their properties (finite/unbounded depth, serializability, width, theoretical bounds values, implications, etc.), even without experiments, even in the appendix. The discussion provided in l.305-317 is a good starting point. The discussion in the appendix (again, no reference) is also nice, but I believe it can be extended. Notably, with more examples. This way the theory you provide could be much more impactful.\n\nl.324 I like that you provide experiments both for theoretically bounded and unbounded depth. However, it's a shame that it turns out that unbounded depth is actually no bottleneck. In general, it's good to show such an environment, but here I'd like to see an experiment in an environment with truly unbounded depth and witness the growing success rate up to a practical bound. Because BlocksWorld actually requires little complexity, there is no experiment that links your theory with practice. How about BlocksWorld with a limited number of stacks? But it may fall in the category of \"resource constraints\", is that a problem? Actually, I do want here an environment with a _small_ but not _extremely small_ circuit.\n\nl.334 What does it mean that a model has breadth 2? Does it mean that a layer has 2 neurons? I suppose not, please clarify that.\n\nLIMITATIONS:\nThe limitations are briefly discussed in the conclusion. Negative societal impact is not an issue with this work.", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-06"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper presents a mapping from tractable but incomplete algorithm classical planning into circuits that can be represented by a class of neural networks called relational neural networks RelNN including transformers architecture and graph neural networks. The paper is divided into two parts.\n\nThe first part presents an extension of Chen and Gomez [CG2007] that introduced a notion of width, in turn, related to the more practical notion of width proposed by Lipovetzky and Geffner [LG2012]. The change is about using regression search –not a big change given that as a theoretical algorithm– while keeping the same idea of keeping track of a reduced context.\n\nThe second part considers how to map policies based on those algorithms –given a set of regression rules– into a RelNN. The paper discusses the size of the required network in different cases. The best scenario is when the size of the NN doesn't grow linear with the size of the problems. Using the ideas of the first part, the paper shows how can that be expressed as a RelNN. Preliminary empirical results over a set of problems suggest a MLP is learning a policy that matches the ideas of the paper.\n\nSTRENGTHS:\n- Innovative combination of tractable planning with the expressivity of some NN. Serialization is a suggestive idea for a tractable fragment for planning.\n- The idea of 'regression rule selector' might be hard to use in an effective symbolic algorithm. Framing it as learning is an innovative idea.\n\nWEAKNESSES:\n- The paper offers little insight into how depth and breadth behave in different problems. In particular, it's not clear how the complexity of the rules is related to that. Perhaps a better title for the current manuscript would be \"Relational Neural Network can Solve some Planning Problems\", as the paper does not help to identify what problems can be solved except by trial-and-error. That might be a good contribution, but then the tone of the paper might be different.\n- The paper doesn't clarify whether the goal of the approach is to solve a single problem or problems from a domain.\n\t- For instance, the width of [LG2012] refers to problems, but then they discuss width across instances of a problem.\n\t- This might be easily fixed in the next discussion.\n- Experiments cannot be replicated as we lack details.\n\nQUESTIONS:\n- What's the goal stack in the algorithms?\n- How are the objects encoded in the RelNN?\n\t- What's the difference/similarity in the encoding of problems where the policy have bounded vs unbouded depth?\n- Is the encoding the same for two problems where we interchange the name of two objects? For instance, rename A s B and B as A in a blocks world problem.\n\t- Might such renaming have any effect in the experiments?\n- Are MLP per se part of the class of relational NN?\n- How are the problems and outputs encoded in the MLP?\n- The proof of 4.3 says that rules include \\rho, any kind of FOL. Lemma 4.1 refers to  FOL but only restricts the number of variables.\n\t- How does D change with the complexities of the formulas?\n- In thm 4.3, why does the breath only grows with max(B_r, max arity of predicates)?\n\t- Perhaps it's because the preconditions are evaluated once at a time, but the end of the proof says \"\\beta is required to store the input state\". A state includes more than one predicate.\n\nLIMITATIONS:\nThis is a very interesting line of work combining results in different areas. The theoretical arguments focus on the existence of a bound, there is little formal study or intuition of how the complexity grows across different problems. For instance, if conditions are free FOL, for a fixed number of variables and objects, then the resulting circuits can have many different sizes.\n\nIn general many intuitions are hidden in the proofs ideas and not in the main text. That's a possible reasonable decision, but sometimes the key intuitions are hidden there and not connected with the overall picture.\n\nLacking intuitions, the empirical study offer some light in this direction, but a reader in NeurIPS might need further details on how this was done.\n\nFinally, I think this would be a better paper with a more clear distinction between problems and domains. Previous work using the notion of width tends to discuss it across a class of problems. In that setting, providing or obtaining a set of rules is more clear that for solving a single problem. The set of rules might be very different for subsets of problems. (For instance, blocks world instances where all the blocks start at the table). That framing would allow to establish connections with work in learning for generalized planning. \n\nFor that reason and others, this one is a very relevant reference:\nLearning Sketches for Decomposing Planning Problems into Subproblems of Bounded Width. Dominik Drexler, Jendrik Seipp, Hector Geffner.\nICAPS 2022.\nhttps://arxiv.org/abs/2203.14852 \n\nAll in all, I learned a lot from the paper. That's a positive sign.\n\nMinor point(s):\n- why can serialization be incomplete? Perhaps explaining this might be useful: https://en.wikipedia.org/wiki/Sussman_anomaly", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 4, "DATE": "2023-07-09"}