Graph Learning for Numeric Planning
Authors: Dillon Ze Chen
Sylvie Thiebaux
Venue: NeurIPS 2024 poster (2024)
Keywords: planning
graph neural networks
graph-based learning
reinforcement learning
Primary area: reinforcement_learning

Abstract: Graph learning is naturally well suited for use in symbolic, object-centric planning due to its ability to exploit relational structures exhibited in planning domains and to take as input planning instances with arbitrary number of objects. Numeric planning is an extension of symbolic planning in which states may now also exhibit numeric variables. In this work, we propose data-efficient and interpretable machine learning models for learning to solve numeric planning tasks. This involves constructing a new graph kernel for graphs with both continuous and categorical attributes, as well as new optimisation methods for learning heuristic functions for numeric planning. Experiments show that our graph kernels are vastly more efficient and generalise better than graph neural networks for numeric planning, and also yield competitive coverage performance over domain-independent numeric planners.

Graph Learning for Numeric Planning
Dillon Z. Chen 1.2
Sylvie Thiébaux 1.2
"LAAS-CNRS" University of Toulouse {dillon. cee.innn.ceen.... sylvie. thi (llaau
TThe Australian National University
Abstract
Graph learning is naturally well suited for use in symbolic, object-centric planning due its ability to exploit relational structures exhibited planning domains and to take as input planning instances with arbitrary numbers of objects. Numeric planning is an extension of symbolic planning in which states may now also exhibit numeric variables. In this work, we propose data-efficient and interpretable ma- chine learning models for learning to solve numeric planning tasks. This involves constructing new graph kernel for graphs with both continuous and categorical at- tributes, as well as new optimisation methods for learning heuristic functions for nu- meric planning. Experiments show that our graph kernels are vastly more efficient and generalise better than graph neural networks for numeric planning, and also numeric
yield competitive coverage performance compared to fomain-indepente planners. Code is available at atttpss:/ /gi thub. com/ DilDillllliiii www.llnllheeenggeeee Conooeeeeenooeeeeeeee goose
Introduction
Planning requires long range reasoning over combinatorially large state spaces. Numeric planning is an extension of classical planning in which states have numeric variables and the underlying transition system is built from inequality conditions and assignments over arithmetic expressions of such variables. It was formalised in PDDL 2.1 (FL03) and is undecidable in the general case (He102) which makes it more difficult than classical planning which ISPACE-complets (By194). Numeric planning is effort [CCFLI3] IM17 SHTR20 KSP+ 22 KSB23 SKB23], but this expressivity result implies that building general, scalable numeric planner is a challenging problem.
problem in the symbolic AI community and exhibits significant research
Learning for Planning (L4P) is research direction which focuses learning solve problems from a specified domain in an automated supervised manner [TTXX20 [ST20 FGT KS21, SBG22
SBG23 MLTK23 CTT24a SDS+24 RTG+ 24 [PK24]. Planning tasks in L4P are assumed to exhibit a factored, symbolic representation, which allow us to generate training data in a matter of seconds from easy to solve tasks with bomminninndependen planner. We can then learn domain knowledge in supervised manner that scales planners to significantly larger planning tasks.
This is in contrast to Reinforcement Learning where agents do not require access to well-defined mod- els but spend significant amounts of time exploring and learning from rewards (S998. Regardless, several works have showed that encoding or learning symbolic models for sequential decision mak- ing reasoning and embodied AI tasks [LCZ+ 21 ZYP+22 LSS 22, SCK+ +33 KVS+23
LPM23]
end-to-end reinforcement learning methods. Furthermore, it was shown recently that classical ML methods are much better suited for L4P than deep learning methods for symbolic planning (CTT24b) they (1) can generalise well from small training data, (2) are orders of magnitude more efficient to train and evaluate than deep learning methods, which important in time sensitive tasks such as planning, and (3) have interpretable features to understand what is being learned.
provided better performance and transparency
In this paper study whether this fact carries over to Learning for Numeric Planning (L4NP) [WT24 which now requires reasoning over logic and arithmetic. It is reasonable to think that because neural networks are function approximators, they may offer better reasoning capabilities numbers than 38th Conference on Neural Information Processing Systems (NcurIPS 2024).

---

(b)
1 NOWLLF
initial state
"ask
W x
Eq
T
SVR
ASCWLF
Sec.
k
pool condition
Defn.
A GNN
Sec.
ront
GNN)
Figure I: The GOOSE framework for learning heuristic functions for numeric planning. Cyan colours
indicate components that are influenced by the training phase. (a) A numeric planning state and goal
condition is encoded into a graph G via the vILG representation defined in Defn. 3.1 (b) Graphs are
either embedded into vectors x in Euclidean space with the CCWL kernel from Sec. 3 or transformed
into a graph G' with real-valued matrix representing node features as inputs into GNNs described
in Sec. 4 (c) Features x are fed into linear model, whereas transformed graphs G' are fed into
GNNs. (d) Linear models are either trained by the ranking formulation in Eq. [or by Support Vector
Regression (SVR) with linear kernel. GNN models are either trained by the ranking formulation in
Eq. 2 or by backpropagation minimising the mean squared error (MSE) loss.
just symbols alone. In this paper, we describe the GOOSE framework with classical ML and deep learning configurations for learning heuristic or value functions for use with search in L4NP. Fig. illustrates the GOOSE framework and we outline our contributions as follows.
We introduce a new graph representation of numeric planning tasks for use with classical and deep graph learning methods, namely graph kemnels and graph neural networks, respectively.
We extend the WL kernel [SSVL*11] to handle graphs with both continuous and categorical attributes in a meaningful way which we call the CCWL kernel.
We introduce new ranking formulations GKL16 CEKP23 HTT+24 for learning heuristic or value functions with linear programs.
The structure of the remainder of the paper is as follows. In Sec. we provide the necessary formalism and background for numeric planning, as well as relevant notation. Sec. B] we introduce a new graph encoding vILG and LL kernel for generating features for numeric planning tasks. In Sec. 4 we introduce deep learning architecture for L4NP using graph neural networks. Sec. [5] we describe optimisation methods for L4NP, involving new ranking formulation for learning heuristic functions. In Sec. [6] we describe our experimental setup and results. Related work discussed in Sec. in the appendix. We conclude the paper with final comments in Sec.
2 Background
Numeric Planning Task. A numeric planning task can be viewed as a compact representation of a deterministic, coal-conditioned Markov Decision Process with the use of predicate logic and relational numeric variables. A majority of the remainder of this section formalises the necessary components of a numeric planning task we use in the paper.
A numeric planning task [FL03] is given by a tuple II (Xp,, Xn, 50, G) where X, is a finite set of propositional variables with domain [T, and X,, is finite of numeric variables with domain R.. Let X Xp U X,, denote the set of state variables, where a state is total assignment of values the propositional and numeric variables. The variables implicitly induce a possibly infinite set of states S, where so is the initial state.
A propositional condition is positive (resp. negative) literal T (resp. L) for some propositional variable x E Xpp and a numeric condition has the form g 2 0 where & is an arithmetic expression over numerie variables and D> {2,>, >,-- =}. We write [xr] (resp. [{]*) for the value of a state variable (resp. expression {) in state and VEE for the set of numeric state variables in g. A state s satisfies set of conditions (i.e. set of propositional and numeric conditions) if each condition in the set evaluates to true given the values of the state variables in s. The goal G is a set of conditions and we write Gp (resp. G.)) for the subset of propositional (resp. numeric) goal conditions.
Graphs Optimised fOr Search Evaluation

---

The set A contains a finite number of actions, each consisting of preconditions and effects. Action preconditions pre(a) are sets of conditions, and action effects assign Boolean values to propositional variables and assign the value of an arithmetic expression to numeric variables. An action a is applicable in a state if 8 satisfies pre(a), which case its successor a(s) the state where the effects eff (a) are applied to the state variables in s. If a is not applicable in s, we set a(s) = $1 & S. Each action a has a cost e(a) given by an arithmetic expression.
A plan for numeric planning task is a sequence of actions π = d.,..., an such that s; = az (s of TT for all I i S n and Sn satisfies G; we call Su, $1..... Sn the plan trace of the plan. The plan length |i] and the plan cost are the number of actions in the plan, and the sum of their cost, respectively, plan is optimal if it has the minimum cost among all plans. A numeric planning task is solvable there exists a plan for it, and is unsolvable otherwise. A state s is a deadend if the task with the initial state replaced with 8 is unsolvable. Satisficing planning refers to the problem of finding a plan if exists, or proving that the problem is unsolvable. Optimal planning refers to the problem of finding an optimal plan if it exists, or proving that the problem is unsolvable.
Lifted representation. Numeric planning tasks can be compactly encoded in a lifted representation (O, Ep. E,, Ea,A, A,, so, G) whereby state variables are derived from a set of predicates, functions, and objects. Formally Σn and En are sets of predicate and function symbols, respectively. Each symbol E Σp U Ey. has an arity ng E NNU {0} which depends on σ. Predicates and functions take the form p(æ1. In,)) and f(z1..... : In, ). respectively, where the 2,S are their arguments. Given the varaabee
set O of objects, the propositional and numeric variables are obtained by substituting objects for the arguments of the predicates and functions, resulting in the grounded form p(o1...... O2,,) and f(o)...., On,) ). respectively, where the o;s are objects. Similarly, actions can be represented in a lifted form via set Σn of action symbols and set A of action schemata mapping action symbols to their lifted precondition and effect definitions in terms of predicates and functions. Grounding the set of action schemata results the set of actions A of the planning task. Details are not needed to understand this paper. A domain is set of numeric planning tasks sharing the same set of Ep, Σ,, Eg, and A, and may have constant objects, objects which are shared across all tasks in the domain.
Example: Capacity Constrained Blocksworld. To help digest some of the definitions of numeric planning, we provide example with planning domain we call Capacity Constrained Blocksworld (ccBlocksworld). It is an extension of the original Blocksworld domain in which state consists of towers of blocks and the objective is also a special case of the Hydraulic Blocksworld domain for planning with state constraints, in which blocks are placed on top of pistons which rise fall depending on the configurations of other pistons HIR 18).
stack and unstack blocks to achieve goal configuration. It
In cc ccBlocksworld, we have maximum number of tower locations, and each tower has base limited by the number of blocks it can hold. To model this domain in the lifted representation, we retain the predicate on(r, y) from the original Blocksworld, which indicates block is on another block or base y. Next, we also introduce the function capacity(2) which denotes the remaining number of blocks that are allowed to be placed base 2. The numeric variables instantiated from capacity ty may increase or decrease depending on whether blocks are unstacked from the tower or stacked on top of it. Action schemata preconditions constrain whether block can be placed on a tower with base that has reached its capacity limit or not. The leftmost figure in Fig. 2 Fillustrates an example of a ccBlocksworld problem with an initial state and goal condition. We refer to the Sec. of the appendix for the complete state representation of the problem as well as its PDDL encoding.
Heuristics and Greedy Best First Search. State-of-the- methods for both satisficing and optimal numeric planning [SHTR20, KSP+22 CT24) employ some variant of heuristic search. A heuristic function maps state R U foo} representing an estimate of the cost to reach the goal from the current state, where a value of so estimates that s is a deadend. The optimal heuristic h maps a state to the cost of an optimal plan if it exists, and oc otherwise. The Greedy Best First Search (GBFS) algorithm consists of a priority queue initialised with the initial state as the only element, and main loop which performs the following steps while the queue is non-empty: (1) pop a state 8 with the lowest heuristic value and some tie-breaking criterion from the queue, (2) generate the successors of s via all applicable actions, and (3) check if successor s' is a goal, in which case terminate with the plan traced back from s'. and otherwise add s' to the queue if it has not been seen before. The algorithm determines a problem is unsolvable the main loop completes, in which case the problem has finitely many states of which all have been seen.
3

---

car, F Foom: L) denote a graph with nodes V, undirected Graph and other notations. Let G = EV, F.
edges E _ ( categorical node features F V - Ey where Ey is a finite set, continuous node features F con V - R with d N, and edge labels (categorical edge features) L : E ΣE where ER is a finite set. The neighbourhood of a node u E V in a graph with respect to an edge label c is defined by N, (n) (u Re E, s.t. E (u, v) (u, u) A ((ee)= = 4}- We use denote the concatenation operator for vectors, and [N] to denote (1....,N] N).
3 Relational features for numeric planning
In this section, we describe an automatic method for generating embeddings for numeric planning tasks that may be used for any downstream inference model. The method is an extension of the feature generation method for classical planning [CTT24b) and consists of two main steps: (1) generating a graph representation of planning task, and (2) running variant of the WL-algorithm for generating features for the graph [SSVL* 11]. Extending the first step of the method is simple as it is easy to extend the graph encoding to capture numeric information of the task. This is done in Sec. 3.1 where Whhee we introduce the Numeric Instance Learning Graph (vIL.) representation for numeric planning tasks. The second step is more difficult
require constructing WL-algorithm variant that can handle both categorical and continuous nodes features in meaningful way for numeric planning. This is where we introduce the CCWL algorithm in Sec. 3.2 that handle such node features. Thus, we can generate features for numeric planning tasks by first converting them into the vILG representation, and then running the CCWL algorithm on them.
3.1 Graph encoding of numeric planning tasks
We begin by describing our graph encoding of planning task, namely the Numeric Instance Learning Graph (wIL.). Similarly to the classical case, the graph representation does not encode the transition model of the planning task nor requires grounding all possible variables in the planning task. Thus, our encoding only requires a first-order representation of states, and therefore applies to problems whose transition model is unknown such as in model-free reinforcement learning.
We begin with a descriptive definition of the graph with an example Fig. 2 jillustrating a subgraph of the vILG representation of the example ccBlocksworld problem. In the figure, the nodes in the graph represent the objects (light blue). propositional variables true in the state (green), numeric variables (red), propositional goals (yellow) and numeric goals (not present in the example) of the problem. Blue (resp. orange) edges connect object nodes to goal and variable nodes where the object is instantiated in the first (resp. second) argument of the corresponding node variable or condition.
We provide the formal definition below. Let (s))) denote the set of propositional variables that are true in s, X,, (s) the set of numeric variables. and X (s) Xs) U (s).
Definition 3.1 (Numeric Instance Learning Graph). The Numeric Instance Learning Graph (vILG) of a numeric planning task in the lifted representation G (V, E, F Pemt, F con: L) with
(O,Ep. Ep. E, Ea, A, s0, G) is a graph
nodes V = O ((O) UG, where we assume w.l.o.g. that V(g) S X(0) for all g E G..
edges E = Up= )e (EX(e)) (pp o.) i e [n-]] UUeeoe, ((5 v V(g)).
categorical node features Fest V Ey with Fu) =
(BB(())
if E O if E Xn(ss) if E Gn
(PRED(u) (u), schieved prapeeedd_roppettiinn...ccc (PRED(u). paacteere.prooositiooaa...ll (PRED(u), prapavad..roppstttonnn.. _
if Np(ss) if E G, (,,(0)) if E Xp(so)
FUNC(U)
ACH(W)) (CH(U))
where (B)( = u if u is a constant object and object otherwise, PRED(u) (FUN((U)UUU)) re- turns the predicate/function symbol from which proposition/fleett was instantiated from, COMP(u) E } encodes the comparator type of the numeric goal condition u, and encodes whether so satisfies u.
ACH(U)
mamerecggg, subiieeed_mmeric_eee
continuous node features Feon: V + R where F (u) [u]*0 for nodes Xx (so), (u) _ for nodes u = { D 0 Gn with [ 0.0, and F con (u) = otherwise, and
edge labels L E + EE where for edges of the form e {p, 0.), we have L(e) = i, and otherwise for edges (E. v), we have L(e) = O.

---

initial
OH(on, A.
Olt(on u.p.p.. s.p-s)
OH(capaci ty)
num. valte
Pg good error
F
O1l(bb))
E
A
capacity())
i
k
Ca x) an b)
goal condition
capacity(()
capacity(1)
A
capacity(
0
B
i
j
k
cafa.b)
on(b,
(b. 37
DEL Cd
Figure 2: An example ccBlocksworld task where each base has capacity 3 (left), subgraph of its ILG representation (middle), and the matrix representation of the node features the ILG (right).
In general, given a domain with predicate and function symbols Mp and En, we have that there are [Ev] + 3 Epl + Enl |constant | categorical node features representing the semantics of a node. Continuous node features indicate the value of numeric variables and the error of the expression in so of unachieved numeric goals, and are zero for any other node.
3.2 The CCWL algorithm for numeric planning
The WL algorithm [WL68] has been adapted to computing features for graphs with categorical node attributes by [SSVL+1I
variant of the WL. algorithm for graphs with continuous node attributes has been proposed by [GL + 19 for the purpose of computing kernels with the Wasserstein distance between graph embeddings. However, the graph embeddings themselves are not invariant to the order of graphs in the nodes. Furthermore, from [CTT24b], non-linear kernels result in poorer generalisation compared to linear models in the context L4P due to overfitting to the range of training targets. Morris et al. [MKKM16] constructed kernels for continuous node attributes by hashing Euclidean embeddings into categorical features but such a method loses the semantic meaning of numbers. Thus, we propose a new variant of the WL algorithm for graphs with both categorical and continuous node attributes for generating graph embeddings (CCWL algorithm). This algorithm is summarised in Alg. and also depicted in Fig.
Algorithm 1: CcWL algorithm
Iter 0
0..2
Data: A graph G [V, E, Fest, F L) with continuous and
categorical attributes. deterministic and injective HASH function, allowed colours C = [[C]]] a pooling function POOL, and number of CcWL iterations L.
10.2, 0.44
10.3, 1.3)
[0.5,
iter
Result: Feature vector of size R. (I++)]C
(w) F (v), Ve E V
for E [L] do for V do
HASH(L. (1) a). (1,
MASH(1, (), (2,
((x R (R) + HASH (o))
MASH(2, [(L,))))
M Uje (vE
com(1) [..7 con 10.5 1.5
con(3) [0.4 (.4.0.8..ooo4..
con(4)
0.3 1.3
5 Teat + [M, ....... COUNT(M, [
9c1)
[3
S, = ( s.t. 3j E [0] [7], (v) - ViEC
Reas
Ccon
7 Tean + [con(1) || ... con( ci) = POOLLES, (Feon(u))
Figure CCWL with one itera- tion, POOL = Σ. and C = [4].
return Fent Econ
Lines 3 Alg. are the original steps of the WL algorithm for generating graph embeddings by iteratively refining categorical node features, which we call colours, with two differences. Firstly, we replaced the multi-set with set in the input of the hashing function. This is because in planning, unseen colours arise from graphs with increasing degrees which occur for aut-of-distributio testing problems of increasing size. This problem limited by relaxing the hash input with set, which trades expressivity for generalisation. Secondly, make use of edge labels in the hashing function.
Lines 5 collect the counts of allowed colours C seen during the main loop of the algorithm to generate the categorical feature vector in the form of a histogram. We assume by relabelling colours that = [[C]]. Lines 6 7 generate features from pooling the continuous attributes from different
5

---

groups of nodes. More specifically, for each colour e E C, we find the set of nodes which have been assigned the colour e some time during the refinement process and pool the continuous attributes of these nodes. Thus, we have C pooled continuous feature vectors which we concatenate together. We note that this pooling and concatenation process is invariant to the order of nodes in a graph in contrast to the intermediate graph embeddings generated for Wasserstein WL. graph kernels by Togninalli et al. [TGL* + 19). The algorithm returns the concatenation of the categorical and continuous feature vectors as the final feature vector output for the graph in Line g We note that d = 1 when running CCWL on the vILG representation of a numeric planning task.
We note that a drawback of the algorithm is that continuous attributes are not refined directly. This could be done by introducing one or more aggregation functions as parameters the algorithm and refining continuous attributes by concatenating the aggregations of their neighbouring attributes. However, this method introduces an increase the size of the continuous feature vector exponential in the number of layers, with base equal to the number of aggregation functions chosen. Moreover, we noted from informal experiments that this method led to overfitting of models to a large number of blended continuous features that do not have an obvious relation to the learning target.
Assuming a constant time hashing function, the complexity of the CCWL algorithm is (( + d)) where n |V| of the input graph, 6 = mäx E.EL (u)] is the degree of the graph, d is the dimension of the continuous node attributes, and L the number of layers. The main computation comes from Line 3 which is performed nL times and the hashing function takes an input of size 6. Collecting the categorical feature vector takes the same time, while collecting the continuous feature vector takes (nIdd) time. For reasonably sized d S 6, as in the case of vILG where d = 1, this is the same complexity as the original WL. algorithm for generating graph features, which is (nIL)).
4 Relational neural networks for numeric planning
Deep learning architectures such as graph neural networks (GNNs) |SCT 09
GSR + 17 bene- fit in generating latent representations automatically with backpropagation when trained end-to- end [LHTTS] GNNs also benefit from being able to train and evaluate on arbitrary sized graphs. However, it is generally understood that the expressive power of GNNs is limited by the WL-algorithm and counting logics with two variables [9 BKM 20). This result translates the impos- sibility result of GNNs not being able to learn features that can work well for arbitrary planning domains [SBG22 CTT244] Nevertheless, their application to numeric planning tasks, in which both logical and numeric reasoning is required, is less well understood. Thus, we still propose GNNs as an additional baseline for L4NP and empirically evaluate their performance for numeric planning in Sec. 6
For our GNN architecture, we perform a transformation on the node features of the vILG from
3.1 as input for GNNs that can handle edge labels. More specifically, given a vILG V E, eat V + Σv, Feon - F. L)) we construct a new graph G' with continuous node R Exil+ defined by X(u) (F cu)) [[ri.r2], where (Frrea())) e {0, 10.1) C g denotes a one-hot encoding of the categorical node feature of u, and ri denotes the numerical value of numeric variable nodes defined by r1 [u]=0 if u E (su)s) and 71 0 otherwise, and r2 denotes the goal error for numeric goal nodes defined by r2 = [u]*0 if u E G, and r2 = 0 otherwise. We denote the zILG for GNNs by (V, X, L) with notation for categorical features removed. Thus, we can use this graph encoding of numeric planning tasks as input into any downstream GNN that can handle edge labels or features.
G
attributes X
Fig. 2 illustrates the node feature matrix representation of the vILG encoding of ccBlocksworld task for input to a GNN. Each row represents a node in the graph, with columns representing the semantics of the node well as the value of the numeric variables in the state and error of numeric goal nodes. We note however, that the ccBlocksworld example does not have any numeric goals and thus the last column is zero for all entries.
5 Optimisation formulations for learning heuristic functions
In this section, we describe two optimisation methods used for learning heuristic functions from training data, namely by minimising cost-to-go estimate error and ranking estimate error. Fig. 4 illustrates examples of learned heuristic functions on states of a planning task when trained to zero loss with both the cost-to-go and ranking formulations. We assume that training data for our models consist of set of numeric planning tasks II,,... II, with corresponding optimal plans Ti... T.

---

10
the concept of ranking to learn Figure 4: Examples of heuristic functions that achieve loss when optimising cost-to-go (left) and ranking (right) on an optimal plan. Coloured nodes indicate states on the optimal plan with the goal state indicated by double circle. Edges indicate successors can achieve O loss on the plan trace but may not generalise correctly to state successors. A ranking heuristic does not need represent correct cost-to-go values and only need to satisfy ranking constraints. GBFS will return plan in linear time for the ranking heuristic here but not for the cost-to-go heuristic.
node. A cost-to-go heuristic
We note that a numeric planning task can offer more training data by generating additional tasks and plans from different states in the state space of the task. Each plan is denoted = =a
(5) [r. with
plan trace (6) (() ................................ ......., |z,l' Each state s in plan trace induces new planning task by replacing So with of the original task, with which we can construct graph or vector representations from our aforementioned models.
(1)
Heuristic functions from cost-to-go estimates. We can use planning tasks and corresponding optimal plans as training data for learning a heuristic function representing the estimated cost-to-go to the plan. Each task and corresponding plan म, contributes training data o56) with targets h" ())
for each state o in the plan trace of π. Then given an estimator H. we may try to find weights that minimise the mean squared error (MSE) L(Ø)= = + >. 2 (h (s Ho )s) where N is the normalisation constant and Hg denotes the estimator with weights
Heuristic functions from ranking estimates. The MSE loss is simple but naive method for training heuristic function. Various researchers have instead proposed heuristic functions [6KL16 CEKP23 H2]+ However, drawback of the formulation of the ranking optimisation of previous works that state in plan trace marked as strictly better as its siblings when it could be the case that the siblings may have the same ht value. Furthermore, the formulation in [CEKP23 scales quadratically in the plan trace. We offer novel ranking optimisation criterion that (1) fixes the problem of siblings being misclassified and (2) also results in sparse model. We also offer a corresponding differentiable loss function for use with any end-to-end model.
Our first ranking formulation requires solving an LP as the optimisation problem, similarly to [FCGP19] but only using states from the plan trace, whereas the latter work uses states from the entire state space of the problem. It can also be viewed as an LP encoding of the formulation by Garrett et al. [6KL]6 but fixing the problem of misrepresented siblings and learning sparse weights. Let (UCCS($) denote the set of successors of the state in a planning task by applying all
applicable actions at s. Hence the set of siblings of state ((() in II, state space is SIBLINGS (s =
sUCCs($)2) { }. Let yp denote our feature generation function with yps) E Rd for any state s. Then we can define our optimisation problem as a linear program defined by
min
Zi.JA ||w]]
s.t.
ZL,,A 2 0, ji
(I)
w (p((s/11)-
> cost(a,)) - Z,,5.0
vi [n] , E [l
(p(ss) -(, -(.))))) jijj
Vi E [M] . [[m]]]] E (IBLINGS(
The vector w represents the weights our linear model aims to learn, and the nonnegative slack variables model the soft inequality constraints representing the ranking of states. The optimisation problem is to minimise the the slack variables corresponding to the error of the constraints, and the L norm the weights to encourage sparsity.
We next offer a differentiable loss function version the previous model which we can use as fair comparison when combining it with our GNN architecture in Sec. commprrd to combining (1) with features generated in Sec. The idea is to replace the slack variables with the max function:
L(0)- > 1J
max (0, ))) Ho(s)1))
claa)) + Σ max (0. (+(5) Ho(s.))) (2)
*o ESIEINNES)
7

---

Number of orbjects
10a
974
train
292
10a
194
10+
98
Time
10:
10
10
10
10
Spanner [ (68)
Blackoworld
(blocks)
Childsnack
(chileren)
Ferry
(Gars]
Micanicc
(paseenger))
Rover
(objectives)
Satalitte
Spanner
Ispannerr
Transport (packages)
Blocksworld
Childsnack (74)
Ferry
Miconic (99)
Rovers (92)
Satellite
(56)
Tranupooi (39)
Domain
Domain
Figure 5: Number of objects in training and testing problems (left) and distributions of training data
generation time with number of training problems (right) per domain. Note the log scales.
Experiments
6.1 Numeric planning benchmarks
We take domains out of 10 domains from the International Planning Competition 2023 Learning Track (IPC-LT) [SSA23] and either convert them to equivalent numeric formulations, or introduce numeric variables to model extra features such capacity constraints. The two domains from the IPC-LT that we do not convert into numeric domains are Floortile and Sokoban which do not have any benefit from compilation a numeric representation nor exhibit any interesting features that can be modelled with numeric variables. The domains we considered from the IPC-LT are summarised in Fig. [ alongside the sizes of training and testing tasks, and time generate training data. Each domain consists of 90 testing problems and at most 99 small training problems for which the median time for generating an optimal training plan is less than second and few outliers taking more than a minute. We refer to the appendix for further details on the domains.
6.2 Experimental setup
Training, As discussed in Sec. 5| we only consider optimal plans from small problems as training data. We compute them with the Numeric Fast Downward planner [AN17] using A search and the admissible & McUT heuristic IKSP#22) with 30 minute timeout and 8GB main memory.
We consider 4 model configurations. Firstly, we CCWL features from Sec. 3 with Support Vector Regression and the linear dot product kernel to learn a linear model for cost-to-go go estimation (LOCWL.F) Next, we use CCWL features in optimisation problem in ( with CPLEX version 22.11 and a timeout of 600 seconds for ranking estimation (ACCWLF). ). Both CCCLFF and / models have allowed colours C in Alg. [ given by all the refined colours seen during training. We also have cost-to-go (NGNN) ) and ranking (N,GN) estimation models using GNNs operating on zILG representations of planning tasks and optimised with the MSE loss function and respectively. For the backbone GNN, we use a Relational Graph Convolution Network (SKB 18 but replacing the mean aggregation function with the element- wise max operator in the passsae-paassing update step: hg (1+1) = (Wah + E.Ex, max (w)
W!!n!!) ("))) where denotes the GNN layer, a is
implemented with the leaky ReLU function, and Wn and w!!) are learnable weight matrices. Each GNN has hidden dimension of 64, and is trained with the Adam optimiser [KB15) with initial learning rate of 10-3 and batch size of 16. A scheduler reduces the training loss by a factor of 10 loss does not improve after 10 epochs. Training then terminates the learning rate falls below 10 10-5 Let L denote the iterations hyperparameter for CCWL models and number of layers for GNN models.
Evaluation. We consider several numeric planners baselines for benchmarking the effectiveness of learning. We first include LLMMT as the only optimal planner baseline as is also the training data generator but solves a more difficult problem of optimal planning compared to satisficing planning. We consider the Metric-FF planner (M-FF) [Hot03), and the h ADD h MRP h MRP +hj and M(3h|33n) configurations in the ENHSP planner (SHTR20 SSSGG2 CT24 We have that ADD and MRP planners that perform GBFS with a single heuristic only, while MRP +hj and M(3) |(3))3n) use additional techniques (macro actions, multiple queues, and novelty heuristics) to boost planning performance. Our WL and GNN models are all used in single-queue GBFS with the learned heuristic function, with Numeric Fast Downward the backend search implementation. All baselines and models are run on single Intel Xcon Platinum 8268 (2.90 GHz) core with a minute timeout for search and

---

Table 1: Coverage of numeric domain- indeeideedddedddtt the new learning planners (N GNNN hCCWLF h
hCCWLF) with = 1, and the best learner configuration score on cach domain (Best Learner). Higher
values are better (t), with the top three scores in each row except the rightmost entry indicated by the cell colour intensity, All planner configurations except op LMCUT are satisficing planners.
Planner Baselines
Learners (new)
GBFS heuristic
Best Learner
M(36] 3m)
NR0 +-b
LMcot
Hpp
M-FF
MRP
4,80
.NNN
.ONN
Ta0k
COWLLF
COWLF
Paak
Domain
Blocksworld
9
23
18
24
22
19
29
Childsnack
20
49
14
53
25
20
17
22
22
90
90
Ferry
33
60
60
57
60
57
60
60
70
71
73
Miconic
30
68
61
64
51
63
64
90
90
90
Rovers
10
22
2.3
30
Satellite
18
38
24
29
21
23
19
14
23
16
26
Spanner
30
6
35
76
42
42
90
90
90
90
90
Transport
12
55
49
40
32
40
34
38
40
46
48
Σ
159
328
273
369
281
264
319
326
379
445
476
8GB of main memory. Tab. summarises the coverage results of all considered planners the benchmarks, with more details provided in the appendix.
How do learning approaches compare to Nomain-independent numeric planners? From Tab. [ we note that our best performing model with I. 1 Mank CCCLLF and outperforms all domain- independent planners for satisficing planning on 4 out of 8 domains. Increasing L to 2 brings heCWLF to achieve the best coverage on Blocksworld. The domains which learners fall behind on are Rovers, Satellite and Transport, even when taking the best hyperparameter configuration. The former two are difficult as they require features more expressive than those generated by graph learning approaches to capture the semantics of reasoning required solve the problems (SBG22), while the latter requires path finding which is not possible for learners with finite receptive fields (TTTX20). These results hold for classical planning and thus also for our extension to numeric planning. Gen- erally the best performing planner on a domain expands fewer nodes than the other planners. With regards to plan length, NCCWLF performs best for Blocksworld but is marginally worse than the best of the Nomain-indeeennn
numeric planners for Rovers, Satellite and Spanner.
How do CCWL models compare to GNN models? From Tab. D we see that the CCWL models always have similar or better performance than the corresponding GNN models, when comparing cost-to-go and ranking estimates. The performance of planners which use GBFS and heuristic depend on the heuristic evaluation speed, in which more search can be done in the time limit, or the quality of the heuristic, in which search can be more informed. Fig. 8 in the appendix shows that GNN are generally at least an order of magnitude slower than CCWL models for heuristic evaluation due performing intensive matrix operations. We note that GNN models and could be sped up with access GPUs. Fig. illustrates the number of node expansions of GNN and CCWL L models and note that there is clear winner between the approaches across all domains, with the exception of CCWLF generalising perfectly on Childsnack where other models rank could not. Thus, can conclude with respect to planning efficiency that CCWL models generally outperform their GNN counterparts due to faster heuristic evaluation speeds, while generally both models have similar generalisation performance.
evaluated CPUs
How do ranking models compare to cost-to-go models? From Tab. ranking models outperform cost-to-go models in total coverage. However, their performance is incomparable across domains even when looking at Fig. 6b with the exception CCWL being able to achieve perfect performance on Childsnack. Nevertheless, on domain-mode pairs for L ranking models achieve strictly better coverage, while the converse is only true for domain-modes pairs. This suggests a bias favouring ranking models which can be explained by their advantages covered in Sec. namely that

---

expanded
expanded
plan length
plan length
10'
107
10a
10a
10a
10a
10a
h
10a
10a
10a
101
10
101
10a
10
104
10a
10a
10
10a
10
10
10-
10
102
10
10-
10a
cua!
(a) GNN (boost
GNN
Prank
GNN
vs. CCWL models (h Brank
expanded
expanded
plan length
plan Length
107
107
10a
103
105
M.COML
17
10
102
10a
10a
10A
10
10a
10
10a
101
10a
10
107
10a
10a
10
10
10a
10
102
10
10a
101
10a
103
hight
hrow
4
h/rW
(b) Cost-to-go (/ooot CCWLF
GNN
vs. ranking heuristic models (aak
Hunk
CCWLF
blocksworld
childsnack
ferry
miconic
rovers
spanner
transport
(c) Plot legend.
Figure 6: Plot comparisons of expanded nodes and plan length of selected pairs of models with A point (I, represents the metric of the models indicated the and axis on the domain. Points on the top left (resp. bottom right) triangle favour the model on the z-axis (resp. y-axis).
they implicitly use more training data by considering successor states of plan trace states, and have larger solution space as they are not restricted to predicting exact value.
What is the effect of number of iterations for CCWL models and layers for GNNs? The hyperparameter T., which denotes the number of iterations (resp. layers) for CCWL (resp. GNN) models, generally plays an important role planning performance. This because increasing improves model expressivity and reasoning capabilities, but comes at the cost of heuristic evaluation time and increased possibility of overfitting the training data. From Tab. in the appendix, that surprisingly for most domains and models L = 0 or L 1 provides the best coverage, while increasing L rarely improves coverage. This suggests that heuristic evaluation time plays an important role in planning performance for domains that cannot be solved with the learner's expressivity.
note
7 Limitations
The setup of our work is limited to the assumption that the problems being solved can be explicitly represented in symbolic language such as PDDL. The assumption of the existence PDDL encodings of planning problems allows us to generate training data quickly with domain -edainnindeeenneett numeric planners for supervised training. Furthermore, experiments and theoretical insights also show that our proposed techniques have room for improvement as there are still classes of numeric planning tasks with which our models cannot learn and generalise well in.
Conclusion
We have proposed new graph embedding algorithm, the CCWL algorithm, and optimisation criterion for learning heuristic functions for numeric planning. Planning tasks are encoded as Numeric Instance Learning Graphs (vILG) which we run our CCWL algorithm for generating features. Our numeric planning features are interpretable and efficient to generate. Experimental results show the effectiveness of our approach by achieving competitive performance over both deep learning architectures and Domain-indeeeene work by improving the expressivity of our algorithms for capturing more complex numeric domains. Lastly, one can learn forms of domain knowledge different from heuristic functions with our new numeric planning features and graph representations such as policies (W224) portfolios [MFH and detecting relevant objects [SCC+21]
numeric planners. Furthermore, we have identified future
10

---

Acknowledgements
The authors thank the reviewers and Giacomo Rosa for the helpful comments and suggestions. The computing resources for the project was partially supported by the Australian Government through the National Computational Infrastructure (NCT) under the ANU Startup Scheme. ST was supported by the Australian Research Council grant DP220103815 15, by the Artificial and Natural Intelligence and by the European
Toulouse Institute (ANITI) under the grant agreement Union's s Horizon Europe Research and Innovation program under the grant agreement TUPLES No. 101070149.
References
[ACSJ22] Javier Segovia Aguas, Sergio Jiménez Celorrio, Laura Sebastiá, and Anders Jonsson. Scaling-up generalized planning as heuristic search with landmarks. In SOCS, 2022.
[AJJ18] Javier Segovia Aguas, Sergio Jiménez, and Anders Jonsson. Computing hierarchical finite state controllers with classical planning. J. Artif. Intell. Res., 62, 2018.
[AJJ21] Javier Segovia Aguas, Sergio Jiménez, and Anders Jonsson. Generalized planning as hcuristic search. In ICAPS, 2021.
[AN17] Johannes Aldinger and Bernhard Nebel. Interval based relaxation heuristics for numeric planning with action costs. In KI, 2017.
[APK24] Forest Agostinelli, Rojina Panta, and Vedant Khandelwal. Specifying goals to deep neural networks with answer set programming. In ICAPS, 2024.
[BG18] Blai Bonet and Hector Geffner. Features, projections, and representation change for generalized planning. In IJCAI, 2018.
[BG20] Blai Bonet and Hector Geffner. Qualitative numeric planning: Reductions and com- plexity. Artif. Intell. Res., 69, 2020.
[BKM-2N 20] Pablo Barceló, Egor V. Kostylev, Mikaël Monet, Jorge Pérez, Juan L. Reutter, and Juan Pablo Silva. The logical expressiveness of graph neural networks. In ICLR, 2020.
[BPG09] Blai Bonet, Héctor Palacios, and Hector Geffner. Automatic derivation of memoryless policies and finite-state controllers using classical planners. In ICAPS, 2009.
[BPG10] Blai Bonet, Héctor Palacios, and Hector Geffner. Automatic derivation of finite-state machines for behavior control. In AAAI, 2010.
[By194] Tom Bylander. The computational complexity of propositional STRIPS planning. Artif. Intell., 69, 1994.
[CA]19] Sergio Jiménez Celorrio, Javier Segovia Aguas, and Anders Jonsson. A review of generalized planning. Knowl. Eng. Rev. 34, 2019.
[CCFL13] Amanda Jane Coles, Andrew Coles, Maria Fox, and Derek Long. hybrid LP-RPG heuristic for modelling numeric resource flows in planning. J. Artif. Intell. Res., 46. 2013.
[CEKP23] Leah Chrestien, Stefan Edelkamp, Antonín Komenda, and Tomás Pevný. Optimize planning heuristics to rank, not to estimate cost-to-goal. In NeurIPS, 2023.
[CH$24] Dillon Z. Chen, Rostislav Horčík, and Gustav Šír. planning with background knowledge. CoRR, abs/2410.07923,, 2024.
Deep learning for generalised
[CT24] Dillon Z. Chen and Sylvie Thiébaux. Novelty heuristics, multi-queue search, and portfolios for numeric planning. In SOCS, 2024.
[CTT24a] Dillon Z. Chen, Sylvie Thićbaux, and Felipe Trevizan. Learning domain domain-independent heuristics for grounded and lifted planning. In AAAI, 2024.
[CTT24b] Dillon Z. Chen, Felipe Trevizan, and Sylvie Thiébaux. Return to tradition: Learning reliable heuristics with classical machine learning. In ICAPS, 2024.
11

---

[FCGP19] Guillem Francès, Augusto B. Corrèa, Cedric Geissmann, and Florian Pommerening. Generalized potential heuristics for classical planning. In IJCAI, 2019.
[FGT-22] Patrick Ferber, Florian Geißer, Felipe Trevizan, Malte Helmert, and Jörg Hoffmann. Neural network heuristic functions for classical planning: Bootstrapping and compari- son to other methods. In ICAPS, 2022.
[FL03] Maria Fox and Derek Long. PDDL2.1: an extension to PDDL for expressing temporal planning domains. J. Artif. Intell. Res., 20, 2003.
22) Clement Gehring, Masataro Asai, Rohan Chitnis, Tom Silver, Leslie Pack Kaelbling. Shirin Sohrabi, and Michael Katz. Reinforcement learning for classical planning: Viewing heuristics dense reward generators. In ICAPS, 2022.
[Gef18] Hector Geffner. Model-free, model-based, and general intelligence. In IJCAI, 2018.
[GKL16] Caelan Reed Garrett, Leslie Pack Kaelbling, and Tomás Lozano-Pérez. Learning to rank for synthesizing planning heuristics. In IJCAI, 2016.
[GRH24] Claudia Grundke, Gabriele Röger, and Malte Helmert. Formal representations of classical planning domains. In ICAPS. AAAI Press, 2024.
[GSR 17] Justin Gilmer, Samuel S. Schoenholz, Patrick F. Riley, Oriol Vinyals, and George E. Dahl. Neural message passing for quantum chemistry. In ICMI., 2017.
[GT04] Charles Gretton and Sylvie Thiébaux. Exploiting first-order regression in inductive policy selection. In UAI. AUAI Press, 2004.
[Hel02] Malte Helmert. Decidability and undecidability results for planning with numerical state variables. In AIPS, 2002.
[HG11] Yuxiao Hu and Giuseppe De Giacomo. Generalized planning: Synthesizing plans that work for multiple environments. In IJCAI, 2011.
[HG13] Yuxiao Hu and Giuseppe De Giacomo. A generic technique for synthesizing bounded finite-state controllers. In ICAPS, 2013.
[HI 18] Patrik Haslum, Franc Ivankovic, Miquel Ramírez, Dan Gordon, Sylvie Thiébaux, Vikas Shivashankar, and Dana S. Nau. Extending classical planning with state constraints: Heuristics and search for optimal planning. Artif. Intell. Res., 62, 2018.
[Hof03] Jörg Hoffmann. The metric-ff planning system: Translating "ignoring delete lists" to numeric state variables. J. Artif. Intell. Res., 20, 2003.
HTT-244 24] Mingyu Hao, Felipe Trevizan, Sylvie Thiébaux, Patrick Ferber, and Jörg Hoffmann. Guiding GBFS through learned pairwise rankings. In IJCAI, 2024.
[KVM222 Rodrigo Toro Icarte, Toryn Q Klassen, Richard Anthony Valenzano, and Sheila A. Mcllraith. Reward machines: Exploiting reward function structure in reinforcement learning. J. Artif. Intell. Res., 73, 2022.
[IM17] Leon Illanes and Sheila A. Mcllraith. Numeric planning via abstraction and policy guided search. In IJCAI, 2017.
[IM19] León Illanes and Sheila A. Mcllraith. Generalized planning via abstraction: Arbitrary numbers of objects. In AAAI, 2019.
[KB15] Diederik P. Kingma and Jimmy Ba. Adam: A method for stochastic optimization. In ICLR, 2015.
[Kha99] Roni Khardon. Learning action strategies for planning domains. Artif. Intell., 113(1-2). 1999.
[KS21] Rushang Karia and Siddharth Srivastava. Learning generalized relational heuristic networks for model-agnostic planning. In AAAI, 2021.
12

---

[KSB23] Ryo Kuroiwa, Alexander Shleyfman, and J. Christopher Beck. Extracting and exploiting bounds of numeric variables for optimal linear numeric planning. In ECAI, 2023.
[KSP Ryo Kuroiwa, Alexander Shleyfman, Chiara Piacentini, Margarita P. Castro, and J. Christopher Beck. The Im-cut heuristic family for optimal numeric planning with simple conditions. J. Artif. Intell. Res., 75, 2022.
[KVS 23] Ken Kansky, Skanda Vaidyanath, Scott Swingle, Xinghua Lou, Miguel Lázaro-Gree Gredilla, and Dileep George. Pushworld: A benchmark for manipulation planning with tools and movable obstacles. CoRR, ftss/2311.10288 2023.
[LBHI5] Yann LeCun, Yoshua Bengio, and Geoffrey E. Hinton. Deep learning. Nat., 521(7553), 2015.
[LCF-22] 22] Xiaoyou Lin, Qingliang Chen, Liangda Fang, Quanlong Guan, Weiqi Luo, and Kaile Su. Generalized linear integer numeric planning. In ICAPS, 2022.
[LCZ-21] Jiaoyang Li, Zhe Chen, Yi Zheng, Shao-Hung Chan, Daniel Harabor, Peter J. Stuckey, Hang Ma, and Sven Koenig. Scalable rail planning and replanning: Winning the 2020 flatland challenge. In ICAPS, 2021.
[LPM23] Xiaotian Liu, Héctor Palacios, and Christian Muise. Egocentric planning for scalable embodied task achievement. In NeurIPS, 2023.
[LSS-22 Leonardo Lamanna, Luciano Serafini, Alessandro Saetti, Alfonso Gerevini, and Paolo Traverso. Online grounding of symbolic planning domains in unknown environments. In KR, 2022.
[MFH 20] Tengfei Ma, Patrick Ferber, Siyu Huo, Jie Chen, and Michael Katz. Online planner selection with graph neural networks and adaptive scheduling. In AAAI, 2020.
[MKKM16] Christopher Morris, Nils M. Kriege, Kristian Kersting, and Petra Mutzel. Faster kernels for graphs with continuous attributes via hashing. In ICDM, 2016
[MKS-15] 15] Volodymyr Mnih, Koray Kavukcuoglu, David Silver, Andrei A. Rusu, Joel Veness, Mare G. Bellemare, Alex Graves, Martin A. Riedmiller, Andreas Fidjeland, Georg Ostrovski, Stig Petersen, Charles Beattie, Amir Sadik, Ioannis Antonoglou, Helen King, Dharshan Kumaran, Daan Wierstra, Shane Legg, and Demis Hassabis. Human-level control through deep reinforcement learning. Nat., 518(7540). 2015.
[MLTK23] Jiayuan Mao, Tomás Lozano-Pérez, Joshua B. Tenenbaum, and Leslie Pack Kaelbling. What planning problems can A relational neural network solve? In NeurlPS, 2023.
[MV21] Andrea Micheli and Alessandro Valentini. Synthesis of search heuristics for temporal planning via reinforcement learning. In AAAL AAAI Press, 2021.
[RTG 24] Nicholas Rossetti, Massimiliano Tummolo, Alfonso Emilio Gerevini, Luca Putelli, Ivan Serina, Mattia Chiari, and Matteo Olivato. Learning general policies for planning through GPT models. In ICAPS, 2024.
[SB98] Richard S. Sutton and Andrew G. Barto. Reinforcement learning an introduction. Adaptive computation and machine learning. MIT Press, 1998.
[SBG22] Simon Ståhlberg, Blai Bonet, and Hector Geffner. Learning general optimal policies with graph neural networks: Expressive power, transparency, and limits. In ICAPS. 2022.
[SBG23] Simon Ståhlberg, Blai Bonet, and Hector Geffner. Learning general policies with policy gradient methods. In KR, 2023.
[SCC-21] Tom Silver, Rohan Chitnis, Aidan Curtis, Joshua B. Tenenbaum, Tomás Lozano-Péree and Leslie Pack Kaelbling. Planning with learned object importance in large problem instances using graph neural networks. In AAAI, 2021.
13

---

[SCK 23] Tom Silver, Rohan Chitmis, Nishanth Kumar, Willie McClinton, Tomás Jozano-Péred Leslie Pack Kaelbling, and Joshua B. Tenenbaum. Predicate invention for bilevel planning. In AAAI, 2023.
[SDS 24] Tom Silver, Soham Dan, Kavitha Srinivas, Joshua B. Tenenbaum, Leslie Kaelbling, and Michael Katz. Generalized planning in pddl domains with pretrained large language models. In AAAI, 2024.
[SGT 09] Franco Scarselli, Marco Gori, Ah Chung Tsoi, Markus Hagenbuchner, and Gabriele Monfardini. The graph neural network model. IEEE Trans. Neural Networks, 20, 2009.
[SHM- 16] David Silver, Aja Huang. Chris Maddison, Arthur Guez, Laurent Sifre, George van den Driessche, Julian Schritwieser, loannis Antonoglou, Vedavyas Panneershelvam, Mare Lanctot, Sander Dieleman, Dominik Grewe, John Nham, Nal Kalchbrenner, llya Sutskever, Timothy P. Lillicrap, Madeleine Leach, Koray Kavukcuoglu, Thore Graepel, and Demis Hassabis. Mastering the game go with deep neural networks and tree search. Nat., 529(7587), 2016.
[SHTR20] Enrico Scala, Patrik Haslum, Sylvie Thicbaux, and Miquel Ramírez. Subgoaling techniques for satisficing and optimal numeric planning. J. Artif. Intell. Res., 68, 2020.
[SIZ08] Siddharth Srivastava, Neil Immerman, and Shlomo Zilberstein. Learning generalized plans using abstract counting. In AAAI, 2008.
[SKB- 18] Michael Sejr Schlichtkrull, Thomas N. Kipf, Peter Bloem, Rianne van den Berg, Ivan Titov, and Max Welling. Modeling relational data with graph convolutional networks. In ESWC, volume 10843, 2018.
[SKB23] Alexander Shleyfmaa, Ryo Kuroiwa, and J. Christopher Beck. Symmetry detection and breaking in linear cost-optimal numeric planning. In ICAPS, 2023.
[Sri10] Siddharth Srivastava. Foundations and Applications of Generalized Planning. PhD thesis, University of Massachusetts Amherst, 2010.
[Sri23] Siddharth Srivastava. Hierarchical decompositions and termination analysis for general- ized planning. J. Artif. Intell. Res., 77, 2023.
[SSA23] Jendrik Seipp and Javier Segovia-Aguas International planning competition 2023 learning track. Sttts:///bcc202-eearnni
github. /to/ 2023.
[SSSG20] Enrico Scala, Alessandro Saetti, Ivan Serina, and Alfonso Emilio Gerevini. Search- guidance mechanisms for numeric planning through subgoaling relaxation. In ICAPS, 2020.
11] Nino Shervashidze, Pascal Schweitzer, Erik Jan Van Leeuwen, Kurt Mehlhorn, and Karsten M Borgwardt. Weisfeiler-lehman graph kernels. Journal of Machine Learning Research, 12, 2011.
[STT20] William Shen, Felipe Trevizan, and Sylvie Thiébaux. Learning Domain-Inddpendent Planning Heuristics with Hypergraph Networks. In ICAPS, 2020.
[ Siddharth Srivastava, Shlomo Zilberstein, Neil Immerman, and Hector Geffner. Quali- tative numeric planning. In AAAI, 2011.
[TGL-19] Matteo Togninalli, M. Elisabetta Ghisu, Felipe Llinares-L.ópez, Bastian Rieck, and Karsten M. Borgwardt. Wasserstein weisfeiler- graph kernels. In NeurlPS, 2019.
[TTTX20] Sam Toyer, Sylvie Thiébaux, Felipe Trevizan, and Lexing Xie. Asnets: Deep learning for generalised planning. JAIR, 68, 2020.
[WL68] Boris Weisfeiler and A. A. Leman. A reduction of graph to canonical form and an algebra arising during this reduction. Nauchno- Tercckoo-Tcchiihesskaya Informatsiya, 2, 1968.
[WT24] Ryan Wang and Sylvie Thiébaux. Learning generalised policies for numeric planning. In ICAPS, 2024.
14

---

[XHLJ19] Keyulu Xu, Weihua Hu, Jure Leskovec, and Stefanie Jegelka. How powerful are graph neural networks? In ICLR, 2019.
[ZYP 22] Yichi Zhang, Jianing Yang, Jiayi Pan, Shane Storks, Nikhil Devraj, Ziqiao Ma, Keun- woo Peter Yu, Yuwei Bao, and Joyce Chai. DANLI: deliberative agent for following natural language instructions. In EMNLP, 2022.
15

---

initial state
goal condition
D PC
Figure 7: Left: initial state of a ceBlocksworld problem, where base i, j. and k each have a load limit of blocks. Right: the goal condition where A is top of B which is top of i.
A More Details for the ccBlocksworld Example
We repeat the running ceBlocksworld example in Fig. 7 Listings 2 and provide the explicit PDDL domain and problem encodings for the running ccBlocksworid example. An optimal plan for the problem is given as follows on the left, and an optimal plan without capacity constraints on the right.
(unstack j)
1. (unstack f j)
2. (stack a 1)
2. (stack f e k)
3. (unstack d b j)
3. (unstack d b j)
4. (stack d i)
4. (stack d f k)
5. (pickup b j)
5. (pickup i)
(stack k)
6. (stack a k)
7. (unstack d i)
7. (pickup j)
8. (putdown d j)
8. (putdown b i)
9.
(unstack a i)
9. (unstack d k)
10.
(stack j)
10. (stack b 1)
(pickup a 1)
12.
(stack j)
13.
(unstack k))
14.
(putdown b 1)
15.
(unstack j)
(stack i)
Listing 1: PDDL encoding for the ccBlocksworld problem in Fig.
(define (prublem running-example) (:domain ceblocksworid)
obojects
b block
jjk- base
(: inin
(arm_empty)
(capacity i)
[= ceapacity }] 0)
[= (capacity k)
(clear
1)
(above a. i))
(clear [)
d)
(on b ))
(above
(above
(above })
(clear e)
(on c)
(above
(ahove e k)
(and
(elear a)
(on a b)
con b
Listing 2: PDDL encoding for the ccBlocksworld domain.
16

---

(define (domain ceblocksworld)
(requirements :strips :tyingg )
(types
block base object
G (predicates
(on
1x block
2y object)
(above
7x block
base
(clear
2x object
(holding
7x blook
(arm_emply
(: (ffunctions
(capacity
7x base
(: actioon pickup
paraameters (?bleck block ?base base)
precondition (and
(on ?block ?base)
(above ?block ?base)
(clear ?block)
(arm_emply)) eeffect (and
bblock 2base
(not (above ?block ?base))
(not (clear ?block))
(clear ?base)
(holding ?block)
(not (arm_empty (inerease (capacity ?base) 1))
(: (:action putdown
: parameters (?bleck block ?base base)
preecnnditoon (and
(halding ?hlack)
(clear ?base)
(<= (capacity 3base )))
(and
(not (holding ?block))
(not (clear Tbase))
(on ?block ?base)
(above ?hlock ?hase)
(clear ?block)
(arm_emply) (decrease (capacily ?base) 1))
(: action unstack
:parameters (?hlock_a block ?black_b block ?hase base)
: :precondiiion (and
(on ?block_a ?block_b)
(above ?block_a ?base)
(clear ?block_a) (arm_empty))
eeffect (and
Phlock (block_b)
(not (above ?block_a ?base))
(elear ?block_a))
(clear ?block_b)
(holding ?hlock_a)
(not (arm_empty)) (increase (capacity ?base) I))
(: (caction stack
: parameters (?block_a block ?block_b- block ?base precondition (and
(holding ?block_a)
(clear ?block_b)
(ahove ?hlock_ ?base)
(<= (capacity ?base)))
: ceffect (and
(holding ?block_a))
(not (clear ?hlock_bi
(on ?block_A a ?block_b)
(above ?block_a ?base)
(clear ?block_a) (arm_empty)
[decreaee (capacity ?base) 1))
17

---

B Related Work
Two related fields Learning For Planning (L4P) and Learning For Numeric Planning (L4NP) are Generalised Planning (GP) and Reinforcement Learning (RL). In the following subsections, we outline the main difference between L4P with the respective related fields as well as corresponding related work.
B.1 Generalised planning
GP consists of automatically characterising the solution of a (possibly infinite) set of planning tasks [Sri10 SIZ08]. The most common characterisations are action policies. but other character- isations also include finite state controllers [BPG09] BPG10 HGII HGI3 (JJI8), and programs
with branching and looping [AJ2T ACS122]. Logic programming approaches involving decision
lists KKa999 GT04] and Datalog programs [GRH24 CH$24 have also been used characterise solutions for planning domains. We refer to articles [CAJ19] and [Sr23] for more detailed surveys of GP. The difference L4P and GP can be subtle given that there a non-empty intersection between the two fields, and works in both fields generally aim to compute structures that solve problems from a given domain. The way we differentiate the two fields is that L4P follows generally follows traditional supervised learning approaches, whereas GP can be likened to performing program synthesis.
With regards to numeric planning, Srivastava et al. SZIG11 introduced Qualitative Numeric Planning (QNP) which is a subset of numeric planning where numeric variables exhibit non-negative domains, and actions increase or decrease the value of numeric variables by indeterminate amounts. A solution for a QNP is policy which can be used to represent solutions for sets of planning tasks. QNP has been shown be equivalent fully observable non-deterministic (FOND) planning (BG20 arising from the non-deterninism of action effects, and the connection between FOND and GP has often shown itself when used to synthesise generalised policies [BG18 [M19] Lin et al. [CF#222 studies GP for a more expressive class of numeric planning, by allowing for integer numeric variables and employing linear expressions in conditions and action effects. Their approach involves synthesising programs that allow for branching and looping. Lastly, *ASNets [WT24] extends ASNets TTTX20 in order to learn policies with a neural network architecture for planning.
B.2 Reinforcement Learning
RL is learning paradigm for decision making that does not have access model and instead learns from rewards (SB98). RL has achieved promising results in games when combined with deep learning [MKS 15 SHM + 16). A major difference between RL and L4P is that the former requires reasoning over dense reward functions, whereas the latter requires reasoning over logic [Gef[8]. Nevertheless, there has been some preliminary work looking the intersection of RL and planning. Reward machines [22 are logical language used for specifying reward functions for RL problems, inspired by the declarative nature of the planning as modelling paradigm. RL has also been applied directly into planning tasks, done by [MV21] for temporal planning. Rewards are mostly sparse, with being reward for achieved goals, minor 10 10-3 rewards for achieved goal propositions, and no reward otherwise. Gehring et al. [GAC+22 explored introducing denser reward functions to planning through domain-indee ienaainnindeendeent heuristics to allow for RL approaches. Supervised RL has also been used for learning planning policies [SBG23). Nevertheless, the cases for RL and planning are generally different, with RL being more suited for control tasks in continuous or dynamic environments such in robotics, and planning being more suited for combinatorial tasks in discrete or abstract environments such as in logistics.
C Description of Benchmark Domains
C.1 Numeric (Capcity Constrained) Blocksworld
This domain was described in Sec. 2 A task from the domain consists of n blocks stacked on top of one another to form towers on top of bases. Each base has a capacity of how many blocks can support. The goal is to stack and unstack blocks to achieve target tower configuration. The numeric component of this domain arises from modelling the capacity of bases. Training problems have n E [2, 11] blocks while testing problems have n E [5, 488] blocks.
C.2 Numeric Childsnack
A task from the domain consists of feeding children with sandwiches in / locations, of which some are allergic to gluten. There are a finite amount of gluten-free (GF) and non-GF ingredients.
18

---

GF sandwiches can only be made from GF ingredients, whereas non-GF sandwiches can be made with any ingredients. Children allergic to gluten are only allowed eat GF sandwiches while the remaining children can eat any type of sandwich. Thus, the problem has deadends because resources are finite and can be wasted. The goal is to make sandwiches and feed all the children satisfying the aforementioned rules. The numeric component of the domain arises from modelling the ingredient and sandwich resources. Training problems have c E [1,8] children while testing problems have E [4, 292] children.
C.3 Numeric Ferry
A task from the domain consists of e cars spread across locations. A ferry is able to transport up to a fixed amount of cars around to different locations. The goal of the domain to transport the cars with the ferry to various target locations. The numeric component of the domain arises from modelling the capacity of the ferry. Training problems have E [1, 20) cars while testing problems have [4, 974] cars.
C.4 Numeric Miconic
A task from the domain consists of passengers with different weights spread across f floors. There is a single elevator with a fixed load capacity that can transport passengers between floors. Furthermore, if the load of the elevator exceeds a secondary threshold, it takes twice as long to move between floors. The goal of the domain to move the passengers to their target floors. The numeric component of the domain arises from modelling the weight of the passengers and load capacity of the elevator. Training problems have E [1, 10] passengers while testing problems have [I, 485] passengers.
C.5 Numeric Rovers
task from the domain consists of rovers some of which can sample rock and soil data, while others have cameras that can take images of objectives. The goal of each problem is to sample rock and soil data as well as take images of objectives and communicate all g data to the lander. The rovers can move around a map with w waypoints and the rover is only able to communicate data to the lander from a subset of waypoints. Furthermore, rovers have a limited energy supply that is consumed with any action, but they can recharge with solar panels at certain waypoints. Thus, the problem has deadends because rovers have limited energy and could exhaust them in waypoints where they cannot recharge. The numeric component of the domain arises from modelling the energy supply of the rovers. Training problems have E [1, 10] goals while testing problems have [2, 728] goals problems
C.6 Numeric Satellite
task from the domain consists of s satellites, each carrying subset of i instruments that can take pictures of space using a subset of modes. Satellites can rotate to take pictures of d locations in space. Each satellite has a fixed amount of fuel that consumed when rotating, and a fixed amount of data capacity that is consumed when taking pictures. Thus, the problem has deadends because resources are finite and can be wasted. The goal of a Satellite problem is to take pictures of a set of locations in space with specified modes while adhering the fuel and data capacity constraints. The numeric component of the domain arises from modelling the fuel and data capacity features. Training problems have 2, 10 satellites and testing problems have [4, 98] satellites.
C.7 Numeric Spanner
A task from the domain consists of spanners scattered along one-way hallway with / locations, and n nuts at the end of the hallway that have to fixed. Each spanner can only be used to fix a single nut before it breaks. The goal of the domain is to fix all the nuts. The problem has deadends if not enough spanners are picked up before reaching the end of the hallway. The numeric component of the domain arises from modelling the number of spanners and nuts. Training problems have [1, 10] spanners while testing problems have s E [1487] spanners.
C.8 Numeric Transport
task from the domain consists of p packages spread across / locations, with number of trucks that can transport pick up and transport packages on map. Each truck has a limited capacity of packages it can carry. The goal of the problem is to transport all the packages to their target locations. The numeric component of the domain arises from modelling the capacity of the trucks. Training problems have p E [1, 7] packages while testing problems have p E [1, 194] packages.
19

---

10-2
Time (s)
10-3
10-4
|
10-5
Blocksworld
Childsnack
Ferry
Miconic
Rovers
Satellite
Spanner
Transport
Domain
Figure 8: Distributions of heuristic evaluation time for GNN and CCWL models with = on problems where both were able to solve in the given timeout. Blue box plots correspond to GNN models and red box plots correspond to CCWL models.
D GNN and CCWL heuristic evaluation times
We refer to Fig- 8 for distributions of heuristic evaluation time for GNN and CCWL models with = Times are computed by taking the total search time for each problem and dividing by number of heuristic evaluations made by the planner. We assume that the heuristic evaluation is the bottleneck of the search which confirmed with informal profiling experiments.
E
Effect of number of iterations and layers
We refer Tab. for the coverage of models with different I values.
20

---

Table Coverage and median of expansions of solved problems for each CCWL and GNN model with varying number of iterations and layers. Higher values are better for coverage, and lower values are better for expansions. The best value per domain and metric coloured. OOM denotes that the training process exceeded the memory limit.
a) A GNN cost
Coverage
Domain
Blocksworld
22
18
24
Childsnack
17
15
Ferry
60
60
60
60
60
Miconic
Rovers
21
18
11
13
13
Satellite
19
11
Spanner
90
90
90
Transport
36
34
31
31
>:
336
319
304
300
286
(b) A GNN yank
Coverage
3
Blocksworld
20
24
22
24
22
Childsacck
22
26
29
Ferry
60
60
Miconic
64
62
Rovers
14
13
Satellite
22
14
14
17
14
Spunner
Transport
36
38
33
27
28
Σ
338
320
321
321
(c) CCWLF
Covergee
Domain
3
Blocksworld
28
22
21
Childsnack
22
22
20
20
20
Ferry
Miconic
90
90
88
87
Rovers
22
19
Satellite
26
23
7
5
Spanner
90
89
Transport
37
407
330
318
(d) 14
DCWLP
Coverage
Domain
Blocksworld
19
29
Childsnack
90
20
23
Ferry
73
71
70
68
Miconic
Rovers
21
23
15
22
21
00M
00M
Spanner
90
90
89
89
89
Transport
46
29
386
445
365
341
343
21

---

NeurIPS Paper Checklist
1. Claims
Question:
Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: the abstract and introduction provide summary of the problem solving and also the main content of the paper.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
It is fine to include aspirational goals as motivation as long as it is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification:
we discuss limitations in the corresponding Limitations section.
Guidelines:
The answer NA means that the paper has limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors encouraged to create separate "Limitations" section in their paper.
The paper should point any strong assumptions and how robust the results violations of these assumptions (e.g.. independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g. if the approach was only tested few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or speech-to-text system might not be used reliably provide closed captions for online lectures because it fails handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and faimess.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't acknowledged the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory Assumptions and Proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [NA]
22

---

Justification:
we do not formalise any theoretical results in this paper, but refer to previous work for theoretical results.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental Result Reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Answer: [Yes]
Justification: we have done our best in the given space limits to describe in detail the experimental setup and algorithms used. We have also provided descriptions of the new datasets used.
Guidelines:
The answer NA means that the paper does not include experiments. If the paper includes experiments, a No answer this question will not perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
the contribution is a dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution a specific model and empirical evaluation, may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make it clear how reproduce that algorithm.
(b) If the contribution is primarily
model architecture, the paper should describe the architecture clearly and fully.
(c) the contribution is a new model (e.g., a large language model), then there should either be way access this model for reproducing the results way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reprodueibility may be tricky in some cases, in which authors are welcome to describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers have some path reproducing verifying the results.
5. Open access to data and code
23

---

Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, as described in supplemental material?
Question:
Answer: [Yes]
Justification: aim to provide open access to our code and new datasets.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://nnis.co cc/
Sablliccguuddes/ooeeSuuimsssss wwt.:/uwwmmiiioooooooooooo. for more details.
While we encourage the release of code and data, we understand that this might not be possible, "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for a new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https
guwdes/ddeeeee CooeeSumeeeeeeeeeeeeee ssi onyylicy) for more details.
The authors should provide instructions on data access and preparation, including how access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental Setting/Details
Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Question:
Answer: [Yes]
Justification: details of training and test details are given in the corresponding experimental section of the paper.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to a level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, as supplemental material.
7. Experiment Statistical Significance
Does the report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [No]
Justification; evaluation of planning tasks is also expensive to run and thus are only ran once. Informal experiments also show that training with different seeds offer minor variance in performance.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by emor bars, confi- dence intervals, or statistical significance tests, least for the experiments that support the main claims of the paper.
24

---

The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
It should be clear whether the error bar is the standard deviation or the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out range (e.g. negative еπτог rates).
If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables text.
8. Experiments Compute Resources
For each experiment, does the paper provide sufficient information the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification: we provide hardware information as well as time and memory limits
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate type of compute workers CPU or GPU, internal cluster, cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported the paper (e.g., preliminary or failed experiments that didn't make it into the paper).
9. Code Of Ethics
Question: Does the research conducted the paper conform, in every respect, with the NeurlPS Code of Ethics httts::///eurippi
Answer: [Yes]
Justification: the authors have read the NeurlPS Code of Ethics and believe that the research conforms with it.
Guidelines:
The answer NA means that the authors have not reviewed the NeurlPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., there a special consid- eration due to laws or regulations in their jurisdiction).
10. Broader Impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [NA]
Justification: our research is mainly foundational and not tied to particular applications. It is mainly focused on improving the efficiency and effectiveness of underlying algorithms.
Guidelines:
The answer NA means that there is no societal impact of the work performed.
25

---

If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification: there is no obvious method for our data (PDDL files) or models (planners) that may have high risk of misuse.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model. for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
recognize that providing effective safeguards challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification: original owners of datasets from which we create new benchmarks are properly cited and their licenses respected.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (c.g., CC-BY 4.0) should be included for each asset.
26

---

For scraped data from a particular source (e.g., website). the copyright and terms of service of that souree should be provided.
assets are released, the license, copyright information, and terms use in the package should be provided. For popular datasets, paperssi thood cww datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
If this information is not available online, the authors encouraged to reach out to the asset's 's creators.
13. New Assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes]
Justification: we will provide detailed documentation of our new datasets and planners when we release them.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/model part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset is used.
At submission time, remember anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and Research with Human Subjects
Question:
For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification: do not do crowdsourcing experiments research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According to the NeurlPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: same justification as previous question.
Guidelines:
The answer NA means that the paper does not involve crowdsoureing nor research with human subjects.
27

---

Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
28


Reviews:
[{"review_id": "UyruEwM5Ti", "reviewer": "Reviewer_YeVP", "summary": "The authors introduce a method to generate features for planning tasks that involve numerical variables. These features can then be used with machine learning to learn a heuristic function from a set of training examples. Architectures used for learning include Gaussian processes and graph neural networks. The authors also introduce a method for learning to rank states and do a search based on the ranking instead of the cost-to-go. Results show that, for benchmarks modified to include numerical variables, learning a heuristic function with Gaussian process regression and ranking performs significantly better than planners that do not make use of a numerical representation.", "questions": "Do any of these baseline planners make use of ranking?\n\nIs it possible that the main increase in performance is due to ranking and not the numerical representation and learning?\n\nOn line 152, V is comprised of G, while, on line 151, it says that G is comprised of V. Do these two Gs represent different concepts?", "limitations": "The learning approach relies on supervised learning, which assumes a planner exists that can already solve problems and may limit performance to what the existing planner can solve in a given time limit. On the other hand, research using deep reinforcement learning does not assume the existence of any solver.", "rating": 4, "confidence": 2, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "Ranking states instead of learning cost-to-go has shown promise. The paper presents a novel ranking method that can be combined with machine learning. The ranking method was significantly better than the corresponding non-ranking method. This could have broader implications for machine learning applied to planning.", "weaknesses": "The only learning approach that performed better than the baseline planners was the Gaussian process regression with ranking, while Gaussian process regression with cost-to-go performed better than three out of four baseline planners. This makes it appear as if ranking is contributing to the overall success and not the numerical representation and learning.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for the comments and questions for helping us identify where we could improve our paper’s clarity.\n\n## Weaknesses clarifications\n\n> This makes it appear as if ranking is contributing to the overall success and not the numerical representation and learning.\n\n- We would first like to clarify that ranking is a learning concept. More specifically, we defined it as an optimisation problem using a Mixed Integer Program with *training data* with the aim of finding weights for the learning models. The simple analogy is that regression is a method for finding a function that best fits the training data, where one can define it as an optimisation problem with the MSE loss on some given training data.\n\n- Secondly, we note that Gaussian process regression (GPR) and ranking does not go together. Instead, the description of the models in the review should be changed as follows:\n\n> GPR with ranking\n\nshould be changed to \"ccWL algorithm (Sec. 3.2) with ranking\"\n> GPR with cost-to-go\n\nshould be changed to \"ccWL algorithm with GPR representing cost-to-go\"\n\nPlease see the pdf of our global rebuttal for helping clarify the general pipelines of the models.\n\n- Lastly, we note that the strongest planner baselines (M(3h||3n), M-FF and h^{MRP+hj}) are representative of the state-of-the-art for numeric planning. They use additional tricks such as helpful and macro actions, multi-queue search, and building novelty heuristics. We decided to not combine our work with these techniques as they are orthogonal methods for improving performance which are also applicable to our methods. As mentioned on line 313, direct comparison of our results is only possible with h^add and h^RMP. All of our learning configurations clearly dominate them.\n\nWe will make these details clearer in the additional page with extra explanations and illustrations.\n\n## Questions\n\nQ1. The baseline planners are state of the art heuristic search planners, which do not make use of ranking or learning.\n\nQ2. Ranking is part of the learning process. Thus, without the numerical representation and learning, the models cannot employ ranking.\n\nQ3. Thanks for clarifying this conflict of notation. You are correct that the Gs represent different concepts, and we will fix this.\n\nThe $G$ and $\\mathcal{V}$ in line 151 refer to graph mentioned in line 124, while the $G$ and $V(g)$ in line 152 refer to the goals and the variables associated with the goal condition, where the $V(\\cdot)$ notation is introduced in line 66. We will use different notations and make this clearer in the final version of the paper.\n\n## Limitations. (Comparison with Reinforcement Learning)\n\nWe first like to mention that although the paper focuses on supervised learning, our methods are applicable to RL settings as well. More specifically, the graphical representation of the task (assuming a factored state representation) and the GNN model, can be applied with typical RL methodologies such as PPO and DQN. We decided to focus on supervised learning over RL because generating optimal plans from the provided training tasks generally take a matter of seconds, with a few outliers taking more than a minute.\n\nSecondly, regarding pros and cons of RL, it has been shown in various works (e.g. the NeurIPS Flatland challenge [1], see RL vs non-RL winners, and a Deepmind paper on planning and RL methods for manipulation planning [2], where RL methods struggle to solve the simplest benchmark problems) that symbolic solvers outperform RL methods in scaling whenever a model is given.\n\n\n[1] NeurIPS 2020 Flatland Winners. https://discourse.aicrowd.com/t/neurips-2020-flatland-winners/4010\n\n[2] Ken Kansky, Skanda Vaidyanath, Scott Swingle, Xinghua Lou, Miguel Lázaro-Gredilla, Dileep George:\nPushWorld: A benchmark for manipulation planning with tools and movable obstacles. CoRR abs/2301.10289 (2023)", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "wl0EQB4wzc", "reviewer": "Reviewer_R2Rg", "summary": "The paper tackles numeric planning problems by proposing two heuristics for numeric planning. The first one is based on graph kernels for graphs and addresses both continuous and categorical attributes. The second uses graph neural networks. The authors experimentally show the effectiveness of the two proposed algorithms by showing that they show better coverage compared to domain independent planners for numeric planning.", "questions": "1. Can you claim theoretically on admissibility, safe pruning, or tractability of the proposed algorithms? I find it strange that the authors refer to prior work re theory. At least a proof sketch can be given which refers to prior work. \n2. Related to 1, can you prove the complexity on the WL algorithm (right before section 4 starts). \n3. Can you please explain with an example how one should read Table 1. \n4. Which planner is h^LMCUT used in. h^LMCUT is a heuristics, not a planner, right? Same with the other heuristics mentioned in the figure, not sure if it is accurate to refer to them as planners. \n5. In section 8, you mention in the last sentence, one can learn forms of domain knowledge different from heuristic functions, can you give an example.", "limitations": "The paper covers that.", "rating": 7, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- The paper shows experimentally that their proposed learned heuristics shows better coverage, and that is significant. \n- The approach is novel and paper is also novel (except for the recent related work which the authors cover in the intro)\n- The authors do a decent job with explaining the literature and citing appropriately. \n- The use of classical machine learning due to being cheap and interpretable", "weaknesses": "The paper presentation can be improved.  Examples below (no particular order):\n\n1. The abstract states “… in comparison to domain-independent planners” whereas it is more informative to state domain-independent numeric planners. (Other places did indicate numeric planners).\n2. The paper can be dense in various places (section 5 for example) \n3. Sometimes the authors miss details that can be helpful in understanding the paper.  For example, what is M(3h||3n) can you explain? Maybe it would be good to give a bit of explanations on each of the configurations.\n4. Fig 1, why x, y, z are not shown as blocks, the figure was confusing at first specially given that there are dotted block spaces above y and z with only 2 available blocks whereas the limit is 3 blocks. \n5. In the Table2 caption, mention which ones are your proposed planners and the two variations (rank/cost). \n6. minor typo: “Requires requires” repeated in section 3.1", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for the suggestions and questions for helping clarify the paper.\n\n## Weaknesses\n\nWe agree that the paper presentation can be improved and found it challenging to fit all of the material in the page limit. We will make good use of an extra page to address the presentation issues mentioned with more detail and illustrations. \n\nThe dotted boxes in Fig. 1 represent the location of the other blocks if the optimal plan was followed.\n\n## Questions\n\nQ1. Learning methods cannot guarantee any admissibility of heuristics. The learned heuristics are safe because predicted values are never infinite. The proposed learning algorithms are tractable because they make use of polynomial architectures. The only exception is the training criteria which may be intractable (e.g. solving a MIP optimally) but we can specify a timeout. The underlying search algorithms which we employ with our proposed learning algorithms are in the worst case intractable because (numeric) planning is in general intractable.\n\nQ2. The complexity of the original WL algorithm is given in the original WL paper and our paragraph before Sec. 4 extends this proof to show the complexity of our ccWL algorithm is the same as that of the original WL algorithm.\n\nQ3. The purpose of the table is to give an idea of how much larger / more difficult the testing problems are, in comparison with the training problems. As an example when reading the first row: The training problems for Blocksworld have between 2-11 blocks, while the testing problems have between 5 and 488 blocks. Furthermore, the optimal plan lengths (number of actions) for the training problems range from 2-34, while planners return between plans with lengths between 8-662. \n\nQ4. You are correct that a heuristic itself is not a planner. However, the “heuristics” in Table 2 are shorthand for the configurations of planners described in Sec. 6.2 as the notation is cumbersome if the planners were also mentioned. For example h^{mrp}+hj and M(3h||3n) are both different configurations of the ENHSP planner. As mentioned in lines 287-288, h^LMCUT is used in Numeric Fast Downward.\n\nWe will make this clear in the figure and table captions.\n\nQ5. One simple example is to learn an action policy instead of a heuristic. We further list below several examples and related work in planning where we can learn domain knowledge in forms different from heuristic functions. All these methods can be used with our ccWL features (Sec. 3.2.) since they are agnostic to the downstream ML task. \n\n- policy rules [1], which are implication statements of the form (Condition($s$) -> Effect($s, s’$)) which tells us that an action $a$ should be applied in state $s$, leading to another state $s’$, if Condition($s$) and Effect($s, s’$) holds. Both the condition and effect can be some learned funcion over state features, such as those generated by the ccWL algorithm (Sec. 3.2)\n- policy sketches [2], a generation of policy rules by viewing the learned (Condition($s$) -> Effect($s, s’$)) statements as subgoals rather than direct action policies\n- portfolios [3] which learn what planner configurations work best for a given domain\n- task transformations such as learning to partially ground problems [4] or ignore objects [5]\n\n[1] Guillem Francès, Blai Bonet, Hector Geffner: Learning General Planning Policies from Small Examples Without Supervision. AAAI 2021: 11801-11808\n\n[2] Dominik Drexler, Jendrik Seipp, Hector Geffner: Learning Sketches for Decomposing Planning Problems into Subproblems of Bounded Width. ICAPS 2022: 62-70\n\n[3] Tengfei Ma, Patrick Ferber, Siyu Huo, Jie Chen, Michael Katz: Online Planner Selection with Graph Neural Networks and Adaptive Scheduling. AAAI 2020: 5077-5084\n\n[4] Daniel Gnad, Álvaro Torralba, Martín Ariel Domínguez, Carlos Areces, Facundo Bustos: Learning How to Ground a Plan - Partial Grounding in Classical Planning. AAAI 2019: 7602-7609\n\n[5] Tom Silver, Rohan Chitnis, Aidan Curtis, Joshua B. Tenenbaum, Tomás Lozano-Pérez, Leslie Pack Kaelbling: Planning with Learned Object Importance in Large Problem Instances using Graph Neural Networks. AAAI 2021: 11962-11971", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "DoN70J0c1C", "reviewer": "Reviewer_99rS", "summary": "The paper proposes a new method for learning a heuristic function to guide search for solving numeric planning problems. In contrast to classical planning, the states in numeric planning may involve numeric variables while the state transitions are defined by mathematical expressions over these kinds of variables. In addition, numeric planning is computationally quite challenging to solve but it may be the right formulation in many interesting real-world applications. The proposed heuristic is learned from training data consisting of example planning instances and their corresponding optimal plans. Features are automatically extracted from a graphical representation of the planning instances and used subsequently by the machine learning models encoding the heuristic functions. The experimental evaluation is carried out on standard benchmarks for numeric planning. The results show clearly that state-of-the-art search algorithms guided by the proposed heuristics improve considerably over their competitors.", "questions": "- The paper claims that grounding is not required for building the graphical representation of the planning problem. However, the graph shown in Figure seems to be partially grounded. So, how much grounding is needed to build such graphs?", "limitations": "The limitations are addressed clearly in the paper.", "rating": 5, "confidence": 4, "soundness": 3, "presentation": 1, "contribution": 3, "strengths": "- The paper targets an important problem in the area of automated planning and proposes more effective heuristics to guide search algorithms for solving these problems.  \n\n- The empirical evaluation is sound and covers standard benchmarks in the numeric planning domain. The results are presented in a relatively clear manner and therefore it is easy to understand the benefits of the proposed approach compared to existing state-of-the-art.", "weaknesses": "- In my opinion, the presentation is the main weakness of the paper. While I believe it is a solid contribution for numeric planning, the way it is currently presented makes it hard to understand the details.\n\n- The examples currently supporting sections 3.1 and 3.2 need to be expanded a little more. Right now, it is kind of difficult to follow them.\n\n- Section 4 which describes the graph neural network is too high level and without a supporting example is very hard to understand.\n\n- There is currently a relatively big disconnect between Sections 3, 4 and 5. Namely it takes a while to figure out that the features extracted by the proposed method are subsequently used to learn the heuristics. The current notation doesn't help much either. I also think that adding illustrative examples in Section 5 would clearly improve the quality of the presentation.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for the comments and suggestions identifying which parts of the paper can be made clearer.\n\n## Weaknesses\n\nWe agree that the paper presentation can be improved and found it challenging to fit all of the material in the page limit. We will make good use of an extra page to address the presentation issues mentioned with more detail and illustrations. Please see the figure in the pdf of our global rebuttal for helping clarify some of our proposed methods as pointed in the final two points of your review.\n\n\n## Question about Grounding\n\nThe short answer to your question is that we do *no* grounding when encoding what is given to us, i.e. a lifted planning task (lines 84-85), to a graph. More specifically, the grounded items you see (green, yellow, and red nodes) in Figure 1 are the minimal amount of information required to define a planning task. This information includes the current state, and the goal condition.\n\nThe choice of nomenclature of a “lifted planning task” is because our graphs need the first-order representation of the task, as opposed to the flattened representation, in order to be able to identify the object (blue nodes) and link them via edges to the predicates and functions that use them as arguments in the initial state and the goal.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "xRpMQF27WL", "reviewer": "Reviewer_bZ6v", "summary": "This paper proposed new learning-based methods for numeric planning. Numeric planning is formalized with the PDDL language. The proposed approaches are based on graph neural networks, and are evaluated in a lot of domains, e.g., blockworld, childsnack.", "questions": "Are the proposed two approaches suitable for different scenarios? Can the authors analyze this issue?", "limitations": "This paper has discussed the potential limitations.", "rating": 3, "confidence": 3, "soundness": 2, "presentation": 1, "contribution": 2, "strengths": "The experiment section seems solid, and the proposed approaches are evaluated in lots of domains. The experiment results demonstrate significant improvement over the compared baseline.", "weaknesses": "As there is only one closely related work on learning for numeric planning, it is hard to assess whether this is an important research problem.\n\nThe abstract has not summarized the novelty of this work, and the relationship between the two proposed approaches.\n\nThere are some missing related works about learning heuristics: evolution of Heuristics github.com/FeiLiu36/EoH.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for the comments and suggestion for related work.\n\n## Questions. (Applicability to Different Scenarios)\nThe proposed approaches are suitable for various different scenarios such as reinforcement learning (RL) and more general graph representation learning (GRL) problems. \n\n- RL\nAlthough we focused on the supervised learning setting in the paper, the proposed methods, namely the graphical representation of relational planning states and feature extraction algorithms, can be applied in the RL setting where a factored representation of states are given. More specifically, the graphical representation of the task and the GNN model/or simply a neural network operating on the features generated from the ccWL algorithm, can be applied with typical RL methodologies such as PPO and DQN. Furthermore, a model for the actions is not even needed and a simulator can even suffice as our methods only assume a symbolic representation of states and nothing about actions. \n\n- GRL\nWe have also proposed a new graph kernel (ccWL kernel in Sec. 3.2) which can handle graphs with both continuous and categorical attributes. As mentioned in the paper, there are few graph kernels which handle non-categorical attributes and the ones that do are more statistical in nature. Our ccWL kernel is a new graph kernel that may be well suited for GRL tasks that require reasoning over logic such as knowledge graphs, as opposed to statistical reasoning as in common GRL benchmarks such as molecular datasets.\n\n## Weaknesses\nW1: You question the importance of research on learning for numeric planning on the basis of the fact that there is only one other published work on this problem (published in 2024). However, the reason for there being few approaches so far, is not that the research problem is unimportant, but that it is *new* and *difficult*. There are a lot of approaches for numeric planning (including the baselines we use), and planning competitions for numeric planning being run regularly since 2002. However these works do not use machine learning at all. \n\nAccording to Turing Award winner Yann Le Cun, “learning to plan complex action sequences” is one of the top 3 challenges for deep learning  – see for instance his invited talks at AAAI 2020 (47:20 on the youtube video) and AAAI 2024 on “Objective-Driven AI: Towards AI systems that can learn, remember, reason, and plan”. The first works using modern deep learning techniques that obtained some success with classical (non-numeric) planning date back from 2018, and so are relatively recent. Numeric planning is the natural next step in this important line of work. As we show, learning has the potential to improve the state of the art in numeric planning. \n\nW2: We agree that the abstract can be improved and will do the suggested changes. Regarding the novelty of this work, this is the first work that learns heuristics for numeric planning – and one of the works in learning for planning that shows a clear improvement over the state-of-the-art. The current work on learning heuristics for planning is centered on classical planning where variables are Boolean.\n\nThe two proposed approaches (Graph Neural Networks and Graph Kernels) are the two common classes of ML methods that operate on graph data. They have their own respective advantages and disadvantages akin to those of deep learning vs classical machine learning. More specifically, GNNs are more well suited at handling large data and extracting latent features via backprop for structured data, while graph kernels benefit more from small training sizes and are much more lightweight in terms of parameters. We proposed using both approaches to be comprehensive in our evaluation on our inputs transformed into graphs. We will make this relationship and motivation for using both approaches clear in our abstract.\n\n\nW3: The main common point between the work of Fei Liu and ours is the use of the word “heuristic” – yet with a different meaning.  In planning and heuristic search, the meaning of the word “heuristic” is very precise:  a heuristic is a function taking as input a state, and returning an estimate of the cost to reach the goal from this state.  The meaning of the word “heuristic” is different in other areas, such as optimisation, where it means “approximate decision”, “rule of thumb”, or “strategy”. The paper you mention is about meta-heuristic optimisation, which is a branch of optimisation where one seeks to design general-purpose strategies to guide the search towards an approximate solution. This is only very loosely related to our work. There is an abundance of recent work on learning branching heuristics for mixed-integer programming, and other types of optimisation solvers, which again target very different types of problems. We are happy to mention them in the final version, taking advantage of the extra page available.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:
We thank all reviewers for their reviews and suggestions for improving our paper. 

We noticed that the common weakness pointed out by reviewers is that our paper could make use of additional details or illustrations to better explain our methods. We agree and believe it was difficult to fit this in the page limit for the submission. We will make use of the extra page available for the final version to handle their suggestions.

Furthermore, we have attached a figure in the global PDF to visually summarise our proposed approaches and showing 
1. how the separate sections and equations tie together into the final products, and 
2. indicating where learning is involved.

Meta-review: Overall, the technical contributions are solid and the evaluation shows strong results compared to state-of-the-art planners. 

A common theme of the reviews was that the presentation is dense and can be improved. The authors have been receptive to this feedback and appear committed to improving the presentation. 

There were also concerns about the relevance of the problem and the limited related work to compare to. These are definitely valid concerns when we see an uncommon problem being studied. In this case, the ACs assessment is that the problem of "symbolic planning for numeric planning problems" is a well-established area in a less populated corner of the AI world (e.g. the ICAPS community). It could be viewed as a positive step to bring this problem to the NeurIPS community. It is hard to predict the impact, but I can imagine a number of directions other researchers might advance the area.
Decision: Accept (poster)