Robust Goal Recognition with Operator-Counting Heuristics
Abstract
Goal recognition is the problem of inferring the correct goal towards which an agent executes a plan, given a set of goal hypotheses, a domain model, and a (possibly noisy) sam- ple of the plan being executed.
This is a key problem in both cooperative and competitive agent interactions and re- cent approaches have produced fast and accurate goal recognition algorithms.
In this paper, we leverage advances in operator-counting heuristics computed using linear programs over constraints derived from classical planning problems to solve goal recognition problems.
Our approach uses additional operator-counting constraints derived from the obser- vations to efﬁciently infer the correct goal, and serves as basis for a number of further methods with additional constraints.
Introduction
Agents that act autonomously on behalf of a human user must choose goals independently of user input and generate plans to achieve such goals (Meneguzzi 2009).
When such agents have complex sets goals and require interaction with multiple agents that are not under the user’s control, the resulting plans are likely to be equally complex and non-obvious for human users to interpret (Chakraborti et al. 2018).
In such environments, the ability to accurately and quickly identify the goals and plans of all involved agents is key to provide meaningful explanation for the observed behavior.
Goal recognition is the problem of inferring one or more goals from a set of hypotheses that best account for a sequence of observations, given a ﬁxed initial state, a goal state and a behavior model of the agent under observation.
Recent approaches to goal recognition based on classical planning domains have leveraged data-structures and heuristic information used to improve planner efﬁciency to develop increasingly accurate and faster goal recognition algorithms (Mart´ın et al. 2015;
Pereira et al. 2017).
Speciﬁcally, Pereira et al.
(2017) use heuristics based on planning landmarks (Hoffmann et al.
to accurately and efﬁciently recognize goals in a wide range of domains with various degrees of observability and noise.
This approach, however, does not deal with noise explicitly, relying on the implicit necessity of landmarks in valid plans for goal hypotheses to achieve com- petitive accuracy with other methods (Sohrabi et al. 2016;
Ram´ırez and Geffner 2010), while increasing the number of recognized goals (spread).
Thus, goal recognition under partial observability (i.e., missing observations) in the presence of noisy observation is a difﬁcult problem to address while achieving both reasonable recognition time (i.e., a few seconds), high accuracy and low spread.
In this paper, we address these limitations by leveraging recent advances on operator-counting heuristics (Pommerening et al. 2014; van den Briel et al. 2007).
Operator-counting heuristics provide a unifying framework for a variety of sources of information from planning heuristics (Hoffmann et al. 2004) that provide both an estimate of the total cost of a goal from any given state and and indication of the actual operators likely to be in such plans.
This information proves to be effective at differentiating between goal hypotheses in goal recognition.
Our contributions are threefold.
First, we develop three, increasingly more accurate goal recognition approaches using operator-counting heuristics.
Second, we empirically show that these heuristics are very effective at goal recognition, overcoming existing approaches in almost all domains in terms of accuracy while diminishing the spread of recognized goals.
Such approaches are substantially more effective for noisy settings.
Third, we discuss a broad class of operator-counting heuristics for goal recognition that can use additional constraints to provide even ﬁner handling of noise and missing observations.
Background
We review the key background for the approaches we develop in this paper.
First, the recognition settings we assume for our approach follows the standard formalization of goal recognition as planningSecond, while there is substantial literature on linear programming heuristic uniﬁed on the operator-counting framework, we focus on the speciﬁc types of operator-counting constraints we actually use in our experimentation.
Planning and Goal Recognition
Planning aims to ﬁnd a sequence of actions that transforms an initial state into a goal state.
Next, we formally deﬁne each of these elements.
Deﬁnition 1 ( Predicates and State ) .
A predicate is denoted by an n-ary predicate symbol p applied to a sequence of zero or more terms ( τ 1 , τ 2 , ..., τ n ) – terms are either constants or variables.
We refer to grounded predicates that represent logical values according to some interpretation as facts, which are divided into two types: positive and negated facts, as well as constants for truth ( (cid:62) ) and falsehood ( ⊥ ).
A state
S is a ﬁnite set of positive facts f that follows the closed world assumption so that if f ∈ S , then f is true in S .
We assume a simple inference relation | = such that S |
= f iff f ∈ S , S (cid:54)| =
f
iff f (cid:54)∈ S , and S | = f 1 ∧ ...
∧ f n iff { f 1 , ..., f n } ⊆ S .
Deﬁnition 2 ( Operator and Action ) .
An operator a is represented by a triple (cid:104) name( a ), pre( a ), eff( a ) (cid:105) : name( a ) represents the description or signature of a ; pre( a ) describes the preconditions of a , a set of predicates that must exist in the current state for a to be executed; eff( a ) represents the effects of a .
These effects are divided into eff( a ) + (i.e., an add-list of positive predicates) and eff( a ) − (i.e., a delete-list of negated predicates).
An action is a ground operator instantiated over its free variables.
Deﬁnition 3 ( Planning Domain ) .
A planning domain deﬁnition Ξ is represented by a pair (cid:104) Σ , A(cid:105) , which speciﬁes the knowledge of the domain, and consists of a ﬁnite set of facts Σ (e.g., environment properties) and a ﬁnite set of actions
A .
Deﬁnition 4 ( Planning Instance ) .
A planning instance Π is represented by a triple (cid:104) Ξ , I , G (cid:105) , where Ξ = (cid:104) Σ , A(cid:105) is the domain deﬁnition
; I ⊆ Σ is the initial state speciﬁcation, which is deﬁned by specifying values for all facts in the initial state; and G ⊆ Σ is the goal state speciﬁcation, which represents a desired state to be achieved.
Deﬁnition 5 ( Plan ) .
An s -plan π for a planning instance
Π = (cid:104) Ξ , I , G (cid:105) is a sequence of actions (cid:104) a 1 , a 2 , ..., a n (cid:105) that modiﬁes a state s into a state S | =
G in which the goal state
G holds by the successive execution of actions in π starting from s .
An I -plan is just called a plan.
A plan π ∗ with length | π ∗ | is optimal if there exists no other plan π (cid:48) for Π such that | π (cid:48) | < | π ∗ | .
A goal recognition problem aims to select the correct goal of an agent among a set of possible goals using as evidence a sequence of observations.
These observations might be actions executed by the agent or noise observation which are part of a valid plan but are not executed by the agent.
Deﬁnition 6 ( Observation Sequence ) .
An observation sequence O = (cid:104) o 1 , o 2 , ..., o n (cid:105) is said to be satisﬁed by a plan π = (cid:104) a 1 , a 2 , ..., a m (cid:105) , if there is a monotonic function f that maps the observation indices j = 1 , ..., n into action indices i = 1 , ..., n , such that a f ( j ) =
o j .
Deﬁnition 7 ( Goal Recognition Problem ) .
A goal recognition problem is
a tuple T GR = (cid:104) Ξ , I , G , O (cid:105) , where: Ξ = (cid:104) Σ , A(cid:105) is a planning domain deﬁnition; I is the initial state;
G is the set of possible goals, which include a correct hidden goal G ∗ (i.e., G ∗ ∈ G ); and O = (cid:104) o 1 , o 2 , ..., o n (cid:105) is an observation sequence of executed actions, with each observation o
i ∈ A , and the corresponding action being part of a valid plan π (from Deﬁnition 5) that transitions I into G ∗ through the sequential execution of actions in π .
Deﬁnition 8 ( Solution to a Goal Recognition Problem ) .
A solution to a goal recognition problem T GR = (cid:104) Ξ , I , G , O (cid:105) is a nonempty subset of the set of possible goals G ⊆
G such that ∀
G ∈
G there exists a plan π G generated from a planning instance (cid:104) Ξ , I , G (cid:105) and π G is consistent with O
. Operator-Counting Heuristics
Recent work on linear programming (LP) based heuristics has generated a number of informative and efﬁcient heuristics for optimal-cost planning (van den Briel et al. 2007;
Pommerening et al. 2014; Bonet 2013).
These heuristics rely on constraints from different sources of information that every plan π (Deﬁnition 5) must satisfy.
All operator-counting constraints contain variables of the form Y a for each operator a such that setting Y a to the number of occurrences of a in π satisﬁes the constraints.
In this paper we adopt the formalism and deﬁnitions of Pommerening et al.
for LP-based heuristics 1 .
Deﬁnition 9 ( Operator-Counting Constraints ) .
Let Π be a planning instance with operator set A and let s be one of tis states.
Let Y be a set of non-negative real-valued and integer variables, including an integer variable Y a for each operator a ∈ A along with any number of additional variables.
The variables Y a are called operator-counting variables.
If π is an s -plan, we denote the number of occurrences of operator a ∈ A in π with Y πa .
A set of linear inequalities over Y is called an operator counting constraint for s if for every s -plan there exists a feasible solution with Y
a =
Y πa for all a ∈ A .
A constraint set for s is a set of operatorcounting constraints for s where the only common variables between constraints are the operator-counting constraints.
Deﬁnition 10 ( Operator-Counting Integer-Linear Program ) .
The operator-counting integer program IP C for constraint set C aims to minimise (cid:88) a ∈A cost ( a ) Y a subject to C where operator-counting linear program LP C is the LPrelaxation of IP C .
Deﬁnition 11 ( IP and LP Heuristic ) .
Let Π be a planning instance, and let C be a function that maps states s of Π to constraint sets for s .
The IP heuristic h IP C ( s ) is the objective value of the integer program IP C ( s ) .
The LP heuristic h LP C ( s ) is the objective value of the linear program LP C ( s ) .
Infeasible IPs/LPs are treated as having ∞ objective value.
While the framework from Pommerening et al.
2013 uniﬁes many types of constraints for operator-counting heuristics, we rely on three types of constraints for our goal recognition approaches: landmarks, state-equations, and post-hoc optimization.
Planning landmarks consist of actions (alternatively state-formulas) that must be executed (alternatively made true) in any valid plan for a particular goal (Hoffmann et al. 2004).
Thus, landmarks are necessary conditions for all valid plans towards a goal, and, as such, provide the basis for a number of admissible heuristics (Karpas and Domshlak 2009) and as conditions to strengthen existing heuristics (Bonet 2013).
Importantly, planning landmarks form the basis for the current state-of-the-art goal recognition algorithms (Pereira et al. 2017; Pereira and Meneguzzi 2018).
Disjunctive action landmarks (Zhu and Givan 2003) for a state s are sets of actions such that at least one action in the set must be true for any s -plan, and make for a natural operator-counting constraint.
Deﬁnition 12 ( Landmark Constraints ) .
Let L ⊆
A be a disjunctive action landmark for state s of task Π .
The landmark constraint c lm s,L for L is:
[Equation]
Net change constraints generalize Bonet’s (2013) state equation heuristic, which itself relate the planning instance in question with Petri nets that represent the transitions of state variables induced by the actions, and such that tokens in this task represent net changes to the states of the problem.
Finally, Post-hoc optimization constraints (Pommerening et al.
) use the fact that certain heuristics can rule out the necessity of certain operators from plans (and thus from the heuristic estimate).
For example, Pattern Database (PDBs) heuristics (Culberson and Schaeffer 1998) create projections of the planning task into a subset of state variables (with this subset being the pattern ), such that the heuristic can partition operators into two sets of each pattern, one that changes variables in the pattern (i.e., contributes towards transitions) and the other than does not (i.e., is non-contributing).
Deﬁnition 13 ( Post-hoc Optimization Constraint ) .
Let Π be a planning task with operator set A , let h be an admissible heuristic for Π , and let N ⊆
A be a set of operators that are noncontributing in that h is still admissible in a modiﬁed planning task where the cost of all operators in N is set to 0 .
Then the post-hoc optimization constraint c PHs,h,N for h , N , and state s of Π consists of the inequality.
[Equation]
Goal Recognition using Operator Counts
We now bring together the operator-counting constraints into three operator-counting heuristics suitable for goal recognition, ranging from the simplest way to employ operator counts to compute the overlap between counts and observed actions, to modifying the constraints used by the operator counts to enforce solutions that agree with such observations,and ﬁnally accounting for possible noise by comparing heuristic values.
Computing Observation Overlap Count
We start with a basic operator-counting heuristic h ( s ) , which we deﬁne to be the LP-heuristic of Def. 11 where C com- prises the constraints generated by Landmarks (Def. 12), post-hoc optimization (Def. 13), and net change constraints as described by Pommerening et al.
This heuristic, computed following Def. 11, yields two important bits of information for our ﬁrst technique, ﬁrst, it generates the actual operator counts Y a for all a ∈ A from Def. 10, whose minimization comprises the objective function h ( s ) .
The heuristic values h ( s ) of each goal candidate G ∈ G tells us about the optimal distance between the initial state
I and G , while the operator counts indicate possible operators in a valid plan from I to G .
We can use these counts to account for the observations O by computing the overlap between operators with counts greater than one and operators observed for recognition.
Algorithm 1 shows how we can use the operator counts directly in a goal recognition technique.
In order to rank the goal hypotheses we keep a dictionary of Hits (Line 2) to store the overlap, or count the times operators counts hit observed actions.
The algorithm then iterates over all goal hypotheses (Lines 3-10) computing the operator counts for each hypothesis G and comparing these counts with the actual observations (Lines 7–10).
We recognize goals by choosing the hypotheses whose operator counts hit the most observations (Line 11).
Enforcing Observation Constraints
The technique of Algorithm 1 is conceptually similar to the
Goal Completion heuristic of Pereira et al.
(2017) in that it tries to compare heuristically computed information with the observations.
However, this initial approach has a number of shortcomings in relation to their technique.
First, while the landmarks themselves are enforced by the LP used to compute the operator counts (and thus observations that correspond to landmarks count as hits), the overlap computation loses the ordering of the landmarks that the Goal Completion heuristic uses to account for missing observations.
Second, a solution to a set of operator-constraints, i.e., a set of operators with non-negative counts may not be a feasible plan for a planning instance.
Thus, these counts may not correspond to the plan that generated the observations.
While operator-counting heuristics on their own are fast and informative enough to help guide search when dealing with millions of nodes, goal recognition problems often re- quire the disambiguation of a dozen or less goal hypotheses.
Such goal hypotheses are often very similar so that the operator-counting heuristic value (i.e., the objective function over the operator counts) for each goal hypothesis is very similar, especially if the goals are more or less equidistant from the initial state.
Thus, we reﬁne the technique of Observation Overlap by introducing additional constraints into the LP used to compute operator counts.
Speciﬁcally, we force the operator counting heuristic to only consider operator counts that include every single observation o ∈ O .
The resulting LP heuristic (which we call h C ) then minimizes the cost of the operator counts for plans that necessarily agree with all observations.
We summarize this Observation Constraint Enforcement approach in Algorithm 2.
This technique is similar to that of Algorithm 1 in that it iterates over all goals computing a heuristic value.
However, instead of computing observation hits by looking at individual counts, it generates the constraints for the operator-counting heuristic (Line 3) and adds constraints to ensure that the count of the operators corresponding to each observation is greater than one (Lines 4–5).
Finally, we choose the goal hypotheses that minimize the operator count heuristic distance from the initial state (Line 8).
Enforcement Delta
Although enforcing constraints to ensure that the LP heuristic computes only plans that do contain all observations helps us overcome the limitations of computing the overlap of the operator counts, this approach has a major shortcoming: it considers all observations as valid operators generated by the observed agent.
Therefore, the heuristic resulting from the minimization of the LP might overestimate the actual length of the plan for the goal hypothesis due to noise.
This may happen for one of two reasons: either the noise is simply a sub-optimal operator in a valid plan, or it is an operator that is completely unrelated to the plan that generated the observations.
In both cases, the resulting heuristic value may prevent the algorithm from selecting the actual goal from among the goal hypotheses.
This overestimation, however, has an important property in relation to the basic operator counting heuristic, which is that h C always dominates the operator counting heuristic h , in Proposition 1. H
11: return
Proposition 1 ( h C dominates h ) .
Let h be the operatorcounting heuristic from Defs.
10-11, h C be the overconstrained heuristic that accounts for all observations o ∈
O , and s a state of Π .
Then h C ( s ) ≥ h ( s ) .
Proof.
Let C h be set of constraints used in h ( s ) , and C h C be set of constraint used to compute h C ( s ) .
Every feasible solution to C h C is a solution to C h .
This is because to generate C h C we only add constraints to C h .
Thus, a solution to
C h C has to satisfy all constraints in C h .
Therefore, since we are solving a minimization problem the value of the solution for C h cannot be larger than the solution to C h C .
The intuition here is that the operator-counting heuristic h estimates the total cost of any optimal plan, regardless of the observations, while h C estimates a plan following all observations, including noise, if any.
If there is no noise, the sum of the counts must agree (even if the counts are different), whereas if there is noise and assuming the noise is evenly distributed, there will be differences in all counts.
Thus, our last approach consists of computing the difference between h C and h , and infer that the goal hypothesis for which these values are closer must be the correct goal.
We call the resulting heuristic h δ and formalize this approach in Algorithm 3.
Here we compute the LP twice, once with only the basic operator-counting constraints (Line 4), and once with the constraints enforcing the observations in the operator counts (Line 8), using these two values to compute h δ (Line 10).
The algorithm then returns goal hypotheses that minimize h δ (Line 11).
Experiments and Results
To evaluate the effectiveness of our approaches, we implemented each of the algorithms described earlier and performed the goal recognition process over the large dataset introduced by Pereira et al.
This dataset contains thousands of problems for goal and plan recognition under varying levels of observability for a number of traditional IPC domains (Vallati et al. 2018), including B LOCKSW ORLD , C AMPUS , D EPOTS , D RIVERLOG , Dockworker robots (DWR), IPC-G RID , F ERRY , Intrusion Detection
(I NTRUSION ), K ITCHEN , L OGISTICS , M ICONIC , R OVER ,
S ATELLITE , S OKOBAN , and Zeno Travel (Z ENO ).
It also contains over a thousand problems under partial observability and noisy observations in the C AMPUS , IPC-G RID , I NTRUSION and K ITCHEN domains.
The baselines of our experimentation were the original deterministic approach from
Ram´ırez and Geffner (2009) (R&G 2009) and the recent algorithms from Pereira et al.
(2017) ( h uniq ) and Mart´ın et al.
(2015) (FG2015) 2 .
We implemented our approaches using P YTHON 2.7 for the main recognition algorithms with external calls to a customized version of the F ASTD OWNWARD (Helmert 2006) planning system to compute the operator counts.
Our customized planner returns not only the operator counts and can also introduce additional constraints before running the CPLEX 128 optimization system.
We ran experiments in a single core of a 24 core
Intel R (cid:13)
Xeon R (cid:13) CPU E5-2620 @2.00Ghz with 48GB of
RAM, with a 2-minute time limit and a 2GB memory limit.
Table 1 shows the results for the partially observable, nonnoisy fragment of the dataset, whereas Table 2 shows the noisy fragment of the dataset 3 .
For the noisy experiments, each set of observations contained at least two spurious actions, which, while valid for the plan, were not actually executed by the agent being observed.
These results show that, while not nearly as fast as the h uniq approach from Pereira et al.
with a θ = 0 recognition threshold, the accuracy (Acc %) of our h δ approach is either competitive or superior in virtually all domains (except for some levels of observability in IPC-G RID , DWR and K ITCHEN ), and, even for the domains where the accuracy is similar, or lower, the spread ( Sin G ) of the resulting goals is consistently lower, i.e., the returned goals are unique for most problems.
The accuracy of our approach, thus, consistently matches or surpasses that of R&G 2009, with a computational cost that is also often smaller than FG 2015.
Importantly, the cost of all of our approaches is basically the same within each domain, regardless of the level of observability and noise, since our technique relies on a single call to a planner that computes the operator counts for a single state and then stops the planner.
We argue that this is attributable to our inefﬁcient implementation rather than the technique, for the h δ approach, the overhead of the F AST -D OWNWARD pre-processing step is paid multiple times.
Unlike R&G 2009, that uses a modiﬁed planning heuristic, and FG 2015, that builds a data structure and explores it at very high computational cost.
We note that the results for noisy observations show the greatest impact of h δ with an overall higher accuracy and lower spread across all domains but K ITCHEN .
Finally, results for the K ITCHEN domain stand out in our experiments in that our some of our approaches consistently show underwhelming performance both in noisy and nonnoisy domains.
Counter-intuitively, for this particular do- Sohrabi main, the more observations we have available, the worse the performance.
This seems to be a problem for all other approaches under noisy conditions, though not under incomplete observations.
Moreover, since the loss of accuracy with fuller observability also occurs for the non-noisy setting, we surmise this to stem from the domain itself, rather than the algorithm’s ability to handle noise, and defer investigation of this issue to future work.
Related Work
Our work follows the traditional of goal and plan recognition as planning algorithms as deﬁned by Ram´ırez and
Geffner (2009; 2010).
The former work yields higher recognition accuracy in our settings (and hence we chose it as a baseline), whereas the latter models goal recognition as a problem of estimating the probability of a goal given the observations.
Such work uses a Bayesian framework to compute the probability of goals given observations by computing the probability of generating a plan given a goal, which they accomplish by running a planner multiple times to estimate the probability of the plans that either comply or not with the observations.
Recent research on goal recognition has yielded a number of approaches to deal with partial observability and noisy observations, of which we single out three key contributions.
First, Mart ´ ın et al. (2015) developed a goal recognition approach based on constructing a planning graph and propagating operator costs and the interaction among operators to provide an estimate of the probabilities of each goal hypothesis.
While their approach provides probabilistic estimates for each goal, its precision in inferring the topmost goals is consistently lower than ours, often ranking multiple goals with equal probabilities (i.e., having a large spread).
Second, Sohrabi et al.
(2016) developed an approach that also provides a probabilistic interpretation and explicitly deals with noisy observations.
Their approach works through a compilation of the recognition problem into a planning problem that is processed by a planner that computes a number of approximately optimal plans to compute goal probabilities under R&G’s Bayesian framework.
Finally, Pereira et al.
(2017) develop heuristic goal recognition approaches using landmark information.
This approach is conceptually closer to ours in that we also compute heuristics, but we aim to overcome the potential sparsity of landmarks in each domain by using operator-count information, as well as explicitly handle noise by introducing additional constraints in heuristic h C and comparing the distance to the unconstrained h heuristic.
Conclusion and Discussion
We developed a novel class goal recognition technique based on operator-counting heuristics from classical planning (Pommerening et al. 2014) which, themselves rely on
ILP constraints to estimate which operators occur in valid optimal plans towards a goal.
The resulting approaches are competitive with the state of the art in terms of high accuracy and low false positive rate (i.e., the spread of returned goals), at a moderate computational cost.
We show empirically that the overall accuracy of our best approach is sub-
[Table] [Table]
[Table] [Table]
M
[Table] [Table]
[Table] [Table]
[Table] [Table]
[Table]
P C - G
D ( 300 )
[Table]
I N
[Table]
I
[Table]
stantially superior to the state-of-the-art over a large dataset.
Importantly, the values of the operator-counting constraints we compute for each of the heuristics can be used as explanations for recognized goals.
The techniques described in this paper use a set of simple additional constraints in the ILP formulation to achieve substantial performance, so we expect substantial future work towards further goal recognition approaches and heuristics that explore more reﬁned constraints to improve accuracy and reduce spread, as well as deriving a probabilistic approach using operator-counting information.
Examples of such work include using the constraints to force the LP to generate the counterfactual operator-counts (i.e., non-compliant with the observations) used by the R&G approach, or, given an estimate of the noise, relax the observation constraints to allow a number of observations to not be included in the resulting operator-counts.
Acknowledgements: This study was ﬁnanced in part by the Coordenac¸˜ao de Aperfeic¸oamento de Pessoal de Nivel
Superior – Brasil (CAPES) - Finance Code 001.
Felipe acknowledges support from CNPq under project numbers 407058/2018-4 and 305969/2016-1, as well as FAPERGS process number 18/2551-0000500-2.