Bounding Quality in Diverse Planning
Abstract
Diverse planning is an important problem in automated plan- ning with various real world applications.
Recently, diverse planning has seen renewed interest, with work that deﬁnes a taxonomy of computational problems with respect to both plan quality and solution diversity.
However, despite the re- cent advances in diverse planning the variety of approaches and the number of available tools for these problems are still quite limited, even nonexistent for several computational problems.
In this work, we aim to extend the portfolio of ap- proaches and tools for various computational problems in diverse planning.
To that end, we introduce a novel approach to ﬁnding solutions for three computational problems within diverse planning and present planners for these three prob- lems.
For one of these problems, our approach is the ﬁrst one that is able to provide solutions to the problem.
For another, we show that top-k and top quality planners can provide, albeit naive, solutions to the problem and we extend these plan- ners to improve the diversity of the obtained solution.
Finally, for the third problem, we show that some existing diverse planners already provide solutions to the problem.
Further, we suggest another approach and empirically show that our suggested approach compares favorably with these existing planners.
Introduction
Diverse planning is an important problem in AI Planning with many practical applications that require generating multiple plans rather than one.
Example applications include automated machine learning (Mohr, Wever, and H¨ullermeier 2018), risk management (Sohrabi et al. 2018), automated analysis of streaming data (Riabov et al. 2015), and malware detection (Boddy et al. 2005).
Diverse planning is also important in the context of re-planning and plan monitoring (Fox et al. 2006), under-speciﬁed user preferences (Myers and Lee 1999; Nguyen et al. 2012), as well as plan recognition and its related applications (Sohrabi, Riabov, and Udrea 2016).
In all these applications it is important to generate multiple diverse plans, and it is of equal importance to be able to control solution quality.
Most diverse planners developed over the last decade are focused on addressing a particular diversity metric.
For example, while the diverse planner DLAMA focuses on ﬁnding a set of plans by considering a landmark-based diversity measure (Bryce 2014), other diverse planners such as LPGd, DIV, DFAA/DFAM, and A ∗ AA/A ∗ AM focus on ﬁnding a set of plans with a particular minimum action distance (Nguyen et al. 2012; Coman and Mu˜noz-Avila 2011;
Vadlamudi and Kambhampati 2016).
Goldman and Kuter (2015) propose a diversity metric based on information retrieval literature.
Roberts, Howe, and Ray (2014) suggest another diversity metric, introducing several planners, such as itA ∗ and MQA, which, in addition to the diversity metrics, consider plan quality.
While all these planners implement the chosen diversity metric and switching to another metric is not trivial, the planners DFAA/DFAM and A ∗ AA/A ∗ AM work in two phases: ﬁnding a set of plans and choosing a proper subset from the found set.
That is, selection of a set of plans is independent of the diversity metric.
Recently, planners FI -diverse were introduced (Katz and Sohrabi 2020).
These planners also separate the phase of ﬁnding candidate plans from choosing a diverse subset of these plans.
Further, the authors provide a tool for selecting a subset of plans for a variety of metrics and computational problems (Katz and
Sohrabi 2019).
Another important recent contribution introduced a taxonomy of computational problems and classiﬁed existing planners according to the problems they tackle (Katz and Sohrabi 2020).
Most existing planners, according to that taxonomy, solve Satisﬁcing Diverse Planning (sat-k), where any sufﬁciently large set of plans is a solution, and the aim is to improve solution diversity.
The planners LPGd (Nguyen et al. 2012) and bFI (Katz and Sohrabi 2020) tackle Bounded Diversity Diverse Planning (bD-k), where a set of plans is a solution only if its diversity is above a certain speciﬁed bound.
Top-k planners (Katz et al.
2018b;
Speck, Mattm¨uller, and Nebel 2020) and top-quality planners (Katz, Sohrabi, and Udrea 2020), while usually are not considered as diverse planners, according to the aforementioned taxonomy return, albeit naive, solutions to the
Bounded Quality Diverse Planning (bQ-k) problem, where plan set quality is required to be above a speciﬁed bound.
The planners DFAA/DFAM and A ∗ AA/A ∗ AM (Vadlamudi and Kambhampati 2016) tackle Bounded Quality and Diversity Diverse Planning (bQbD-k), where both the quality and the diversity of plan sets is bounded from above.
Despite these recent advances in diverse planning, the pool of existing tools is still quite limited.
The planners
DFAA/DFAM and A ∗ AA/A ∗ AM by Vadlamudi and Kambhampati (2016) are the only existing planners for bQbD-k.
Top-quality planners (Katz, Sohrabi, and Udrea 2020), although technically solving bQ-k, do not aim at improving the diversity of the solution.
No planners exist for other computational problems, such as Optimal Diversity Bounded
Quality Diverse Planning (bQoptD-k), where a solution corresponds to a set of plans of best diversity among the sets of bounded quality.
In this work, expand the pool of available planners for diverse planning. We introduce novel planners for the three aforementioned computational problems, bQ-k, bQbD-k, and bQoptD-k, exploiting the recently introduced topquality planners.
To that end, we introduce a novel quality metric that reﬂects bounded plan costs, making a connection between the costs of plans in a set and quality of a set of plans.
Focusing on the most popular diversity metric (Nguyen et al.
2012; Coman and Mu ˜ noz-Avila 2011; Vadlamudi and Kambhampati 2016), for all three planners we generate a subset of all plans of bounded quality as a ﬁrst step.
In the second step, we select a subset of plans found in the ﬁrst step that constitutes a solution to the respective computational problem.
For bQ-k, as any sufﬁciently large subset of plans is, albeit naive, a solutions to the problem, we extend these planners by using a previously suggested greedy algorithm to choose a subset of plans of higher diversity (Katz and Sohrabi 2020).
For bQbD-k, we show that the decision problem that corresponds to the second step is NP-complete and suggest using previously proposed integer linear programming formulation.
As the formulation was not previously formally described, we describe it formally and prove that it provides us with a solution to bQbD-k. For bQoptD-k, as the optimization problem in second step is NPhard, we propose using a novel mixed integer linear program to solve it.
We formally describe the program and prove that a solution to the program can be used for solving bQoptD-k.
Our approach is the ﬁrst one that is able to provide solutions to the problem.
Finally, we perform an empirical evaluation of our proposed planners.
For bQbD-k, we show to favorably compare to the existing planners.
For bQ-k, as no previous planners exist, we test the quality of our solution by comparing it to the quality of the optimal solution, obtained by our proposed planner for bQoptD-k, where no previous planners exist either.
We show that the greedy algorithm works very well on tested domains, producing results close to the optimum.
Our novel contributions, thus, include (i) the introduction of the new quality metric that allows us to connect between the cost of plans and quality of a set of plans, (ii) a concrete algorithmic scheme that uses top quality planners for the ﬁrst step, ﬁnding a set of plans of bounded cost, (iii) computational complexity investigation of the second step, choosing a proper subset from the found set, for various computational problems, and (iv) introduction of the new mixed integer linear program for the resulting optimization problem.
Preliminaries and Related Work
In this work we follow the notation of Katz and Sohrabi (2020).
A SAS + planning task (B¨ackstr¨om and Nebel 1995) is given by a tuple (cid:104)V , A , s 0 , s ∗ (cid:105) , with V being a set of state variables and A being a ﬁnite set of actions .
Each state variable v ∈ V has a ﬁnite domain dom ( v ) of values.
A pair (cid:104) v, ϑ (cid:105) with v ∈ V and ϑ ∈ dom ( v ) is called a fact .
A (partial) assignment to V is called a (partial) state .
Often it is convenient to view partial state p as a set of facts with (cid:104) v, ϑ (cid:105) ∈ p
if and only if p [ v ] = ϑ .
Partial state p is consistent with state s if p ⊆ s .
We denote the set of states of a planning task by S .
s 0 is the initial state , and the partial state s ∗ is the goal .
Each action a is a pair (cid:104) pre ( a ) , eﬀ ( a ) (cid:105) of partial states called preconditions and effects .
An action cost is a mapping
C :
A → R 0+ .
An action a is applicable in a state s ∈ S if and only if pre ( a ) is consistent with s .
Applying a changes the value of v to eﬀ ( a )[ v ] , if deﬁned.
The resulting state is denoted by s (cid:74) a (cid:75) .
An action sequence π = (cid:104) a 1 , . .
. , a k (cid:105) is applicable in s if there exist states s 0 , · · · , s k such that (i) s
= s , and (ii) for each 1 ≤ i ≤ k , a i is applicable in s i
−and s i
= s i − 1 (cid:74) a i (cid:75) .
We denote the state s k by s (cid:74) π (cid:75) .
π is a plan iff π is applicable in s
and s ∗ is consistent with s(cid:74)
π (cid:75) .
We denote by P (Π) (or just P when the task is clear from the context) the set of all plans of Π .
The cost of a plan π , denoted by C ( π ) is the summed cost of the actions in the plan.
Inregard to reasoning about sets of plans rather than individual plans, there are two main measures deﬁned on sets of plans, quality and diversity .
Previous work has introduced one deﬁnition of quality, mirroring the International Planning Competition (IPC) quality metric for individual plans (Katz and Sohrabi 2020).
Deﬁnition 1 Let P be the set of known plans of Π and let P (cid:48) ⊆
P be a subset of plans.
The relative quality of P (cid:48) with respect to P is deﬁned as
[Equation]
where π 1 , . .
, π | P (cid:48) | and π (cid:48) 1 , . . .
, π (cid:48)| P (cid:48) | are plans in P and P (cid:48) , respectively, sorted in ascending order of their costs.
The relative quality of a set of plans is always betweenand 1 , being
if and only if there is no plan in P \ P (cid:48) that is cheaper than any plan in P (cid:48) .
Switching now our attention to diversity metrics, pairwise plan distance is deﬁned by δ ( π, π (cid:48) )
= 1 − sim ( π, π (cid:48) ) , where sim is a similarity measure, a value between 0 (unrelated) and 1 (equivalent).
The diversity of a set of plans, D ( P ) ,
P ⊆ P is then deﬁned as some aggregation (e.g., min or average) of the pairwise distance within the set P .
In this work, we focus on one of the most popular similarity measures, stability (Fox et al.
2006; Coman and Mu˜noz-Avila 2011).
Stability similarity measures the ratio of the number of actions that appear on both plans to the total number of actions on these plans, referring to plans as action multisets (sets with repetitions).
Given two plans π , π (cid:48) , it is deﬁned as sim stability ( π, π (cid:48) )
= | A ( π ) ∩
A ( π (cid:48) )
| / | A ( π ) ∪ A ( π (cid:48) )
| , where A ( π ) is the multi-set of actions in π .
In what follows, by D ma we denote the diversity metric computed as minimum over the pairwise plan distance under stability similarity, the diversity metric implemented by multiple existing diverse planners (Nguyen et al. 2012; Coman and Mu˜nozAvila 2011; Vadlamudi and Kambhampati 2016).
There is a variety of computational problems that fall under the umbrella of diverse planning.
In our work, focusing on bounded quality problems, we follow the recently introduced taxonomy (Katz and Sohrabi 2020).
Deﬁnition 2 (Diverse planning solution) Let Π be a planning task and P be the set of all plans for Π .
Given a natural number k , P ⊆ P is a k -diverse planning solution if | P | = k or P = P if |P| < k .
Deﬁnition 3 (Quality-bounded solution) Let Π be a planning task, Q be some quality metric, c be some bound, and
P be the set of all Π ’s plans.
Given a natural number k ,
P ⊆
P is a c -quality-bounded k -diverse planning solution if it is a k -diverse planning solution and Q ( P ) ≥ c .
Bounded Quality Diverse Planning computational problem is deﬁned as follows.
bQ-k : Given k and c, ﬁnd a c -quality-bounded k -diverse planning solution.
Deﬁnition 4 (Diversity-bounded solution) Let Π be a planning task, D be some diversity metric, b be some bound, and P be the set of all Π ’s plans.
Given a natural number k ,
P ⊆
P is a b -diversity-bounded k -diverse planning solution if it is a k -diverse planning solution and D ( P ) ≥ b .
Bounded Quality and Diversity Diverse Planning computational problem is deﬁned as follows.
bQbD-k : Given k, b, and c, ﬁnd a c -quality-bounded and b -diversity-bounded k -diverse planning solution
Note that the deﬁnition above generalizes the previously suggested search problem described in Equation 1 below and implemented for the diversity metric D ma by Vadlamudi and Kambhampati (2016).
cCOSTdDISTANTkSET : ﬁnd P with P ⊆ P , | P | =
k, min π,π (cid:48) ∈ P δ ( π, π (cid:48) ) ≥ d, C ( π ) ≤ c ∀ π ∈ P.
Finally, Optimal Diversity Bounded Quality Diverse Planning optimization problem is deﬁned as follows.
bQoptD-k : Given k and c, ﬁnd a diversity-optimal among c -quality-bounded k -diverse planning solutions.
Bounded Quality in Diverse Planning
As stated above, in this work, we focus on the three computational problems in diverse planning taxonomy of Katz and Sohrabi (2020) that deal with bounded quality, bQ-k, bQbD-k, and
bQoptD-k. Our proposed solutions to these three problems all have in common the ﬁrst step - ﬁnding a set of plans of bounded quality.
While existing planners for bounded quality diverse planning took the same approach (Vadlamudi and Kambhampati 2016), they used planners for the top-k planning problem (Riabov, Sohrabi, and Udrea 2014).
Speciﬁcally, A ∗ AA/A ∗ AM apply the m - A ∗ algorithm (Flerova, Marinescu, and Dechter 2016), while
DFAA/DFAM apply the well-known branch-and-bound algorithm.
We suggest using a different approach, generating plans with a planner for a recently proposed unordered topquality problem (Katz, Sohrabi, and Udrea 2020).
Switching to top-quality allows to ensure that all plans of bounded cost are found.
Unordered top-quality allows to disregard plans that are reorderings of the found plans.
For some diversity metrics, this is highly beneﬁcial, with pairwise diversity between a plan and its reordering might be very low.
Further, ignoring plan orders reduces the computational effort required for ﬁnding all plans of bounded cost.
The second step, after ﬁnding a set of plans of bounded quality, is different for the different computational problems that we consider in this work.
For bQ-k, although any set of k plans is a solution, we strive to obtain solutions of higher diversity. Therefore, we apply a greedy algorithm that iteratively increases the set of plans by adding at each step the candidate plan that increases the overall diversity score the most, the same algorithm that was used for satisﬁcing diverse planning (Katz and Sohrabi 2020).
For bQbD-k and bQoptD-k, we cast the problem of ﬁnding the subset of plans as (mixed) integer linear programs.
We describe these programs in detail in what follows.
We start with the discussion of the quality metric that we consider in this work.
Quality Metric
While we consider the planners DFAA/DFAM and
A ∗ AA/A ∗ AM to be solving the Bounded Quality and Diversity Diverse Planning problem as deﬁned by Katz and
Sohrabi (2020), the quality metric they maximize is not obvious.
These planners consider the set P of plans of cost smaller or equal than a given absolute bound value c , or max π ∈ P C ( π ) ≤ c .
Alternatively, the criterion can be expressed via Q a ( P ) ≥ c (cid:48) for the quality measure Q [Equation] ( P
[Equation]
where c ∗ is the task’s optimal plan cost and c (cid:48)
=
c ∗ c ∈ [0 , 1] .
Thus, the planners above solve the bQbD-k problem for the quality metric Q a as in Eq. 2.
Note that this quality measure is different from the measure described in Deﬁnition 1, as introduced by Katz and Sohrabi (2020), where the quality of the set of plans is affected by the costs of all plans, not only most expensive ones.
The above proposed quality metric makes it possible to connect the quality metric to the cost bound.
Bounded Quality Planning
Let us consider now the (unordered) top quality planners (Katz, Sohrabi, and Udrea 2020), that, given a multiplier q m ≥ 1 , return the set of all plans P such that ∀ π ∈ P we have C ( π ) ≤ q m × c ∗ , or a subset thereof with a single representative for plans that differ only in the order of their actions for the unordered case.
For such sets, we have
Q a ( P ) ≥ 1 q m , and therefore these planners can be used to derive solutions of bounded quality according to the quality metric Q a .
Furthermore, top quality planners produce a set of plans that is a super-set of the sets of plans that constitute solutions to all three computational problems of interest, bQ-k, bQbD-k, and bQoptD-k.
Therefore, in what follows, we will focus on ﬁnding subsets of plans out of a given set of plans, according to the relevant solution deﬁnition for the corresponding computational problem.
Focusing ﬁrst on bQ-k, while any subset of required size of the set of plans returned by top quality planners is a solution to bQ-k, different subsets can vary signiﬁcantly in their diversity measure score.
Since bQ-k does not pose any restrictions on these subsets beyond the desired size, one possible way of coming up with subsets of high diversity is to employ the same greedy selection algorithm that was used for Satisﬁcing Diverse Planning (Katz and Sohrabi 2020).
The algorithm iteratively constructs a set of plans by greedily adding a plan that will contribute the most to already added plans.
Bounding Diversity
Switching now our attention to bQbD-k, ﬁrst, note that for a set of plans and a number k , the decision problem of whether there exists a subset of bounded diversity of size k is NPcomplete.
The membership in NP is trivial.
We show the hardness by a polynomial reduction from the clique problem (Garey and Johnson 1979). For a graph G = ( V, E ) , let P G =
{ π v | v ∈ V } be a collection of plans. For a pair of plans π u , π v ∈ P d ( π u , π v ) = (cid:26) d, ( u, v ) ∈ E, 0 , otherwise . Theorem 1 Given a number k and diversity bound d >for diversity metrics minimal pairwise diversity, if there is a subset of plans P G of bounded by d diversity of size at least k , then there is a clique in G of the same size. Proof: Let P ⊆
P G be a subset of plans such that | P | ≥ k and D ( P ) ≥ d .
Then for all π u , π v ∈ P we have d ( π u , π v ) ≥ d and therefore d ( π u , π v ) > 0 .
Thus, it must be the case that ( u, v ) ∈ E for all π u , π v ∈ P and thus the set V (cid:48)
= { v | π v ∈ P } is a clique, of size | V (cid:48) | = | P | ≥ k . Next, we describe the mixed integer linear program that is used for ﬁnding a subset of plans of bounded diversity. While the program is not novel, 1 its description was not presented in the literature.
Here, we describe the program in detail.
Given a set of plans P and a bound on the diversity d , the variables are as follows.
• A binary variable v π per plan π ∈ P , describing whether the plan is selected for the subset.
The constraints are as follows.
(i) ∀ π, π (cid:48) ∈ P, s.t. d ( π, π (cid:48) )
<
d : v π + v π (cid:48) ≤ 1 , stating that if the pairwise diversity of π and π (cid:48) is below d , then at most one of these plans can be selected for the subset, and (ii) (cid:80) π ∈ P v π ≥ k , forcing the size of the subset be at least k .
The objective of the program is to minimize (cid:80) π ∈ P v π .
In words, the program encodes a subset selection and restrict the selected subset to not have pairs of plans with diversity outside of the provided bound.
In what follows, we prove that the program can be used for devising solutions for bQbD-k.
Theorem 2 For a planning task Π with a set of all plans of bounded quality P such that | P | ≥ k , and a bound d , the binary program ﬁnds a subset of size k with the bounded by d diversity score, for diversity metrics maximizing minimal pairwise diversity, if such subset exists.
Otherwise, the program is infeasible.
Proof: We ﬁrst show that a solution to bQbD-k corresponds to a feasible assignment.
Let P (cid:48)
⊂
P be a solution to bQbD-k for the bound d , with | P (cid:48) | = k .
Then, let v assignto plans π ∈ P (cid:48) and 0 otherwise.
Since | P (cid:48) | = k , constraint (ii) holds.
For π, π (cid:48) ∈ P , if either of the plans is not in P (cid:48) , then v π + v π (cid:48) ≤ 1 .
If both plans are in P (cid:48) , then d ( π, π (cid:48) ) ≥ d .
Thus, constraint set (i) holds for v and the program is feasible.
Now, let v be some feasible solution and let P (cid:48)
= { π ∈
P | v π = 1 } be the corresponding subset of P .
Then, (a) from constraint (ii) we have | P (cid:48)
| ≥ k , and (b) for all π, π (cid:48) ∈ P (cid:48) , since v π + v π (cid:48)
= 2 , we know that the corresponding constraint is not in the constraint set (i), and therefore d ( π, π (cid:48) ) ≥ d .
Therefore, P (cid:48) (or any of its subset of size k ) is a solution to bQbD-k.
We now switch our attention to the next computational problem, bQoptD-k.
Optimizing Diversity
Due to the NP-completeness of the decision problem of selecting a subset of plans of bounded diversity, the corresponding optimization problem is NP-hard.
To solve it efﬁciently, we encode it in mixed integer linear programming.
We present a novel mixed integer linear program that we use for ﬁnding a subset of size k that optimizes the diversity metric.
Given a set of plans P , we deﬁne the variables as follows.
• A binary variable v π per plan π ∈ P , describing whether the plan is selected for the subset, and • a single continuous variable d for bounding the pairwise diversity.
The constraints are as follows.
(i) (cid:80) π ∈ P v π = k , stating that the size of the subset is exactly k , and (ii)
∀ π, π (cid:48) ∈ P : d + v π + v π (cid:48) ≤
d ( π, π (cid:48) )
+2 , stating that d is bounded by the diversity of each chosen pair, if the pair is chosen. q m =1 .q m =1 .q m =1 .q m =
Coverage
DFA A ∗ A FI Sym
DFA
A ∗ A
FI
Sym
DFA
A ∗ A
FI
Sym
DFA
A ∗ A
FI
Sym airport (28)               7 barman11 (8)               4 barman14 (4)               4 blocks (30)               22 childsnack
[Table]
6)               1 data-ntwrk18 (13)               10 depot (12)               3 driverlog (14)               9 elevators08 (24)               0 elevators11 (18)               0 ﬂoortile11 (14)               5 ﬂoortile14 (20)               8 freecell (22)               18 ged14 (19)               15 grid (2)               2 gripper (20)               15 hiking14 (19)               5 logistics00 (20)               3 logistics98 (6)               2 miconic (143)               57 movie (30)               0 mprime (24)               16 mystery (17)               11 nomystery11 (17)               13 openstacks08 (30)
[Table]
[Table]
[Table]
[Table]
[Table]
[Table]
[Table]
[Table]
[Table]
[Table]
[Table]
[Table]
[Table]
[Table]
[Table]
[Table]
[Table]
[Table]
[Table]
[Table]
42 rovers (12)               6 satellite (15)               7 scanalyzer08 (19)               10 scanalyzer11 (15)               7 snake18 (11)               3 sokoban08 (30)               0 sokoban11 (20)               0 spider18 (11)               3 storage (18)               13 termes18 (16)               10 tetris14 (9)               5 tidybot11 (16)               9 tidybot14 (10)               3 tpp
[Table]
)               4 transport08 (12)               4 transport11 (8)               3 transport14 (7)               2 trucks (12)               5 visitall
[Table]
12)               5 visitall
[Table]
6)               0 woodwork08 (28)               22 woodwork11 (20)               16 zenotravel
[Table]
[Table]
[Table]
q m =1 .q m
=1 .q m
=1 .q m
=1 .k
DFA
A ∗ A
FI
Sym
DFA
A ∗ A
FI
Sym
DFA
A ∗ A
FI
Sym
DFA
A ∗ A
FI
{+Sym               
 [Table] The objective of the program is then to maximize d .+} In words, as in the previous case, the program encodes a subset selection, but in this case all subsets of size k correspond to valid assignments.
We additionally have a continuous variable d that is bounded by the diversity score of the selected subset.
In the case of diversity metrics that correspond to minimal pairwise diversity, this would mean to require the variable d to be bounded by the diversity of each selected pair.
In other words, if a pair of plans is selected, then d should be no greater than their diversity score.
If a pair is not selected, there is no such restriction, but since there is a natural upper bound of 1 on the overall diversity, d can be required to be upper bounded by any value that is larger or equal to 1 .
If at least one of v π , v π (cid:48) gets 0 assigned to it, the constraint d + v π + v π (cid:48) ≤ d ( π, π (cid:48) )
+ 2 is then satisﬁed.
Therefore, the constraint is valid whether the variables v π and v π (cid:48) are assigned 0 or 1 .
In what follows, we prove that the program can be used for devising solutions for bQbD-k.
Theorem 3 For a planning task Π with a set of all plans of bounded quality P such that | P | ≥ k , the mixed integer program ﬁnds a subset of size k with the optimal diversity score, for diversity metrics maximizing minimal pairwise diversity.
Proof: Let v, d be a feasible assignment to the variables of the mixed integer program and let P (cid:48)
= { π ∈ P | v π = 1 } be the corresponding subset of P .
Then, from the constraint set (i) we have | P (cid:48)
| = k and from constraint set (ii) we have d ≤ d ( π, π (cid:48) ) for all π, π (cid:48) ∈ P (cid:48) .
Further, for a plan π ∈ P \ P (cid:48) and a plan π (cid:48) ∈ P (cid:48)
we have d ≤ 1 + d ( π, π (cid:48) ) , which does not pose additional constraint on the values of d since all pairwise distances d ( π, π (cid:48) ) are upper-bounded by 1 .
Similarly, for π, π (cid:48) ∈ P \ P (cid:48) , we have d ≤ 2 + d ( π, π (cid:48) ) , which also does not pose additional constraint on the values of d .
Therefore we have d ≤ d ( π, π (cid:48) ) for all π, π (cid:48) ∈ P (cid:48) and maximizing d without changing v would lead to d = min π,π (cid:48) ∈ P (cid:48)
d ( π, π (cid:48) ) .
Thus, the linear program ﬁnds a subset of size k with maximum minimal pairwise diversity.
Experimental Evaluation
To empirically evaluate the feasibility of our suggested approach, we have implemented our diverse planners on top of the Diversity Score Computation component (Katz and Sohrabi 2019), using CPLEX v12.8.0 for solving the mixed integer linear programs. The code will be available upon acceptance.
The experiments were performed on Intel(R) Xeon(R) CPU E7-8837 @2.67GHz machines, with the time and memory limit of 30min and 2GB, respectively.
The benchmark set consists of all STRIPS benchmarks from optimal tracks of International Planning Competitions (IPC) 1998-2018, a total of 1797 tasks in 64 domains.
For Bounded Quality and Diversity Diverse Planning (bQbD-k), we compare to the existing planners for that computational problem DFAM and A ∗ AM (Vadlamudi and Kambhampati 2016).
Since these planners are implemented for the diversity metric D ma , we focus our experimental evaluation on D ma , although our approach works with any metric.
Further, since these planners require an absolute bound on the solution cost to be provided as a parameter, we further restrict the benchmark set to tasks where optimal costs could be found with a state-of-the-art cost-optimal planner.
For that, we used the 17 single planners from the portfolio of Delﬁ (Katz et al. 2018a).
As a result, for the bQbD-k computational problem, the benchmark set consists of 1192 tasks.
As a ﬁrst step, we generate a set of plans of bounded quality.
Focusing on D ma allows us to use unordered topquality planners (Katz, Sohrabi, and Udrea 2020) to derive all plans (modulo reorderings) of bounded cost.
This is due to the fact that two plans that differ only in the order of their actions would produce pairwise diversity of 0 and thus any set of plans P that includes two such plans would get D ma ( P ) = 0 .
For other diversity metrics we might need to produce the set of all plans of bounded cost (Katz, Sohrabi, and Udrea 2020).
Note that some top-k planners, such as K ∗ -based
(Katz et al. 2018b) and symbolic search based (Speck, Mattm¨uller, and Nebel 2020) can be easily adapted to produce solutions for top-quality planning.
Further, these two planners can be rather naively adapted to produce unordered toq-quality solutions, by performing a duplicate check and skipping plans for which a reordering was previously found. In our experiments, we have performed the ﬁrst step with each of these three planners, namely FI , K ∗ , and Sym . We run these planners with a 29min time bound, to allow at least one minute for the second step.
In all cases, the overall time bound for both steps is 30min.
Further, to avoid generating a larger amount of plans, the overall bound on the number of generated plans for the ﬁrst step is set to 10000.
As a second step, we select a subset of plans according to the computational problem of interest.
For bQ-k, we use the greedy approach suggested by Katz and Sohrabi (2020).
For the bQbD-k and bQoptD-k computational problems, we solve a mixed-integer linear program, as described in the previous section.
This results in three conﬁgurations for each computational problem of interest.
For space reasons, in what follows, we focus on the two best performing ones, FI and Sym .
Table 1 presents a domain-wise comparison of our planners, FI-bQbD and Sym-bQbD to the existing planners DFAM and A ∗ AM (Vadlamudi and Kambhampati 2016), for k = 5 .
We use the diversity bound of 0 . 15 and experiment [Figure] Quality multipliers: q m =1. q m =1.
q m =1. q m =1. [Figure] Quality multipliers
[Figure] q m =
[Figure]
[-(30) 5 5 3 31 sokoban11 (20) 3 3 1 00 spider18 (11) 5 6 8 85 storage (18) 8 118 117 1111 11 17 tetris14 (9) 1 1 2 43 tidybot11 (16) 1 4 12 125 tidybot14 (10) 4 6 6 70 tpp (11) 0 0 0 06 transport08 (12) 4 5 6 68 transport11 (8) 2 2 2 23 transport14 (7) 1 0 0 12 trucks (12) 6 5 6 78 visitall11 (12) 7 8 9 98 visitall14 (6) 6 6 6 63 woodwork08 (28) 8 710 614 612 6 13 woodwork11 (20) 7 8 9 78 zenotravel (13) 7 
 [Table] [Table] [Table]-]q m =1.
[Figure]
q m =1.
[-FI DFA A ∗ A FI DFA A ∗ A FI DFA A ∗ A FI 10 390 373 662 389 354 636 426 338 610 505 318100 189 238 512 194 216 472 229 185 437 291 123 79 
-]q m =1.
[Figure]
[-We use the diversity bound of 0 . 15 and experiment with four quality bounds, deﬁned by multipliers of the optimal plan cost, from q m = 1 . 
-]
d) q m = 1 . 05 , q m = 1 . 10 , and to q m = 1 . 20 (up to 120% of the optimal plan cost).
The results for these four quality bound multipliers are depicted in the four parts of the table.
Each part presents the coverage value for the four planners.
A planner gets a coverage of 1 on a planning task if it was able to either ﬁnd a solution of size k or prove that no such solution exists.
Otherwise, the planner gets coverage 0.
The coverage of a domain is a sum over coverages of all tasks in the domain.
Best results are highlighted in bold.
While both FI-bQbD and Sym-bQbD outperform the existing approaches in terms of overall coverage, it is worth mentioning that for each of the approaches there are multi- ple domains where that approach exhibit superior behavior.
To show how these planners scale with larger values of k , Table 2 presents aggregated overall coverage for the values of k = 10 , 100 , 1000 .
Going deeper into the coverage results, note that DFAM does not prove unsolvability.
A ∗ AM, on the other hand, for large values of k = 100 and k = 1000 , did not ﬁnd any solutions, and all the instances reported in the table for A ∗ AM and these values of k correspond to unsolvable cases.
For our suggested approach, both FI-bQbD and Sym-bQbD are able to cope with both, for any value of k .
It is worth mentioning that the performance of DFAM often improves, sometimes signiﬁcantly, with larger quality bounds.
We conjecture that this increase is due to the nature of the branch-and-bound algorithm, that does not necessarily produce plans in the order of their costs.
Switching to bQ-k and bQoptD-k, in order to evaluate the quality of the solution obtained using the greedy algorithm, we compare the diversity metric score of the subset chosen by FI-bQ (respectively, Sym-bQ) to the best possible score, obtained with the FI-bQoptD (respectively, Sym-bQoptD) planner.
Figure 1(a,b) depict the comparison for k = 5 , for all four quality multipliers 1.0, 1.05, 1.1, and 1.2, for tasks where both planners were able to ﬁnd a solution, and the ﬁrst step produced at least k + 1 plans.
Note that the greedy approach works surprisingly well.
On these tasks, in most cases the greedy algorithm has reached the optimum (nodes on the diagonal): 80 out of 121, 90 out of 126, 71 out of 105, and 50 out of 105 for FI-bQ and 106 out of 176, 99 out of 162, 67 out of 118, and 63 out of 129 for Sym-bQ (for the four quality multipliers, respectively).
When it hasn’t reached the optimum, the scores are still mostly below the y = x + 0 .
1 line.
There are only 21, 18, 17, and 26 tasks for FI-bQ and 26, 23, 20, and 30 tasks for Sym-bQ for the four quality multipliers 1.0, 1.05, 1.1, and 1.2, respectively above the y = x + 0 .
1 line, and only 21 tasks for FI-bQ and 16 tasks for Sym-bQ in total for all quality multipliers above the y = x
+ 0 .
2 line.
While our experiments show that the greedy approach often produces solutions of diversity close to optimum, the question remains how these algorithms compare in their run time. Figure 1 (c) and (d) present such run time comparison between the greedy and the optimal approaches. The greedy algorithm always ﬁnished in under 1.2 seconds, while solving mixed integer linear program takes signiﬁcantly longer on these tasks, up to 500 seconds for FI-bQoptD andseconds for Sym-bQoptD. Finally, note that an inherent limitation of our approach to solving bQoptD-k is that the ﬁrst step must produce a solution to the (unordered) top quality planning problem.
There is no such limitation when solving bQ-k. As a result, FI-bQ successfully solves bQ-k in 617, 617, 615, and 613 tasks for the four quality multipliers, while FI-bQoptD solves bQoptD-k in only 369, 333, 273, and 191 tasks.
Similarly,
Sym-bQ successfully solves bQ-k in 702, 675, 648, andtasks for the four quality multipliers, while Sym-bQoptD solves bQoptD-k in only 379, 338, 262, and 196 tasks.
Discussion and Future Work
In this work, we extend the portfolio of existing tools for various computational problems in diverse planning by introducing three new such tools.
We follow the recently introduced taxonomy and, focusing on bQbD-k, map existing planners DFAA / DFAM and A ∗ AA / A ∗ AM to that problem.
For that, we introduce a novel quality metric under which these planners can be considered to solve bQbD-k.
The metric also allows us to use top quality planners as a basis for our proposed planners, for bQbD-k as well as for other computational problems, choosing a subset of plans from the solution for the top quality problem.
We show that it is NPcomplete to ﬁnd a solution to bQbD-k, given the set of all plans of bounded cost and suggest using a previously proposed integer linear programming based approach, which is experimentally shown to favorably compete with existing planners.
As the integer linear program was not previously detailed in the literature, we present it in detail and formally prove that it can be used for solving bQbD-k. Switching from bounding to optimizing diversity, we suggest a novel mixed integer linear program and formally prove that this program solves bQoptD-k. For another computational problem, bQ-k, we use an existing greedy approach of selecting a subset of plans, and empirically show that such a simple approach is able to often achieve the optimum in practice.
Our suggested approach is similar to the one of Vadlamudi and Kambhampati (2016), in that it is also separated into two steps: (i) ﬁnding a set of plans of bounded cost, and (ii) choosing a proper subset from the found set.
There are two major differences.
The ﬁrst one is the stopping criteria for step (i): while Vadlamudi and Kambhampati (2016) iterate until enough plans are found or no more plans exist, and can stop before ﬁnding all plans of bounded cost, we are using an existing (unordered) top-quality planner as is, and therefore will produce the set of all plans of bounded cost in step (i).
While it is possible to adapt the top quality planner that we used to terminate earlier, we decided not to do so, to allow for easily replacing the top quality planner with a different one.
The second major difference is that instead of trying to construct a feasible solution during the execution of step (i), we perform step (ii) after the ﬁrst step is ﬁnished, as a post-processing.
Further, instead of implementing a dedicated algorithm, we cast the problem of choosing the proper subset as an integer linear program, allowing us to use existing solvers.
Thus, our solution is highly modular, allowing us to easily replace the solvers when better ones become available.
Whilethere has been signiﬁcant progress in the ﬁeld of diverse planning recently, there are still several interesting computational problems for which no planners currently exist.
For example, in this work we show how to optimize diversity when the set of candidate plans is given.
However, if the quality restriction is alleviated, it is not clear how to choose a set of maximal diversity.
It is not even clear whether all plans must be considered while searching for such a set.
Another possible problem of interest is ﬁnding a subset of optimal quality among the bounded diversity ones.
Focusing on these planning problems is an interesting research direction.