Venue: ICAPS 2019 Workshop HSDIP

Reviews:

final_score: rating: 8: Top 50% of accepted papers, clear accept
review_content: This paper introduces a novel approach to finding possibly large sets 
of diverse plans to a planning task with quality that is still within 
a given ratio from the optimal cost. The authors baptize their 
approach top-quality planning, finding all plans up to a fixed cost q 
and representing these plans is a very compact form.

The algorithm proposed by the paper works by repeatedly solving a 
planning task with an optimal planner, by, after each iteration, 
forbidding all permutations of the last found plan for later 
iterations. The latter is done by reformulating the planning task,
essentially counting the number of actions occurrences.

The paper introduces an interesting new problem and a nice (and 
well-working) solution to it. The theory is sound and the experimental 
evaluation shows that the proposed algorithm works well (compared to 
known approaches to somewhat different problems that can be used to 
solve the top-quality planning problem). This makes it a good 
contribution to the workshop.

- Why do you restrict the constant q to natural numbers? Why would it
not work for (positive) reals?

Minor things:
- abstract: "introduced additional*ly* restrictions"
- please remove the copyright statement from the first page
- "(e.g., (Katz et al. 2018)..)" should probably be "(e.g., Katz et al. (2018)..)"
- plans should probably be defined as starting from the initial state.
alternatively, you could define a plan for a task as starting in its
initial state.
- you should mention that the blue dashed lines in figure 1 indicate
the goals
- in the definition of structural symmetries, and also after that in
the paper, you mix notation for the domain of a variable v ({\cal D}(v) vs dom(v))
and also the variables of an assignment s ({\cal V}(s) vs vars(s))
- a symmetry that "stabilizes the initial state" is not defined
- you should show the indices of the trucks in figure 1, it is not
clear that T_2 and T_3 are the ones starting in the right
- the function "reord()" is not defined
- the bibliography does not seem to be in AAAI style
review_title: Interesting new problem introduced + good first algorithm to solve it

final_score: rating: 9: Top 15% of accepted papers, strong accept
review_content: The paper is written well, makes interesting theoretical and practical contributions and is therefore a nice fit for the workshop. I only have minor comments:

I think it would be good to name the optimal planner you use for the experiments.

I think Figure 3 would benefit from an explanation. Why is the solution encoding size so much bigger for K-tq?

I think Figure 4 shows a difference of almost 2 orders of magnitude, not "several" orders of magnitude.

The paper doesn't promise to make any code or results available. In order to allow others to build on this work, I would encourage the authors to share the code and the empirical results.

Typos:
* Avoid nesting brackets for "e.g." citations.
* a planning task--s--
* only a handful of work++s++ take the plan quality --under-- ++into++ consideration
* harder for ++a++ classical planner
review_title: The paper formally defines the problem of finding all plans up to a certain solution cost, introduces a planner for this problem and shows that it outperforms an approach using a top-k planner.

Meta-review: metareview: Dear Authors,
thank you very much for your submission. We are happy to inform you that
we have decided to accept it and we look forward to your talk in the workshop.
Please, go over the feedback in the reviews and correct or update your papers
in time for the camera ready date (May 24).
Best regards
HSDIP organizers
confidence: 5: absolutely certain
Decision: Accept