Original: Towards Explainable  Planning as a Service
Revised: Towards Explainable AI Planning as a Service

Original: Abstract
Revised: Abstract

Original: Explainable AI is an important area of research within which Explainable Planning is an emerging topic.
Revised: Explainable AI is an important area of research within which Explainable Planning is an emerging topic.

Original: In this paper, we argue that Explainable Planning can be designed as a service – that is, as a wrapper around an existing planning system that utilises the existing planner to assist in answering contrastive questions.
Revised: In this paper, we argue that Explainable Planning can be designed as a service – that is, as a wrapper around an existing planning system that utilises the existing planner to assist in answering contrastive questions.

Original: We introduce a  framework to facilitate this, along with some examples of how a planner can be used to address certain types of contrastive questions.
Revised: We introduce a prototype framework to facilitate this, along with some examples of how a planner can be used to address certain types of contrastive questions.

Original: We discuss the main advantages and limitations of such an approach and we identify open questions for Explainable Planning as a service that identify several possible research directions.
Revised: We discuss the main advantages and limitations of such an approach and we identify open questions for Explainable Planning as a ser- vice that identify several possible research directions.

Original:  Introduction
Revised:  Introduction

Original: Explainable Artiﬁcial Intelligence (XAI) is an emerging research area in AI, motivated by the need to engender trust in users by explaining to them why the AI is making a particular decision.
Revised: Explainable Artiﬁcial Intelligence (XAI) is an emerging research area in AI, motivated by the need to engender trust in users by explaining to them why the AI is making a particular decision.

Original: Whilethe main focus of XAI has been in Machine Learning, recently there has been growing interest in Explainable Planning, as shown by many planning contributions at the IJCAI International Workshops on XAI (XAI 2017; 2018) and the successful ﬁrst ICAPS Workshop on Explainable Planning (XAIP 2018).
Revised: Whilethe main focus of XAI has been in Machine Learning, recently there has been growing interest in Explainable Planning, as shown by many planning contributions at the IJCAI International Workshops on XAI (XAI 2017; 2018) and the successful ﬁrst ICAPS Workshop on Explainable Planning (XAIP 2018).

Original: Since the initial ideas of
Revised: Since the initial ideas of

Original: Smith (2012), there has been signiﬁcant effort, in particular on the topic of Human-Aware Planning and Model
Revised: Smith (2012), there has been signiﬁcant effort, in particular on the topic of Human-Aware Planning and Model

Original: Reconciliation (Chakraborti et al. 2017; Zhang et al. 2017;
Revised: Reconciliation (Chakraborti et al. 2017; Zhang et al. 2017;

Original: Sreedharan, Chakraborti and Kambhampati 2018).
Revised: Sreedharan, Chakraborti, and Kambhampati 2018).

Original: Other signiﬁcant works include topics such as Explainable Agency (Langley et al. 2017), moral values (Lindner, Mattm¨uller, and Nebel 2018), insights from social science (Miller 2019), and preferred explanations (Sohrabi, Baier and McIlraith 2011).
Revised: Other signiﬁcant works include topics such as Explainable Agency (Langley et al. 2017), moral values (Lindner, Mattm¨uller, and Nebel 2018), insights from social science (Miller 2019), and preferred explanations (Sohrabi, Baier, and McIlraith 2011).

Original: Aroadmap for Explainable Planning (XAIP) was proposed by Fox, Long, and Magazzeni (2017).
Revised: Aroadmap for Explainable Planning (XAIP) was proposed by Fox, Long, and Magazzeni (2017).

Original: They discussed some types of user questions that should be addressed.
Revised: They discussed some types of user questions that should be addressed.

Original: In particular, one type of question is of the form “ Why did you do A rather than B? ”.
Revised: In particular, one type of question is of the form “ Why did you do A rather than B? ”.

Original: We refer to this type of question as a contrastive question .
Revised: We refer to this type of question as a contrastive question .

Original: To answer this kind of question one must reason about the hypothetical alternative “B”, which likely means constructing an alternative plan where “B” is used rather than “A”.
Revised: To answer this kind of question one must reason about the hypothetical alternative “B”, which likely means constructing an alternative plan where “B” is used rather than “A”.

Original: The hypothetical alternative would be a plan that is not better than the one found by the planner or a plan which is better than the original one.
Revised: The hypothetical alternative would be a plan that is not better than the one found by the planner or a plan which is better than the original one.

Original: Providing such ss k g s k+1 s k+ r k+1 r k+ ss k g g s k+1 s k+ r k+1 r k+2 r k+n s k+m
Revised: Providing such ss k g s k+1 s k+ r k+1 r k+ ss k g g s k+1 s k+ r k+1 r k+2 r k+n s k+m

Original: A
Revised: A

Original: B
Revised: B

Original: A
Revised: A

Original: B a comparison between alternatives is what is called a contrastive explanation .
Revised: B a comparison between alternatives is what is called a contrastive explanation .

Original: Fig.
Revised: Fig.

Original: 1 shows an example of a contrastive question (Fox,
Revised: 1 shows an example of a contrastive question (Fox,

Original: Long, and Magazzeni 2017).
Revised: Long, and Magazzeni 2017).

Original: Given a plan from the initial state s 0 to the goal g , the users might suggest an alternative action B rather than A at the state s k .
Revised: Given a plan from the initial state s 0 to the goal g , the users might suggest an alternative action B rather than A at the state s k .

Original: To provide a contrastive explanation means forcing action B and then replanning from the resulting state to see the alternative plan.
Revised: To provide a contrastive explanation means forcing action B and then replanning from the resulting state to see the alternative plan.

Original: One possible alternative is a sequence of different actions that rejoin the original plan with a different cost (Fig. 1 left).
Revised: One possible alternative is a sequence of different actions that rejoin the original plan with a different cost (Fig. 1 left).

Original: Another possibility is a completely alternative sequence of actions that achieves the goal (Fig. 1 right).
Revised: Another possibility is a completely alternative sequence of actions that achieves the goal (Fig. 1 right).

Original: In both cases the user can compare costs to gain conﬁdence on the quality of the plan found by the planner.
Revised: In both cases the user can compare costs to gain conﬁdence on the quality of the plan found by the planner.

Original: Given that planning is now used in safety-critical applications, for example oil-well drilling (Long 2018), explanations play a key role.
Revised: Given that planning is now used in safety-critical applications, for example oil-well drilling (Long 2018), explanations play a key role.

Original: Crucially, the greater the expense or risks in executing the plan, the more important the role of explanations for engendering trust in the users who are responsible and accountable for authorising the execution of a plan.
Revised: Crucially, the greater the expense or risks in executing the plan, the more important the role of explanations for engendering trust in the users who are responsible and accountable for authorising the execution of a plan.

Original: In this paper, we propose that Explainable Planning can be designed and constructed as a service – i.e., as a wrapper around an existing planning system that takes as input the current planning problem and domain model, the current plan, and the user’s question.
Revised: In this paper, we propose that Explainable Planning can be designed and constructed as a service – i.e., as a wrapper around an existing planning system that takes as input the current planning problem and domain model, the current plan, and the user’s question.

Original: It must have the ability to invoke the existing planning system on hypothetical problems in order to address contrastive questions.
Revised: It must have the ability to invoke the existing planning system on hypothetical problems in order to address contrastive questions.

Original: This approach allows users to get explanations constructed from their own trusted planner and model.
Revised: This approach allows users to get explanations constructed from their own trusted planner and model.

Original: In complex or safety-critical domains this requirement is a crucial one.
Revised: In complex or safety-critical domains this requirement is a crucial one.

Original: There is one im- portant requirement, however; in order to effectively use the existing planning system, the XAIP service must be able to add constraints on the planning problem and domain model.
Revised: There is one im- portant requirement, however; in order to effectively use the existing planning system, the XAIP service must be able to add constraints on the planning problem and domain model.

Original: However, the user will not accept an explanation generated using a model that differs from the original one that is potentially veriﬁed and trusted.
Revised: However, the user will not accept an explanation generated using a model that differs from the original one that is potentially veriﬁed and trusted.

Original: Hence the explanation generated using the model revised with constraints has to be validated against the original model.
Revised: Hence the explanation generated using the model revised with constraints has to be validated against the original model.

Original: In other words, the contrastive explanation should contain an executable plan which leads to the goal state that the original planner could have created using the original model.
Revised: In other words, the contrastive explanation should contain an executable plan which leads to the goal state that the original planner could have created using the original model.

Original: Ideally, constraints over models should be described using a rich language designed for specifying constraints on the form of a desired plan.
Revised: Ideally, constraints over models should be described using a rich language designed for specifying constraints on the form of a desired plan.

Original: However, in many cases, these constraints can be compiled down into the domain model directly, which requires that the XAIP service have visibility and access into that model.
Revised: However, in many cases, these constraints can be compiled down into the domain model directly, which requires that the XAIP service have visibility and access into that model.

Original: This approach is otherwise agnostic about the domain model and the planner.
Revised: This approach is otherwise agnostic about the domain model and the planner.

Original: We have implemented this approach in a  framework for XAIP as a Service, in a PDDL setting, as it is a widely used planning language.
Revised: We have implemented this approach in a prototype framework for XAIP as a Service, in a PDDL setting, as it is a widely used planning language.

Original: In particular, in this paper, we (1) present a  framework that enables Explainable Planning as a Service for contrastive questions, (2) describe some important categories of contrastive questions, (3) describe how the service compiles these contrastive questions into hypothetical planning problems that can then be solved by the existing planner to facilitate contrastive explanations, and (4) discuss the current state of our implementation and a roadmap for future work on Explainable Planning as a Service.
Revised: In particular, in this paper, we (1) present a prototype framework that enables Explainable Planning as a Service for contrastive questions, (2) describe some important categories of contrastive questions, (3) describe how the service compiles these contrastive questions into hypothetical planning problems that can then be solved by the existing planner to facilitate contrastive explanations, and (4) discuss the current state of our implementation and a roadmap for future work on Explainable Planning as a Service.

Original: The paper is organised as follows: we start with a running example in the next section.
Revised: The paper is organised as follows: we start with a running example in the next section.

Original: In Section 3 we describe the XAIP as a Service framework, providing details for each component.
Revised: In Section 3 we describe the XAIP as a Service framework, providing details for each component.

Original: In Section 4 we brieﬂy discuss the framework that implements this approach.
Revised: In Section 4 we brieﬂy discuss the framework that implements this approach.

Original: In Section 5 we discuss open issues and present a roadmap that identiﬁes several possible research directions.
Revised: In Section 5 we discuss open issues and present a roadmap that identiﬁes several possible research directions.

Original: Section 6 concludes the paper.
Revised: Section 6 concludes the paper.

Original:  Running Example
Revised:  Running Example

Original: As a running example throughout the paper, we use a simpliﬁed version of a safety-critical model that might be used in industry.
Revised: As a running example throughout the paper, we use a simpliﬁed version of a safety-critical model that might be used in industry.

Original: The model describes a warehouse organisation delivery system.
Revised: The model describes a warehouse organisation delivery system.

Original: There are one or more robots that work together to move pallets from their delivery location to the correct storage shelf.
Revised: There are one or more robots that work together to move pallets from their delivery location to the correct storage shelf.

Original: Before the pallets can be stored the shelf must be scanned.
Revised: Before the pallets can be stored the shelf must be scanned.

Original: Fig.
Revised: Fig.

Original: 2 deﬁnes the domain for this model.
Revised: 2 deﬁnes the domain for this model.

Original: There are four temporal actions, goto waypoint , scan shel f , load pallet , and unload pallet .
Revised: There are four temporal actions, goto waypoint , scan shel f , load pallet , and unload pallet .

Original: The goto waypoint action is used for the robots to navigate the factory.
Revised: The goto waypoint action is used for the robots to navigate the factory.

Original: It ensures that the shelf the robot is moving to is not occupied by another robot to stop congestion.
Revised: It ensures that the shelf the robot is moving to is not occupied by another robot to stop congestion.

Original: The scan shel f action is a sensing action.
Revised: The scan shel f action is a sensing action.

Original: The load pallet action loads the pallet from a shelf on to the robot.
Revised: The load pallet action loads the pallet from a shelf on to the robot.

Original: The robots do not have the ability to scan the shelf while holding a pallet.
Revised: The robots do not have the ability to scan the shelf while holding a pallet.

Original: Finally, the unload pallet action unloads the pallet onto a previously scanned shelf.
Revised: Finally, the unload pallet action unloads the pallet onto a previously scanned shelf.

Original: The problem we use as an example is shown in Fig.
Revised: The problem we use as an example is shown in Fig.

Original: There are two robots, two pallets, and six waypoints.
Revised: There are two robots, two pallets, and six waypoints.

Original: An example plan for this planning problem is shown in
Revised: An example plan for this planning problem is shown in

Original: Fig.
Revised: Fig.

Original: 4, its cost is its duration (20.003) which in this case is the optimal plan 1 .
Revised: 4, its cost is its duration (20.003) which in this case is the optimal plan 1 .

Original: Upon close examination, the plan does not appear straight-forward.
Revised: Upon close examination, the plan does not appear straight-forward.

Original: Both pallets are delivered by a single robot, and there are a lot of movements that appear to be inefﬁcient.
Revised: Both pallets are delivered by a single robot, and there are a lot of movements that appear to be inefﬁcient.

Original: For example, the robot Tom moves away from waypoint sh 6, even though there is an undelivered pallet at that location.
Revised: For example, the robot Tom moves away from waypoint sh 6, even though there is an undelivered pallet at that location.

Original: It might seem more efﬁcient to pick up that pallet while the robot is beside it.
Revised: It might seem more efﬁcient to pick up that pallet while the robot is beside it.

Original: Without access to the domain and problem shown in Fig.
Revised: Without access to the domain and problem shown in Fig.

Original: 2 and 3, or without understanding of PDDL semantics, the behaviour of these robots will be opaque to a user, and explanation required.
Revised: 2 and 3, or without understanding of PDDL semantics, the behaviour of these robots will be opaque to a user, and explanation required.

Original: [Equation]
Revised: [Equation]

Revised: [Table]

Original: [Table]
Revised: [Table]

Revised: holding_pallet ?

Original: v - robot) )
Revised: v - robot) )

Original: (:functions (travel_time ?
Revised: (:functions (travel_time ?

Original: wp1 ?
Revised: wp1 ?

Original: wp2 - waypoint)) (:durative-action goto_waypoint :parameters (?v - robot ?
Revised: wp2 - waypoint)) (:durative-action goto_waypoint :parameters (?v - robot ?

Original: from ?to - waypoint) :duration(= ?
Revised: from ?to - waypoint) :duration(= ?

Original: duration (travel_time ?
Revised: duration (travel_time ?

Original: from ?to)) :condition (and (at start (robot_at ?v ?from))
Revised: from ?to)) :condition (and (at start (robot_at ?v ?from))

Original: (at start (not_occupied ?to))
Revised: (at start (not_occupied ?to))

Original: (over all (connected ?
Revised: (over all (connected ?

Original: from ?to))) :effect (and (at start (not (not_occupied ?to)))
Revised: from ?to))) :effect (and (at start (not (not_occupied ?to)))

Original: (at end (not_occupied ?from))
Revised: (at end (not_occupied ?from))

Original: (at start (not (robot_at ?v ?from)))
Revised: (at start (not (robot_at ?v ?from)))

Original: (at end (robot_at ?v ?to))) )
Revised: (at end (robot_at ?v ?to))) )

Original: (:durative-action scan_shelf :parameters (?v - robot ?
Revised: (:durative-action scan_shelf :parameters (?v - robot ?

Original: shelf - waypoint) ...)
Revised: shelf - waypoint) ...)

Original: (:durative-action load_pallet :parameters (?v - robot ?
Revised: (:durative-action load_pallet :parameters (?v - robot ?

Original: p - pallet ?
Revised: p - pallet ?

Original: shelf - waypoint) ...)
Revised: shelf - waypoint) ...)

Original: (:durative-action unload_pallet :parameters (?v - robot ?
Revised: (:durative-action unload_pallet :parameters (?v - robot ?

Original: p - pallet ?
Revised: p - pallet ?

Original: shelf - waypoint) ...)
Revised: shelf - waypoint) ...)

Original: [Figure]
Revised: [Figure]

Original: [Figure]
Revised: [Figure]

Original: goto_waypoint
Revised: goto_waypoint

Original: [Figure]
Revised: [Figure]

Original: sh6 sh
Revised: sh6 sh

Original: [Figure]
Revised: [Figure]

Original: [Table]
Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original: [Table]
Revised: [Table]

Original: 503: (unload_pallet Jerry p
Revised: 503: (unload_pallet Jerry p

Original: [Table] [Table]
Revised: [Table] [Table]

Original: [Figure]
Revised: [Figure]

Original:  Providing explanations as a Service
Revised:  Providing explanations as a Service

Original: We present an XAIP Service framework for providing contrastive explanations.
Revised: We present an XAIP Service framework for providing contrastive explanations.

Original: Figure 5 summarises the approach taken by the framework, following these steps:
Revised: Figure 5 summarises the approach taken by the framework, following these steps:

Original: Step 1: The XAIP Service takes as input the planning problem and model, the plan, and the question from the user.
Revised: Step 1: The XAIP Service takes as input the planning problem and model, the plan, and the question from the user.

Original: Step 2: The contrastive question implies a hypothetical model characterised as an additional set of constraints on the actions and timing of the original problem.
Revised: Step 2: The contrastive question implies a hypothetical model characterised as an additional set of constraints on the actions and timing of the original problem.

Original: These constraints can then be compiled into a revised domain model (HModel) suitable for use by the original planner.
Revised: These constraints can then be compiled into a revised domain model (HModel) suitable for use by the original planner.

Original: [Figure]
Revised: [Figure]

Original: Step 3: The original planner uses the HModel as input to produce the hypothetical plan (HPlan).
Revised: Step 3: The original planner uses the HModel as input to produce the hypothetical plan (HPlan).

Original: Step 4: The XAIP Service validates the HPlan according to the original model.
Revised: Step 4: The XAIP Service validates the HPlan according to the original model.

Original: Step 5: A contrastive explanation is constructed from the original plan and HPlan, and shown to the user.
Revised: Step 5: A contrastive explanation is constructed from the original plan and HPlan, and shown to the user.

Original: Step 6: The user can choose to iterate the process from Step 1 with a new question.
Revised: Step 6: The user can choose to iterate the process from Step 1 with a new question.

Original: The user can choose to repeat the process with the original model and plan, or any HModel and HPlan.
Revised: The user can choose to repeat the process with the original model and plan, or any HModel and HPlan.

Original: The role of the HModel is to coerce the planner into creating the alternative plan that includes the actions and temporal constraints the user has in mind.
Revised: The role of the HModel is to coerce the planner into creating the alternative plan that includes the actions and temporal constraints the user has in mind.

Original: One or more constraints must be added to the original model to create the
Revised: One or more constraints must be added to the original model to create the

Original: HModel.
Revised: HModel.

Original: We distinguish between three levels of abstraction in this process:
Revised: We distinguish between three levels of abstraction in this process:

Original: In the ﬁrst level the user question is given in natural language.
Revised: In the ﬁrst level the user question is given in natural language.

Original: The second level is a formal question that is derived from the natural language question.
Revised: The second level is a formal question that is derived from the natural language question.

Original: The formal question represents a set of constraints that are to be imposed upon the original model.
Revised: The formal question represents a set of constraints that are to be imposed upon the original model.

Original: Finally, the third level is a compilation of the formal question into the planning language.
Revised: Finally, the third level is a compilation of the formal question into the planning language.

Original: In our framework we focus on PDDL2.1 as the planning language, as we are interested in temporal and numeric planning problems.
Revised: In our framework we focus on PDDL2.1 as the planning language, as we are interested in temporal and numeric planning problems.

Original: In this paper we describe the overall approach, and present a framework that encapsulates this process.
Revised: In this paper we describe the overall approach, and present a framework that encapsulates this process.

Original: The framework is modular, and allows different interfaces for providing user questions, and presenting explanations.
Revised: The framework is modular, and allows different interfaces for providing user questions, and presenting explanations.

Original: The interface currently implemented in the framework allows the user to select a formal question directly.
Revised: The interface currently implemented in the framework allows the user to select a formal question directly.

Original: This represents constraints upon the plan.
Revised: This represents constraints upon the plan.

Original: The set of formal question types from which the selection is made is described below in Section 3.1.
Revised: The set of formal question types from which the selection is made is described below in Section 3.1.

Original: For example, the user might have a question for the running example such as: “At the point in the plan where action ( goto waypoint Tom sh 6 sh 1 ) is used, there’s a pallet, so why doesn’t Tom pick it up?”
Revised: For example, the user might have a question for the running example such as: “At the point in the plan where action ( goto waypoint Tom sh 6 sh 1 ) is used, there’s a pallet, so why doesn’t Tom pick it up?”

Original: From the user question above the following formal question is derived:
Revised: From the user question above the following formal question is derived:

Original: Why is action A used in state S, rather than action B?
Revised: Why is action A used in state S, rather than action B?

Original: Where action A is ( goto waypoint Tom sh 6 sh 1 ) , action B is ( load pallet Tom p 2 sh 6 ) , and the state S is the state in which action A was originally applied.
Revised: Where action A is ( goto waypoint Tom sh 6 sh 1 ) , action B is ( load pallet Tom p 2 sh 6 ) , and the state S is the state in which action A was originally applied.

Original: This constraint enforces that the plan includes action B in state S instead.
Revised: This constraint enforces that the plan includes action B in state S instead.

Original: Finally, this constraint can be compiled into the original model to produce the HModel, as described in Section 3.2.
Revised: Finally, this constraint can be compiled into the original model to produce the HModel, as described in Section 3.2.

Original: 3. Encoding User Questions
Revised: 3. Encoding User Questions

Original: The user question is encoded as a set of constraints, which represent the formal question, and this is be done through an user interface where the user is guided to select the constraints that match their question.
Revised: The user question is encoded as a set of constraints, which represent the formal question, and this is be done through an user interface where the user is guided to select the constraints that match their question.

Original: The questions we are interested in are contrastive questions of the form, ”Why A rather than B?” , where A is the fact (i.e. what occurred in the plan) and B is the foil (i.e. the hypothetical alternative expected by the user).
Revised: The questions we are interested in are contrastive questions of the form, ”Why A rather than B?” , where A is the fact (i.e. what occurred in the plan) and B is the foil (i.e. the hypothetical alternative expected by the user).

Original: The formal questions currently handled by our approach are:
Revised: The formal questions currently handled by our approach are:

Original: • “ Why is action A used in the plan, rather than not being used? ”
Revised: • “ Why is action A used in the plan, rather than not being used? ”

Original: This constraint would prevent the action A from being used in the plan.
Revised: This constraint would prevent the action A from being used in the plan.

Original: • “ Why is action A not used in the plan, rather than being used? ”
Revised: • “ Why is action A not used in the plan, rather than being used? ”

Original: This constraint would enforce that the action A is applied at some point in the plan.
Revised: This constraint would enforce that the action A is applied at some point in the plan.

Original: • “ Why is action A used, rather than action B? ”
Revised: • “ Why is action A used, rather than action B? ”

Original: This constraint is a combination of the previous two, which enforces that the plan include action B and not action A. • “ Why is action A used before/after action B (rather than after/before)? ”
Revised: This constraint is a combination of the previous two, which enforces that the plan include action B and not action A. • “ Why is action A used before/after action B (rather than after/before)? ”

Original: This constraint enforces that if action A is used, action B must appear earlier/later in the plan.
Revised: This constraint enforces that if action A is used, action B must appear earlier/later in the plan.

Original: • “ Why is action A used outside of time window W, rather than only being allowed inside W? ”
Revised: • “ Why is action A used outside of time window W, rather than only being allowed inside W? ”

Original: This constraint forces the planner to schedule action A within a speciﬁc time window.
Revised: This constraint forces the planner to schedule action A within a speciﬁc time window.

Original: One speciﬁc form of the question: “ Why is action A used, rather than action B? ”, represented in Fig.
Revised: One speciﬁc form of the question: “ Why is action A used, rather than action B? ”, represented in Fig.

Original: 1, is “ Why is action A used in state S, rather than action B? ”.
Revised: 1, is “ Why is action A used in state S, rather than action B? ”.

Original: This reﬁnement forces the plan to include action B in state S, where
Revised: This reﬁnement forces the plan to include action B in state S, where

Original: B is an action (different from A) that is valid in that state.
Revised: B is an action (different from A) that is valid in that state.

Original: Using this constraint, the actions leading up to state S would remain unchanged, and the action A would still be allowed in other parts of the plan.
Revised: Using this constraint, the actions leading up to state S would remain unchanged, and the action A would still be allowed in other parts of the plan.

Original: Deriving a formal question from a user question in natural language represents a signiﬁcant research challenge.
Revised: Deriving a formal question from a user question in natural language represents a signiﬁcant research challenge.

Original: We discuss how a formal question might be identiﬁed automatically from natural language in Section 5.
Revised: We discuss how a formal question might be identiﬁed automatically from natural language in Section 5.

Original: While this set of formal questions covers a wide a range of possible situations and explanations, this does not comprise a complete set of constraints that could be applied to the problem.
Revised: While this set of formal questions covers a wide a range of possible situations and explanations, this does not comprise a complete set of constraints that could be applied to the problem.

Original: However, a single explanation does not exist in a vacuum, and a user might have a series of questions that will iteratively increase their understanding of the plan proposed by the planner, or the user can realise that the ﬁrst question was not complete.
Revised: However, a single explanation does not exist in a vacuum, and a user might have a series of questions that will iteratively increase their understanding of the plan proposed by the planner, or the user can realise that the ﬁrst question was not complete.

Original: For this reason, following ideas of Smith (2012), the framework allows the user to apply a sequence of formal questions to a single plan.
Revised: For this reason, following ideas of Smith (2012), the framework allows the user to apply a sequence of formal questions to a single plan.

Original: For example, the general question “Why is action A used, rather than action
Revised: For example, the general question “Why is action A used, rather than action

Original: B” can be seen as a combination of the ﬁrst two questions, which force action B to appear in the plan and not action A.
Revised: B” can be seen as a combination of the ﬁrst two questions, which force action B to appear in the plan and not action A.

Original: This allows for a much wider range of explanations.
Revised: This allows for a much wider range of explanations.

Original: Clearly the requirement of the plan being valid according to the original model must be satisﬁed at each iteration.
Revised: Clearly the requirement of the plan being valid according to the original model must be satisﬁed at each iteration.

Original: To allow the user to explore the space of hypothetical plans, it is possible to generate a tree of contrastive explanations in our framework, as illustrated in Fig.
Revised: To allow the user to explore the space of hypothetical plans, it is possible to generate a tree of contrastive explanations in our framework, as illustrated in Fig.

Original: Each node of the tree represents a hypothetical plan that was generated following a user question.
Revised: Each node of the tree represents a hypothetical plan that was generated following a user question.

Original: The user can impose additional formal questions to a given plan in order to more precisely explore the behaviour that they are interested in.
Revised: The user can impose additional formal questions to a given plan in order to more precisely explore the behaviour that they are interested in.

Original: If the ﬁrst question did not result in expected behaviour, the user can ask new questions to reﬁne the hypothetical model until they reach a contrastive explanation that is satisfying to them.
Revised: If the ﬁrst question did not result in expected behaviour, the user can ask new questions to reﬁne the hypothetical model until they reach a contrastive explanation that is satisfying to them.

Original: As previously identiﬁed, the corresponding formal question in our running example is “Why is action A used in state S , rather than action B?” , where action A is ( goto waypoint Tom sh 6 sh 1 ) , action B is ( load pallet Tom p 2 sh 6 ) , and the state S is the state in which action A was originally applied.
Revised: As previously identiﬁed, the corresponding formal question in our running example is “Why is action A used in state S , rather than action B?” , where action A is ( goto waypoint Tom sh 6 sh 1 ) , action B is ( load pallet Tom p 2 sh 6 ) , and the state S is the state in which action A was originally applied.

Original: 3. Constructing the HModel
Revised: 3. Constructing the HModel

Original: Constructing an HModel consists of taking the constraints in the formal question and compiling them into the planning domain model.
Revised: Constructing an HModel consists of taking the constraints in the formal question and compiling them into the planning domain model.

Original: We give one example of this step, using the original model and plan shown in Section 2, and the question above.
Revised: We give one example of this step, using the original model and plan shown in Section 2, and the question above.

Original: The compilation is formed such that the ground action B appears in the plan in place of the action A .
Revised: The compilation is formed such that the ground action B appears in the plan in place of the action A .

Original: Given a plan:
Revised: Given a plan:

Original: [Equation]
Revised: [Equation]

Original: The ground action a i =
Revised: The ground action a i =

Original: A is replaced with B .
Revised: A is replaced with B .

Original: The state directly after B has ﬁnished executing is found.
Revised: The state directly after B has ﬁnished executing is found.

Original: This then becomes the new initial state I (cid:48) in the HModel.
Revised: This then becomes the new initial state I (cid:48) in the HModel.

Original: The framework uses the effects at each happening, computed by
Revised: The framework uses the effects at each happening, computed by

Original: VAL (Howey, Long, and Fox 2004), up to the replacement action to compute I (cid:48) .
Revised: VAL (Howey, Long, and Fox 2004), up to the replacement action to compute I (cid:48) .

Original: In addition, the new initial state I (cid:48) is extended with a set of timed-initial-literals (TILs) which model the effects of actions that have started but not yet ﬁnished execution in the state selected by the user.
Revised: In addition, the new initial state I (cid:48) is extended with a set of timed-initial-literals (TILs) which model the effects of actions that have started but not yet ﬁnished execution in the state selected by the user.

Original: A TIL is a tuple (cid:104) t , p (cid:105) where p is the effect that is asserted, and t is the time at which it is applied.
Revised: A TIL is a tuple (cid:104) t , p (cid:105) where p is the effect that is asserted, and t is the time at which it is applied.

Original: Speciﬁcally, for each action a j not ﬁnished executing in state S, we add the TIL:
Revised: Speciﬁcally, for each action a j not ﬁnished executing in state S, we add the TIL:

Original: [Equation]
Revised: [Equation]

Original: [Figure]
Revised: [Figure]

Original: where start ( a j ) ( start ( a j ) ) is the time at which the action a j ( B ) started execution, duration ( a j ) is the planned duration of action a j , and ef fect ( a j ) is the end effect of action a j .
Revised: where start ( a j ) ( start ( a j ) ) is the time at which the action a j ( B ) started execution, duration ( a j ) is the planned duration of action a j , and ef fect ( a j ) is the end effect of action a j .

Original: In this example the action ( goto waypoint Jerrysh 3 sh 4 ) is still executing in the state where our action is replaced.
Revised: In this example the action ( goto waypoint Jerrysh 3 sh 4 ) is still executing in the state where our action is replaced.

Original: We add a TIL which makes ( not occupied sh 3 ) , and ( robot at Jerrysh 4 ) true at time 2.999 in I (cid:48) .
Revised: We add a TIL which makes ( not occupied sh 3 ) , and ( robot at Jerrysh 4 ) true at time 2.999 in I (cid:48) .

Original: This simulates ﬁnishing of the concurrent action execution.
Revised: This simulates ﬁnishing of the concurrent action execution.

Original: A plan is then generated from this new state for the original goal, which gives us the plan:
Revised: A plan is then generated from this new state for the original goal, which gives us the plan:

Original: [Equation]
Revised: [Equation]

Original: The HPlan is then the initial actions of the original plan π up to a i , concatenated with the replaced action B and the new plan π (cid:48) :
Revised: The HPlan is then the initial actions of the original plan π up to a i , concatenated with the replaced action B and the new plan π (cid:48) :

Original: [Equation]
Revised: [Equation]

Original: The result is the HPlan shown in Fig.
Revised: The result is the HPlan shown in Fig.

Original: The replaced action (B) is shown in bold.
Revised: The replaced action (B) is shown in bold.

Original: The initial actions before B are the actions from the original plan.
Revised: The initial actions before B are the actions from the original plan.

Original: The remaining actions are those of the new plan π (cid:48) .
Revised: The remaining actions are those of the new plan π (cid:48) .

Original: As part of the service, the HPlan is validated with respect to the original model before a contrastive explanation is formed.
Revised: As part of the service, the HPlan is validated with respect to the original model before a contrastive explanation is formed.

Original: Note that in this HPlan, the user action is immediately reversed.
Revised: Note that in this HPlan, the user action is immediately reversed.

Original: This would not be a satisfactory explanation to the user, who wishes to see the plan in which carrying the pallet is essential to achieve the goal.
Revised: This would not be a satisfactory explanation to the user, who wishes to see the plan in which carrying the pallet is essential to achieve the goal.

Original: Thus, it is not sufﬁcient for the user suggested action to just be included in the plan, it must be part of the plan’s key causal structure.
Revised: Thus, it is not sufﬁcient for the user suggested action to just be included in the plan, it must be part of the plan’s key causal structure.

Original: Fink and Yang (1992) deﬁne four categories of redundant actions, and Chrpa, Mccluskey, and Osborne (2012) present a simple algorithm to determine if an action is redundant in a sequential plan.
Revised: Fink and Yang (1992) deﬁne four categories of redundant actions, and Chrpa, Mccluskey, and Osborne (2012) present a simple algorithm to determine if an action is redundant in a sequential plan.

Original: As a post-processing step, after generating an HPlan, redundant actions of this kind can be detected.
Revised: As a post-processing step, after generating an HPlan, redundant actions of this kind can be detected.

Original: If suggested actions are not essential (redundant) to the HPlan, the planner needs to continue to search for additional plans until it ﬁnds one where the suggested actions are part of the causal structure for achieving the original goals.
Revised: If suggested actions are not essential (redundant) to the HPlan, the planner needs to continue to search for additional plans until it ﬁnds one where the suggested actions are part of the causal structure for achieving the original goals.

Original: This additional search could potentially be made more efﬁcient by introducing additional constraints (nogoods) into the HModel that rule out alternative ways of achieving those goals, ultimately leaving only plans where the suggested actions are essential.
Revised: This additional search could potentially be made more efﬁcient by introducing additional constraints (nogoods) into the HModel that rule out alternative ways of achieving those goals, ultimately leaving only plans where the suggested actions are essential.

Original: For example, if the action B is redundant in the
Revised: For example, if the action B is redundant in the

Original: HPlan and action C is used instead of A, an additional constraint could be introduced to disallow C.
Revised: HPlan and action C is used instead of A, an additional constraint could be introduced to disallow C.

Original: In general, it is still an open question how to automatically infer useful nogood constraints from redundant HPlans.
Revised: In general, it is still an open question how to automatically infer useful nogood constraints from redundant HPlans.

Original: An alternative is to allow the user to reﬁne their question to rule out additional alternative solutions in which the suggested action is not essential.
Revised: An alternative is to allow the user to reﬁne their question to rule out additional alternative solutions in which the suggested action is not essential.

Original: For example, the user’s question might be expanded to include the constraint “and don’t use actions C or D either”.
Revised: For example, the user’s question might be expanded to include the constraint “and don’t use actions C or D either”.

Original: In either case it is clear that XAIP as a Service needs to follow an iterative approach where the planner generates a sequence of progressively more reﬁned solutions, as additional constraints are imposed by nogoods, or by successive reﬁnement of the user question.
Revised: In either case it is clear that XAIP as a Service needs to follow an iterative approach where the planner generates a sequence of progressively more reﬁned solutions, as additional constraints are imposed by nogoods, or by successive reﬁnement of the user question.

Original: This is discussed further in Section 3.3.
Revised: This is discussed further in Section 3.3.

Original: Ideally, we would like to compile the original user question into an HModel that guarantees that suggested actions are essential to the causal structure of the plan (not redundant).
Revised: Ideally, we would like to compile the original user question into an HModel that guarantees that suggested actions are essential to the causal structure of the plan (not redundant).

Original: However, it is an open question as to whether it is possible to do this.
Revised: However, it is an open question as to whether it is possible to do this.

Original: We suspect not.
Revised: We suspect not.

Original: It is easy to force an action into a plan by adding a phantom effect to the action, and adding this phantom proposition to the goal.
Revised: It is easy to force an action into a plan by adding a phantom effect to the action, and adding this phantom proposition to the goal.

Original: However, it is not obvious how to ensure that the action play an essential part in the achievement of other goals.
Revised: However, it is not obvious how to ensure that the action play an essential part in the achievement of other goals.

Original: In addition to the example shown here, our framework includes compilations for all of the questions introduced in
Revised: In addition to the example shown here, our framework includes compilations for all of the questions introduced in

Original: Section 3.1.
Revised: Section 3.1.

Original: Contrary to the compilation process of the presented example, the constraints posed from other questions are directly compiled into HModels which are used to produce HPlans.
Revised: Contrary to the compilation process of the presented example, the constraints posed from other questions are directly compiled into HModels which are used to produce HPlans.

Original: 3. Explainable Planning as an Iterative Process
Revised: 3. Explainable Planning as an Iterative Process

Original: Following the ideas of iterative processes by Smith (2012), we follow the same approach for explainable planning.
Revised: Following the ideas of iterative processes by Smith (2012), we follow the same approach for explainable planning.

Original: The user is able to use the framework to iterate the process by asking further questions, and reﬁning the HModel.
Revised: The user is able to use the framework to iterate the process by asking further questions, and reﬁning the HModel.

Original: If the explanation does not completely satisfy the user, this allows the user to impose additional constraints that can be compiled into the HModel.
Revised: If the explanation does not completely satisfy the user, this allows the user to impose additional constraints that can be compiled into the HModel.

Original: For example, given the HPlan in
Revised: For example, given the HPlan in

Original: Fig.
Revised: Fig.

Original: 7, the user may have an additional question: “Wait, shouldn’t Tom have taken the pallet to its destination?”
Revised: 7, the user may have an additional question: “Wait, shouldn’t Tom have taken the pallet to its destination?”

Original: Through the user interface with the framework the user selects a formal question.
Revised: Through the user interface with the framework the user selects a formal question.

Original: The formal question which encapsulates this user question is
Revised: The formal question which encapsulates this user question is

Original: Why is the action A not used in the plan, rather than being used?
Revised: Why is the action A not used in the plan, rather than being used?

Original: Where action A is the action ( unload pallet Tom p 2 sh 1 ) .
Revised: Where action A is the action ( unload pallet Tom p 2 sh 1 ) .

Original: This constraint enforces that action A is used in the plan.
Revised: This constraint enforces that action A is used in the plan.

Original: This constraint compiled into the HModel to enforce the user’s suggestion, resulting in a new HModel.
Revised: This constraint compiled into the HModel to enforce the user’s suggestion, resulting in a new HModel.

Original: The HModel is then solved with the original planner to obtain the plan shown in 8.
Revised: The HModel is then solved with the original planner to obtain the plan shown in 8.

Original: [Figure]
Revised: [Figure]

Original: 3. Forming Contrastive explanations
Revised: 3. Forming Contrastive explanations

Original: A contrastive explanation draws from the original plan π , the
Revised: A contrastive explanation draws from the original plan π , the

Original: HPlan π H and the validation outcome.
Revised: HPlan π H and the validation outcome.

Original: Deﬁning a contrastive explanation is a complex task.
Revised: Deﬁning a contrastive explanation is a complex task.

Original: In this paper we introduce a foundation for the contrastive comparison as a result of a comparison between the original plan and the HPlan.
Revised: In this paper we introduce a foundation for the contrastive comparison as a result of a comparison between the original plan and the HPlan.

Original: The contrastive explanation can be deﬁned as
Revised: The contrastive explanation can be deﬁned as

Original: [Equation]
Revised: [Equation]

Original: where comparison contains relevant information about the differences in plans that were caused by the user question Q : comparison ( π , π H )
Revised: where comparison contains relevant information about the differences in plans that were caused by the user question Q : comparison ( π , π H )

Original: = (cid:104) existing , removed , added , dif fcost (cid:105) where:
Revised: = (cid:104) existing , removed , added , dif fcost (cid:105) where:

Original: • existing - actions in the plan which remained the same • removed - actions which were removed from the plan • added - actions which were added in HPlan and were not in original plan • diffcost -
Revised: • existing - actions in the plan which remained the same • removed - actions which were removed from the plan • added - actions which were added in HPlan and were not in original plan • diffcost -

Original: the difference between the cost of the plans
Revised: the difference between the cost of the plans

Original: By observing the comparison , a user can reason about the effect of the constraints that their question imposed and about the behaviour of the model.
Revised: By observing the comparison , a user can reason about the effect of the constraints that their question imposed and about the behaviour of the model.

Original: The validation outcome (performed against the original model) reassures the user about the validity of the HPlan.
Revised: The validation outcome (performed against the original model) reassures the user about the validity of the HPlan.

Original: It also helps the user understand the difference between the costs of the two plans.
Revised: It also helps the user understand the difference between the costs of the two plans.

Original: The contrastive explanation for the HPlan in Fig. 8 is shown in Fig.
Revised: The contrastive explanation for the HPlan in Fig. 8 is shown in Fig.

Original:  XAIP as a Service Framework
Revised:  XAIP as a Service Framework

Original: For the purpose of evaluating the proposed approach to Explainable Planning as a Service we implemented a modular framework for domains and problems written in PDDL2.1.
Revised: For the purpose of evaluating the proposed approach to Explainable Planning as a Service we implemented a modular framework for domains and problems written in PDDL2.1.

Original: The framework works with any planner capable of reasoning with PDDL2.1.
Revised: This prototype works with any planner capable of reasoning with PDDL2.1.

Original: The architecture of the framework is illustrated in Fig.
Revised: The architecture of the framework is illustrated in Fig.

Original: Interaction with a user is enabled through a console interface as well as a graphical user interface.
Revised: Interaction with a user is enabled through a console interface as well as a graphical user interface.

Original: The Controller controls the behaviour of the XAIP framework and communicates with all other segments of the framework.
Revised: The Controller controls the behaviour of the XAIP framework and communicates with all other segments of the framework.

Original: The XAIP Interface creates a knowledge base existing :
Revised: The XAIP Interface creates a knowledge base existing :

Original: [Table]
Revised: [Table]

Original: pallet Jerry p2 sh1)
Revised: pallet Jerry p2 sh1)

Original: [1.500] added : 7.004: (goto_waypoint Tom sh1 sh6)
Revised: [1.500] added : 7.004: (goto_waypoint Tom sh1 sh6)

Original: [4.000] 11.004: (load_pallet Tom p2 sh6) [2.000] 13.004: (
Revised: [4.000] 11.004: (load_pallet Tom p2 sh6) [2.000] 13.004: (

Original: goto_waypoint Tom sh6 sh1) [4.000] 17.004: (unload_pallet Tom p2 sh1) [1.500] dif fcost = 21 .
Revised: goto_waypoint Tom sh6 sh1) [4.000] 17.004: (unload_pallet Tom p2 sh1) [1.500] dif fcost = 21 .

Original: 505 − 20 .
Revised: 505 − 20 .

Original: 003 = 1 .
Revised: 003 = 1 .

Original: [Figure]
Revised: [Figure]

Original: from the PDDL ﬁles of the original model.
Revised: from the PDDL ﬁles of the original model.

Original: This module also interacts with a planner.
Revised: This module also interacts with a planner.

Original: The Compilation module uses the constraints of the formal question to create the HModel.
Revised: The Compilation module uses the constraints of the formal question to create the HModel.

Original: The validation module VAL (Howey, Long, and Fox 2004) is used as the validation technique.
Revised: The validation module VAL (Howey, Long, and Fox 2004) is used as the validation technique.

Original: The Human-XAIP interface and interpreter receives the questions from the user, creates a formal question instance, and demonstrates the explanation to the user.
Revised: The Human-XAIP interface and interpreter receives the questions from the user, creates a formal question instance, and demonstrates the explanation to the user.

Original: There are two implementations of this interface: a console and graphical user interface (Fig. 11).
Revised: There are two implementations of this interface: a console and graphical user interface (Fig. 11).

Original: Both provide the same functionality, in which a user can see the plan, ask a formal question from the set of questions presented in Section 3.1, either by using a simple console interface or a set of forms for ﬁlling in the necessary details of the question.
Revised: Both provide the same functionality, in which a user can see the plan, ask a formal question from the set of questions presented in Section 3.1, either by using a simple console interface or a set of forms for ﬁlling in the necessary details of the question.

Original: The output of the system is a visualisation of the original plan, HPlan, and VAL report.
Revised: The output of the system is a visualisation of the original plan, HPlan, and VAL report.

Original: Actions in the plan are highlighted to correspond with the comparison described in Section 3.4.
Revised: Actions in the plan are highlighted to correspond with the comparison described in Section 3.4.

Original: [Figure]
Revised: [Figure]

Original:  Discussion
Revised:  Discussion

Original: In order for Explainable Planning as a service to become effective, we discuss some challenges and outline some potential future work.
Revised: In order for Explainable Planning as a service to become effective, we discuss some challenges and outline some potential future work.

Original: • Understanding user questions.
Revised: • Understanding user questions.

Original: The framework is modular to enable different ways for communicating a question to the AI system.
Revised: The framework is modular to enable different ways for communicating a question to the AI system.

Original: Depending on the user interface a question could be given in different forms, for example speech, visual gestures or text input.
Revised: Depending on the user interface a question could be given in different forms, for example speech, visual gestures or text input.

Original: Different technologies can be used to translate the question into a formal question such as: speech recognition, Natural Language Processing methods or human body tracking.
Revised: Different technologies can be used to translate the question into a formal question such as: speech recognition, Natural Language Processing methods or human body tracking.

Original: Context can play a crucial role in understanding the question that the user asks.
Revised: Context can play a crucial role in understanding the question that the user asks.

Original: Borgo, Cashmore, and Magazzeni
Revised: Borgo, Cashmore, and Magazzeni

Original: ) showed an example of a question requiring two different explanations depending on the context in which it was asked.
Revised: ) showed an example of a question requiring two different explanations depending on the context in which it was asked.

Original: In each case, improperly interpreting the question lead to an unsatisfying explanation.
Revised: In each case, improperly interpreting the question lead to an unsatisfying explanation.

Original: One promising direction for addressing this challenge is the use of argumentation (Cyras et al. 2019).
Revised: One promising direction for addressing this challenge is the use of argumentation (Cyras et al. 2019).

Original: • Formally categorising the set of questions that can be answered with contrastive explanations.
Revised: • Formally categorising the set of questions that can be answered with contrastive explanations.

Original: Although some philosophers, such as van Fraassen (1980), noted that “why”-questions can be implicitly or explicitly understood as: “why is A better than some alternative?”, there might be questions in the planning space for which contrastive explanations are not well-suited.
Revised: Although some philosophers, such as van Fraassen (1980), noted that “why”-questions can be implicitly or explicitly understood as: “why is A better than some alternative?”, there might be questions in the planning space for which contrastive explanations are not well-suited.

Original: For example, if the user is simply trying to understand the conditions or requirements for various actions in the plan, or the causal or temporal structure of the plan, a contrastive explanation may not be appropriate.
Revised: For example, if the user is simply trying to understand the conditions or requirements for various actions in the plan, or the causal or temporal structure of the plan, a contrastive explanation may not be appropriate.

Original: Instead it might be more appropriate to highlight the causal structure in an abstracted version of the plan.
Revised: Instead it might be more appropriate to highlight the causal structure in an abstracted version of the plan.

Original: An important issue for future work is the development of a formal taxonomy of the types of questions that should be addressed using contrastive explanations (Miller 2018).
Revised: An important issue for future work is the development of a formal taxonomy of the types of questions that should be addressed using contrastive explanations (Miller 2018).

Original: • Expressing constraints on actions and plan structure.
Revised: • Expressing constraints on actions and plan structure.

Original: A contrastive question requires creating a hypothetical planning model, which is often characterised by constraints on what actions are permissible in the plan and how they are arranged.
Revised: A contrastive question requires creating a hypothetical planning model, which is often characterised by constraints on what actions are permissible in the plan and how they are arranged.

Original: For example, the question “Why did you use action A rather than action B for achieving P?” requires planning with the hypothetical model where B is required to be in the causal support for achieving P, but A is not in that causal support.
Revised: For example, the question “Why did you use action A rather than action B for achieving P?” requires planning with the hypothetical model where B is required to be in the causal support for achieving P, but A is not in that causal support.

Original: This is substantially more difﬁcult than just universally excluding A from the plan and forcing B into the plan because A or B might be required or prohibited elsewhere in the plan.
Revised: This is substantially more difﬁcult than just universally excluding A from the plan and forcing B into the plan because A or B might be required or prohibited elsewhere in the plan.

Original: Currently we do not have a good language for expressing these kinds of constraints.
Revised: Currently we do not have a good language for expressing these kinds of constraints.

Original: PDDL 3 allows the expression of simple constraints on the order in which goals are achieved, but does not have the ability to express constraints on action inclusion, exclusion, or ordering, and does not allow us to place more complex constraints on how something is achieved or on plan structure.
Revised: PDDL 3 allows the expression of simple constraints on the order in which goals are achieved, but does not have the ability to express constraints on action inclusion, exclusion, or ordering, and does not allow us to place more complex constraints on how something is achieved or on plan structure.

Original: We would like to be able to say something simple like “Supports ( B , P ) ∧¬ Supports ( B , P ) ”.
Revised: We would like to be able to say something simple like “Supports ( B , P ) ∧¬ Supports ( B , P ) ”.

Original: LTL will likely play a key role in deﬁning the semantics of any such language, but additional concepts concerning plan structure are needed, such as the ability to specify that an action is part of the causal support for a goal or subgoal.
Revised: LTL will likely play a key role in deﬁning the semantics of any such language, but additional concepts concerning plan structure are needed, such as the ability to specify that an action is part of the causal support for a goal or subgoal.

Original: • Compiling constraints into the HModel.
Revised: • Compiling constraints into the HModel.

Original: We showed examples of how a constraint derived from a user question could be compiled to form an HModel.
Revised: We showed examples of how a constraint derived from a user question could be compiled to form an HModel.

Original: However, providing compilations for more general constraints (like the one above) and ensuring their correctness is an important issue.
Revised: However, providing compilations for more general constraints (like the one above) and ensuring their correctness is an important issue.

Original: Additionally, the compilation can lead to producing plans which might differ from the original plan  unrelated to the user question.
Revised: Additionally, the compilation can lead to producing plans which might differ from the original plan in ways unrelated to the user question.

Original: We believe that the work on planning with preferences (Gerevini, Saetti, and Serina 2006) and state-trajectory constraints (Baier et al.
Revised: We believe that the work on planning with preferences (Gerevini, Saetti, and Serina 2006) and state-trajectory constraints (Baier et al.

Original: ) is an important ﬁrst step, but does not yet address the full range of constraints needed.
Revised: ) is an important ﬁrst step, but does not yet address the full range of constraints needed.

Original: • Forming and presenting contrastive explanations.
Revised: • Forming and presenting contrastive explanations.

Original: The form of contrastive explanation we provide, as discussed in Section 3.4 and shown in the GUI in Fig.
Revised: The form of contrastive explanation we provide, as discussed in Section 3.4 and shown in the GUI in Fig.

Original: 12, is a very simple one that presents the original plan and HPlan and highlights the action differences between them.
Revised: 12, is a very simple one that presents the original plan and HPlan and highlights the action differences between them.

Original: Also, it is possible to obtain hierarchical contrastive explana-
Revised: Also, it is possible to obtain hierarchical contrastive explanations

Original: tions by asking consecutive question.
Revised: by asking consecutive questions.

Original: However, this does not show the causality of the plans, or the differences in their causal structure.
Revised: However, this does not show the causality of the plans, or the differences in their causal structure.

Original: Fig.
Revised: Fig.

Original: 13 shows a possible composite causal representation for both the original plan and the HPlan, with the differences shown in different colors.
Revised: 13 shows a possible composite causal representation for both the original plan and the HPlan, with the differences shown in different colors.

Original: This form of an explanation helps to elucidate how the two plans achieve, or fail to achieve, the (sub)goals of the problem with respect to speciﬁc actions in the domain.
Revised: This way of visualising the explanation can help to elucidate how the two plans achieve, or fail to achieve, the (sub)goals of the problem with respect to speciﬁc actions in the domain.

Original: However, for larger  more complex plans we expect that some form of abstraction will be necessary in order to effectively compare and contrast plans; the user only wants to see the important differences between two plans, not allof the details.
Revised: However, for larger and more complex plans we expect that some form of abstraction will be necessary in order to effectively compare and contrast plans; the user might wants to see the important differences between two plans, not all the details.

Original: What counts as details remains an open research question, but is likely related to action costs and to the ease and importance of achieving various subgoals.
Revised: What counts as details remains an open research question, but is likely related to action costs and to the ease and importance of achieving various subgoals.

Original: Sreedharan, Srivastava, and Kambhampati (2018) have done some initial work in this area, and have considered milestones as important abstractions for purposes of explanation.
Revised: Sreedharan, Srivastava, and Kambhampati (2018) have done some initial work in this area, and have considered milestones as important abstractions for purposes of explanation.

Original: The issues of what constitutes a good explanation, and how to visualize it or present it remain intertwined.
Revised: The issues of what constitutes a good explanation, and how to visualize it or present it remain intertwined.

Original: Some synergy between researchers in planning, data visualization (e.g., Chakraborti et al.
Revised: Some synergy between researchers in planning, data visualization (e.g., Chakraborti et al.

Original: (2018) or Mennatallah et al.
Revised: (2018) or Mennatallah et al.

Original: (2018)), and social sciences (Miller 2019) would be fruitful.
Revised: (2018)), and social sciences (Miller 2019) would be fruitful.

Original: • Providing explanations for complex questions.
Revised: • Providing explanations for complex questions.

Original: In the presented approach, a user is able to iteratively ask questions to reﬁne the explanation.
Revised: In the presented approach, a user is able to iteratively ask questions to reﬁne the explanation.

Original: If the explanation does not satisfy the user, or the question they have is more complex, this approach can provide the user with a deeper understanding.
Revised: If the explanation does not satisfy the user, or the question they have is more complex, this approach can provide the user with a deeper understanding.

Original: However, this process could be automated by analysing a more complex question the user might have, and decomposing it into several formal questions.
Revised: However, this process could be automated by analysing a more complex question the user might have, and decomposing it into several formal questions.

Original: In this case new constraints can be added to the HModel automatically until the explanation addresses the intended question and potentially the context it was asked within.
Revised: In this case new constraints can be added to the HModel automatically until the explanation addresses the intended question and potentially the context it was asked within.

Original: • Assessing the effectiveness of explanations.
Revised: • Assessing the effectiveness of explanations.

Original: We believe it is crucial to be able to acquire evidence of the effectiveness of an explanation.
Revised: We believe it is crucial to be able to acquire evidence of the effectiveness of an explanation.

Original: In particular, if engendering trust is the motivation for Explainable Planning and XAI in general, then we should look at the actual experience of the usersand check whether they
Revised: In particular, if engendering trust is the motivation for Explainable Planning and XAI in general, then we should look at the actual experience of the users

Original:  gain conﬁdence in the planner or not.
Revised: and check whether they gain conﬁdence in the planner or not.

Original: For this, a vital step for planning researchers is to include user studies to assess the effectiveness of the explanations they are providing.
Revised: For this, a vital step for planning researchers is to include user studies to assess the effectiveness of the explanations they are providing.

Original: While of course this is not an exhaustive list of all the necessary next steps, it already provides an interesting set of challenges that should be addressed.
Revised: While of course this is not an exhaustive list of all the necessary next steps, it already provides an interesting set of challenges that should be addressed.

Original: Note that while we are advocating for Explainable Planning as a service, we are well aware that this is not the only way to provide explanations for planning.
Revised: Note that while we are advocating for Explainable Planning as a service, we are well aware that this is not the only way to provide explanations for planning.

Original: In particular, we envisage at least the following possible limitations that nevertheless represent important research questions that should be considered by the Explainable Planning community.
Revised: In particular, we envisage at least the following possible limitations that nevertheless represent important research questions that should be considered by the Explainable Planning community.

Original: First we acknowledge that contrastive explanations are not suitable to answer every type of question that the user might have.
Revised: First we acknowledge that contrastive explanations are not suitable to answer every type of question that the user might have.

Original: However, we argue that contrastive questions are common and that contrastive explanations therefore play a signiﬁcant role, as also acknowledged by other researchers in Explainable AI, e.g, (Miller 2018).
Revised: However, we argue that contrastive questions are common and that contrastive explanations therefore play a signiﬁcant role, as also acknowledged by other researchers in Explainable AI, e.g, (Miller 2018).

Original: Second, by lifting the requirement that the explanation is generated by the planner used to generate the original plan, it could be possible to modify the search procedure used by the planner to generate explanations from a wider set of constraints.
Revised: Second, by lifting the requirement that the explanation is generated by the planner used to generate the original plan, it could be possible to modify the search procedure used by the planner to generate explanations from a wider set of constraints.

Original: Third, we are assuming that there is no uncertainty in the original planning model and that the model is correct.
Revised: Third, we are assuming that there is no uncertainty in the original planning model and that the model is correct.

Original: However, this is not always the case and
Revised: However, this is not always the case and

Original: [Figure]
Revised: [Figure]

Original: for this, the body of research on model reconciliation plays a very important role (Chakraborti et al. 2017).
Revised: for this, the body of research on model reconciliation plays a very important role (Chakraborti et al. 2017).

Original:  Conclusions
Revised:  Conclusions

Original: In this paper we have presented a  framework forExplainable
Revised: In this paper we have presented a prototype framework for

Original:  Planning as a service, which we believe represents an effective way of providing explanations, particularly in safety critical domains.
Revised: Explainable Planning as a service, which we believe represents an effective way of providing explanations, particularly in safety critical domains.

Original: In such scenarios the user would not accept an explanation that is generated by a planner or a model different from the ones that they use and whose performance they trust.
Revised: In such scenarios the user would not accept an explanation that is generated by a planner or a model different from the ones that they use and whose performance they trust.

Original: To this end, Explainable Planning as a service is based on providing explanations using the users’ planners and models.
Revised: To this end, Explainable Planning as a service is based on providing explanations using the users’ planners and models.

Original: Note that while the users can trust their planners and their models, there might still be reasonable questions on why a particular plan was found.
Revised: Note that while the users can trust their planners and their models, there might still be reasonable questions on why a particular plan was found.

Original: This is where explanations are important, and we propose contrastive explanations to allow the user to compare the plan found by the planner with what the user was expecting.
Revised: This is where explanations are important, and we propose contrastive explanations to allow the user to compare the plan found by the planner with what the user was expecting.

Original: The effectiveness of the proposed approach is shown through the implemented framework.

Original: The Explainable Planning as a Service framework is modular androbust. It can be used with all planners and domains that adhere to PDDL2.1.
Revised: The Explainable Planning as a Service framework is modular, and can be used with all planners and domains that ahere to PDDL2.1.

Revised: In order to foster the use of Explainable

Revised: Planning in robotics applications the proposed framework is now being integrated in ROSPlan (Cashmore et al. 2015).

Original: We proposed a roadmap with some of the challenges that should be addressed for this framework to become effective, also highlighting promising directions.
Revised: We proposed a roadmap with some of the challenges that should be addressed for Explainable Planning to become effective, also highlighting promising directions.

Original: We believe that synergies between researchers in planning and in other disciplines, such as data visualization, social science, humancomputer-interaction, and cognitive science, are key for the practical success of Explainable Planning.
Revised: We believe that synergies between researchers in planning and in other disciplines, such as data visualization, social science, human-computer-interaction, and cognitive science, are key for the practical success of Explainable Planning.

Original: Acknowledgements This work was partially supported by
Revised: Acknowledgements This work was partially supported by

Original: Innovate UK grant 133549: Intelligent Situational Awareness Platform , and by EPSRC grant EP/R033722/1: Trust in
Revised: Innovate UK grant 133549: Intelligent Situational Awareness Platform , and by EPSRC grant EP/R033722/1: Trust in

Original: Human-Machine Partnerships .
Revised: Human-Machine Partnerships .