Original: A General Framework for Synthesizing and Executing Self-Explaining Plans for Human-AI Interaction
Revised: A General Framework for Synthesizing and Executing Self-Explaining Plans for Human-AI Interaction

Original: Anonymous

Original: Abstract
Revised: Abstract

Original: In this work, we present a general formulation for decision making in human-in-the-loop planning problems where the human’s expectations about an autonomous agent may differ from the agent’s own model.
Revised: In this work, we present a general formulation for decision making in human-in-the-loop planning problems where the human’s expectations about an autonomous agent may differ from the agent’s own model.

Original: We show how our formulation for such multi-model planning problems allows us to capture existing approaches to this problem and also be used to generate novel explanatory behaviors.
Revised: We show how our formulation for such multi-model planning problems allows us to capture existing approaches to this problem and also be used to generate novel explanatory behaviors.

Original: Our formulation also reveals a deep connection between multi-model planning and epistemic planning and we show how we can leverage classical planning compilations designed for epistemic planning for solving multi-model planning problems.
Revised: Our formulation also reveals a deep connection between multi-model planning and epistemic planning and we show how we can leverage classical planning compilations designed for epistemic planning for solving multi-model planning problems.

Original: We empirically show how this new compilation provides a computational advantage over previous approaches that separate reasoning about model reconciliation and identifying the agent’s plan.
Revised: We empirically show how this new compilation provides a computational advantage over previous approaches that separate reasoning about model reconciliation and identifying the agent’s plan.

Original:  Introduction
Revised:  Introduction

Original: As automated agents and users start working closely together, it becomes increasingly important that the agents are capable of acting in a manner that is intuitive and explicable to users in the loop.
Revised: As automated agents and users start working closely together, it becomes increasingly important that the agents are capable of acting in a manner that is intuitive and explicable to users in the loop.

Original: A major challenge to achieving such ﬂuent collaboration is the fact that the human’s expectations regarding the agent’s capabilities and preferences may differ from reality.
Revised: A major challenge to achieving such ﬂuent collaboration is the fact that the human’s expectations regarding the agent’s capabilities and preferences may differ from reality.

Original: Such knowledge asymmetry implies that even in cases where the human teammate is a passive observer, the agent can no longer solely reason with their individual models to generate desirable plans.
Revised: Such knowledge asymmetry implies that even in cases where the human teammate is a passive observer, the agent can no longer solely reason with their individual models to generate desirable plans.

Original: Instead, the agent needs to explicitly take into account the user’s expectations about the agent when coming up with its plans.
Revised: Instead, the agent needs to explicitly take into account the user’s expectations about the agent when coming up with its plans.

Original: Previous works have mostly focused on two strategies to handle such scenarios, (a) explanation - the agent chooses to perform its optimal plan and explain the effectiveness of the chosen plan;  (b) explicable planning - the agent chooses to follow viable plans that are closest to user’s expectations.
Revised: Previous works have mostly focused on two strategies to handle such scenarios, (a) explanation - the agent chooses to perform its optimal plan and explain the effectiveness of the chosen plan; (c.f [Chakraborti et al., 2017]) (b) explicable planning - the agent chooses to follow viable plans that are closest to user’s expectations (c.f [Zhang et al., 2017]).

Original: In the end, we would want an approach that is able to combine the strengths of these two strategies.
Revised: In the end, we would want an approach that is able to combine the strengths of these two strategies.

Original: This would require the agent to move away from standard notions of decision making where the agent is solely trying to optimize the cost of the plan it will follow, but also take into account the ease of explaining the plan as one of the criteria for choosing its actions.
Revised: This would require the agent to move away from standard notions of decision making where the agent is solely trying to optimize the cost of the plan it will follow, but also take into account the ease of explaining the plan as one of the criteria for choosing its actions.

Original: We will refer to the problem of generating such plans as multi-model planning .
Revised: We will refer to the problem of generating such plans as multi-model planning .

Original: In this work, we will present a general characterization of the problem of multi-model planning and discuss how we could view the solutions to such multi-model planning problems as self-explaining plans , where explanations are themselves provided through robot actions.
Revised: In this work, we will present a general characterization of the problem of multi-model planning and discuss how we could view the solutions to such multi-model planning problems as self-explaining plans , where explanations are themselves provided through robot actions.

Original: These actions could be purely communicative actions that are meant to update the human’s mental model or task level actions that could also have epistemic side effects.
Revised: These actions could be purely communicative actions that are meant to update the human’s mental model or task level actions that could also have epistemic side effects.

Original: The contributions of this paper include, • Presenting a formalization of multi-model planning that allows us to characterize solutions identiﬁed by earlier works (Section 2).
Revised: The contributions of this paper include, • Presenting a formalization of multi-model planning that allows us to characterize solutions identiﬁed by earlier works (Section 2).

Original: • We look at two additional problem considerations; the phase of interaction (is the explanation occurring during plan selection or is it happening during plan execution); the attentiveness of the user (Section 3) and discuss how such considerations poses both new challenges and provides us with opportunities to generate novel behaviors.
Revised: • We look at two additional problem considerations; the phase of interaction (is the explanation occurring during plan selection or is it happening during plan execution); the attentiveness of the user (Section 5) and discuss how such considerations poses both new challenges and provides us with opportunities to generate novel behaviors.

Original: • We present a new planning compilation to solve such planning problems that allows for a uniform treatment of explanation and task level actions.
Revised: • We present a new planning compilation to solve such planning problems that allows for a uniform treatment of explanation and task level actions.

Original: We empirically show how such a compilation could provide a computational advantage (Sections 5 and 6).
Revised: We empirically show how such a compilation could provide a computational advantage (Sections 6 and 7).

Original: We will use Urban Search and Rescue domain as a running example to motivate and illustrate the discussions.
Revised: We will use Urban Search and Rescue domain as a running example to motivate and illustrate the discussions.

Original:  Multi-Model Planning and Explanation as Model Reconciliation
Revised:  Multi-Model Planning and Explanation as Model Reconciliation

Original: The planning models used by both the human and the robot are described by the tuple M = (cid:104) F, A, I, G (cid:105) .
Revised: The planning models used by both the human and the robot are described by the tuple M = (cid:104) F, A, I, G (cid:105) .

Original: In this formulation, F is the set of propositional ﬂuents used to describe the planning task states, A the set of actions, I the initial state and
Revised: In this formulation, F is the set of propositional ﬂuents used to describe the planning task states, A the set of actions, I the initial state and

Original: G the goal.
Revised: G the goal.

Original: Each action a ∈ A is further deﬁned as a tuple of the form a = (cid:104) prec a , adds a , dels a (cid:105) , where prec a lists the preconditions of the action and adds a and dels a provides the add and delete effects of the action.
Revised: Each action a ∈ A is further deﬁned as a tuple of the form a = (cid:104) prec a , adds a , dels a (cid:105) , where prec a lists the preconditions of the action and adds a and dels a provides the add and delete effects of the action.

Original: In general, the precondition can be some logical formula deﬁned over state ﬂuents and an action a can only be executed in a state S if S |
Revised: In general, the precondition can be some logical formula deﬁned over state ﬂuents and an action a can only be executed in a state S if S |

Original: = prec a .
Revised: = prec a .

Original: The effects are generally of the form c → e , where the antecedent represents the condition under which the effect e should be applied (where the ﬂuent corresponding to e is set true in the state if c → e is part of the add effects and if it is part of the delete it is set to false).
Revised: The effects are generally of the form c → e , where the antecedent represents the condition under which the effect e should be applied (where the ﬂuent corresponding to e is set true in the state if c → e is part of the add effects and if it is part of the delete it is set to false).

Original: Each action is also associated with a cost (represented as C M ( a ) ).
Revised: Each action is also associated with a cost (represented as C M ( a ) ).

Original: A plan in this setting is deﬁned as a sequence of actions ( π = (cid:104) a 1 , ..., a n (cid:105) ) and is said to be valid (denoted as π ( I ) | =
Revised: A plan in this setting is deﬁned as a sequence of actions ( π = (cid:104) a 1 , ..., a n (cid:105) ) and is said to be valid (denoted as π ( I ) | =

Original: M G ) for a model M if G ⊆ π ( I ) .
Revised: M G ) for a model M if G ⊆ π ( I ) .

Original: Each plan is additionally associated with a cost C M ( π ) such that
Revised: Each plan is additionally associated with a cost C M ( π ) such that

Original: C M ( π ) =
Revised: C M ( π ) =

Original: (cid:80) ni
Revised: (cid:80) ni

Original: =1 C M ( a i ) .
Revised: =1 C M ( a i ) .

Original: A plan π is said to be optimal if there exist no valid plan π (cid:48) such that C M ( π (cid:48) ) <
Revised: A plan π is said to be optimal if there exist no valid plan π (cid:48) such that C M ( π (cid:48) ) <

Original: C M ( π ) .
Revised: C M ( π ) .

Original: We will use Π ∗M to represent the set of all plans optimal for M .
Revised: We will use Π ∗M to represent the set of all plans optimal for M .

Original: In this setting we will assume that the robot uses a model
Revised: In this setting we will assume that the robot uses a model

Original: M R = (cid:104) F, A R ,
Revised: M R = (cid:104) F, A R ,

Original: I R , G R (cid:105) and the human evaluates the plan using a model M H =
Revised: I R , G R (cid:105) and the human evaluates the plan using a model M H =

Original: (cid:104) F, A H , I H , G H (cid:105) .
Revised: (cid:104) F, A H , I H , G H (cid:105) .

Original: For ease of discussion, we concentrate on the speciﬁc case where conditions for actions only consist of conjunction of positive literals and with no action cost difference between models.
Revised: For ease of discussion, we concentrate on the speciﬁc case where conditions for actions only consist of conjunction of positive literals and with no action cost difference between models.

Original: We start with the assumption that the robot is aware of M H and hence knows whether a given plan π R (that is optimal in M R ) is explicable or not, i.e, whether or not the human would identify π R to be one of the optimal plans for the given planning problem.
Revised: We start with the assumption that the robot is aware of M H and hence knows whether a given plan π R (that is optimal in M R ) is explicable or not, i.e, whether or not the human would identify π R to be one of the optimal plans for the given planning problem.

Original: In cases where the given plan may appear inexplicable, one way the robot could resolve the confusion would be by informing the human about its own model so they can correctly evaluate the current plan.
Revised: In cases where the given plan may appear inexplicable, one way the robot could resolve the confusion would be by informing the human about its own model so they can correctly evaluate the current plan.

Original: Thus an explanation ( E ) for this setting can be represented by a set of model updates (where E represents the set of all possible model updates).
Revised: Thus an explanation ( E ) for this setting can be represented by a set of model updates (where E represents the set of all possible model updates).

Original: The different types of model updates include –
Revised: The different types of model updates include –

Original: (1) Turn a ﬂuent p true or false in initial state (represented by the operator { add / remove } -p-from-I) (2) Add or remove a ﬂuent p from the precondition (also add or delete) list of an action a (represented by the operator { add / remove } -p-from-prec-of-a) (3) Add or remove a ﬂuent p from the goal list (represented by the operator { add / remove } -p-from-G)
Revised: (1) Turn a ﬂuent p true or false in initial state (represented by the operator { add / remove } -p-from-I) (2) Add or remove a ﬂuent p from the precondition (also add or delete) list of an action a (represented by the operator { add / remove } -p-from-prec-of-a) (3) Add or remove a ﬂuent p from the goal list (represented by the operator { add / remove } -p-from-G)

Original: We will use the function T : M × E → M to represent the transition function induced by the model update messages (where M represents the set of all possible models).
Revised: We will use the function T : M × E → M to represent the transition function induced by the model update messages (where M represents the set of all possible models).

Original: A set of model updates ( E ) explains a given plan if in the model resulting from applying the model updates ( ˆ M = T ( M H , E ) ) , the current plan is optimal (i.e., π R ∈ Π ∗ ˆ M ).
Revised: A set of model updates ( E ) explains a given plan if in the model resulting from applying the model updates ( ˆ M = T ( M H , E ) ) , the current plan is optimal (i.e., π R ∈ Π ∗ ˆ M ).

Original: Unless otherwise mentioned, when we refer to updated model , we are referring to this new human model obtained by applying the explanations.
Revised: Unless otherwise mentioned, when we refer to updated model , we are referring to this new human model obtained by applying the explanations.

Original: Once we have such a set of model updates, the ﬁnal explanation (presented to the explainee) can be generated by converting the model updates to corresponding natural language statements [Tellex et al., 2014] or through some appropriate visualization [Chakraborti, Sreedharan, and Kambhampati, 2018a].
Revised: Once we have such a set of model updates, the ﬁnal explanation (presented to the explainee) can be generated by converting the model updates to corresponding natural language statements [Tellex et al., 2014] or through some appropriate visualization [Chakraborti, Sreedharan, and Kambhampati, 2018a].

Original: Among the valid explanations for a given plan, we refer to the shortest explanation as the minimally complete explanation or MCE.
Revised: Among the valid explanations for a given plan, we refer to the shortest explanation as the minimally complete explanation or MCE.

Original: The original work [Chakraborti et al., 2017] on model reconciliation viewed the problem of generating MCE explanation as a problem of searching over the space of possible model updates that can be performed on the human model and the validity of each possible explanation was measured by checking the optimality of the plan in the corresponding planning problem.
Revised: The original work [Chakraborti et al., 2017] on model reconciliation viewed the problem of generating MCE explanation as a problem of searching over the space of possible model updates that can be performed on the human model and the validity of each possible explanation was measured by checking the optimality of the plan in the corresponding planning problem.

Original: In this work, we will be looking at the more general case where we are interested in identifying both the  agent plan and the relevant explanation, and we will refer to this problems as multi-model planning
Revised: Before we go into more details, let us brieﬂy look at the urban search and rescue (USAR) domain that will act as the running example for the rest of the paper.  Urban Search and Rescue USAR presents an ideal testbed for research on explainable planning as it looks at cases where the decision to follow suboptimal or in-executable plan can be potentially disastrous, yet limitations in communications capability could prevent the agents from providing detailed explanations. The basic scenario consists of an autonomous agent that has been deployed to the disaster scene and an external commander who is monitoring the activities of the robot.

Revised: Both agents start with the same model of the world (i.e the map of the building before the disaster) but the models diverge over time owing to the fact that robot has access to more accurate information about the current status of the building.

Revised: This model divergence could lead to the commander incorrectly evaluating valid robot plans as sub-optimal or unsafe.

Revised: One way to satisfy the commander would be to point out possible changes to it’s model that led the robot to come up with the plan in the ﬁrst place.

Revised: Figure 2 illustrates a typical scenario where the robot needs to travel from P1 to its goal at P15.

Revised: Here the human believes the robot should be moving to waypoint P6 and follow that corridor to go to P15, while the robot knows it should be moving to P7.

Revised: This disagreement rises from the fact that the human incorrectly believes that the path from P6 to P is clear while that from P8 to P12 is blocked.

Revised: If the robot were to follow the explanation scheme that was established in [Chakraborti et al., 2017] then the robot would stick to its own plan and provide the following explanation – > remove-

Revised: ( c

Revised: [Equation]

Revised: e a r p6 p5 )-from-I

Revised: ( i .

Revised: e . , Path from P6 to P5 i s blocked )

Revised: >

Revised: add-( c l e a r p8 p12 )-to-I ( i .

Revised: e . , Path from P8 to P12 i s c l e a r )

Revised:  Multi-Model Planning

Revised: In this work, we will be looking at the more general case where we are interested in identifying both the agent plan and the relevant explanation, and we will refer to this problems as multi-model planning

Original: Deﬁnition 1.
Revised: Deﬁnition 1.

Original: A multi-model planning problem is deﬁned by the tuple Ψ =
Revised: A multi-model planning problem is deﬁned by the tuple Ψ =

Original: (cid:104)M R , M H , T (cid:105) , where M H is the human model, M R the robot model and T is the transition function.
Revised: (cid:104)M R , M H , T (cid:105) , where M H is the human model, M R the robot model and T is the transition function.

Original: A solution to the problem Ψ is given by the tuple (cid:104)E Ψ , π Ψ (cid:105) , where E Ψ is set of model updates and π Ψ a plan, such that[Equation] and [Equation]
Revised: A solution to the problem Ψ is given by the tuple (cid:104)E Ψ , π Ψ (cid:105) , where E Ψ is set of model updates and π Ψ a plan, such that

Original: T

Original: [Equation]
Revised: [Equation]

Original:  M H [Equation] E Ψ [Equation]
Revised: T ( M H , E Ψ ) .

Original: The above problem only deals with cases where we care about establishing the validity of a robot plan in the human model.
Revised: The above problem only deals with cases where we care about establishing the validity of a robot plan in the human model.

Original: The above problem is already PSPACE-complete 1 , but we need to go beyond just ﬁnding valid solutions to ﬁnding explanations that establish the optimality of a robot plan in the human model.
Revised: The above problem is already PSPACE-complete 1 , but we need to go beyond just ﬁnding valid solutions to ﬁnding explanations that establish the optimality of a robot plan in the human model.

Original: Deﬁnition 2.
Revised: Deﬁnition 2.

Original: The tuple (cid:104)E Ψ , π Ψ (cid:105) is said to be a complete solution for the problem Ψ
Revised: The tuple (cid:104)E Ψ , π Ψ (cid:105) is said to be a complete solution for the problem Ψ

Original: if π Ψ ∈ Π ∗M R and π Ψ ∈ Π ∗T ( M H , E Ψ ) .
Revised: if π Ψ ∈ Π ∗M R and π Ψ ∈ Π ∗T ( M H , E Ψ ) .

Original: The use of the term complete here is in accordance with it’s use in [Chakraborti et al., 2017].
Revised: The use of the term complete here is in accordance with it’s use in [Chakraborti et al., 2017].

Original: E Ψ constitutes an MCE for a plan π Ψ when there exists no solution of the form (cid:104)E Ψ2 , π Ψ (cid:105) , such that C E ( E Ψ2 ) <
Revised: E Ψ constitutes an MCE for a plan π Ψ when there exists no solution of the form (cid:104)E Ψ2 , π Ψ (cid:105) , such that C E ( E Ψ2 ) <

Original: C E ( E Ψ ) , where C E is the cost of providing an explanation.
Revised: C E ( E Ψ ) , where C E is the cost of providing an explanation.

Original: The use of complete solutions are best suited for cases, where the agents choose to stick to a plan that is optimal and explain away any confusion an observer may have regarding the chosen plan.
Revised: The use of complete solutions are best suited for cases, where the agents choose to stick to a plan that is optimal and explain away any confusion an observer may have regarding the chosen plan.

Original: While the agent could choose to follow the optimal plan that is easiest to explain, in many scenario, communicating the explanation could still constitute a considerable expense on the agent’s part.
Revised: While the agent could choose to follow the optimal plan that is easiest to explain, in many scenario, communicating the explanation could still constitute a considerable expense on the agent’s part.

Original: It may in fact be more desirable for the agent to follow a sub-optimal plan if the choice of plan results in lower cost for transmitting the explanations.
Revised: It may in fact be more desirable for the agent to follow a sub-optimal plan if the choice of plan results in lower cost for transmitting the explanations.

Original: This is a trade-off we make frequently in our day-to-day life and it would be desirable for an explicable agent to be capable of balancing these two sources of cost (i.e the cost of the plan being followed and the cost of communicating the corresponding explanations)
Revised: This is a trade-off we make frequently in our day-to-day life and it would be desirable for an explicable agent to be capable of balancing these two sources of cost (i.e the cost of the plan being followed and the cost of communicating the corresponding explanations)

Original: [Chakraborti, Sreedharan, and Kambhampati, 2018b].
Revised: [ Chakraborti, Sreedharan, and Kambhampati, 2018b ] .

Original: Deﬁnition 3.
Revised: Deﬁnition 3.

Original: For a problem Ψ , the tuple (cid:104)E Ψ , π Ψ (cid:105) is said to be the optimal balanced explicable solution if,
Revised: For a problem Ψ , the tuple (cid:104)E Ψ , π Ψ (cid:105) is said to be the optimal balanced explicable solution if,

Revised: [Equation]

Revised: , such that the tuple satisﬁes (1.) and (2.),

Original: π Ψ
Revised: and

Original: ( I R ) | =
Revised: [Equation]

Original: G R . π Ψ ∈ Π ∗T ( M H , E Ψ ) . 3. (cid:64) (cid:104) ˆ E , ˆ π (cid:105) , such that  the tuple satisﬁes (1.) and (2.), and C ( ˆ E ) + C M R
Revised: In the USAR domain, to choose a balanced plan the robot can follow the path through P9 that also involves clearing the movable rubble between P9 and P10. In this case the robot only needs to provide a single explanation.

Original: (ˆ π ) <
Revised: > remove-

Original: C ( E Ψ ) + C M R ( π Ψ ) .
Revised: ( c l e a r p6 p5 )-from-I ( i .

Revised: e . , Path from P6 to P5 i s blocked )

Original: The original work that studied the problem of balancing (c.f [Chakraborti, Sreedharan, and Kambhampati, 2018b]) used an additional hyperparameter α , that captures the agent’s relative preference to providing explanations versus following a costlier explicable plan.
Revised: The original work that studied the problem of balancing (c.f [Chakraborti, Sreedharan, and Kambhampati, 2018b]) used an additional hyperparameter α , that captures the agent’s relative preference to providing explanations versus following a costlier explicable plan.

Original: In our formulation, we assume that such preferences are automatically captured by the cost of the explanation.
Revised: In our formulation, we assume that such preferences are automatically captured by the cost of the explanation.

Original: This formulation allows us to better capture the fact that certain aspects of the model may be harder to communicate than others.
Revised: This formulation allows us to better capture the fact that certain aspects of the model may be harder to communicate than others.

Original: Conceptually, this formulation should also allow us to include the effort required at the user’s end into the explanation cost, but capturing such considerations faithfully would require us to go beyond simple additive models cost explanation cost and so we will ignore them in the current work.
Revised: Conceptually, this formulation should also allow us to include the effort required at the user’s end into the explanation cost, but capturing such considerations faithfully would require us to go beyond simple additive models cost explanation cost and so we will ignore them in the current work.

Original: Note that the above formulation looks for solutions where the given plan is completely explicable to the user.
Revised: Note that the above formulation looks for solutions where the given plan is completely explicable to the user.

Original: In fact the above formulation doesn’t allow for the generation of purely explicable plans of the type discussed in [ Kulkarni et al., 2019; Zhang et al., 2017 ] , which tries to identify plans closest to the human optimal plan without relying on any explanations.
Revised: In fact the above formulation doesn’t allow for the generation of purely explicable plans of the type discussed in [Kulkarni et al., 2019; Zhang et al., 2017], which tries to identify plans closest to the human optimal plan without relying on any explanations.

Original: In many cases, generating such plans that may just be “good-enough” may be more desirable to the agent than always trying to stick to completely explainable plans (at the cost of providing more explanation).
Revised: In many cases, generating such plans that may just be “good-enough” may be more desirable to the agent than always trying to stick to completely explainable plans (at the cost of providing more explanation).

Original: We can do this by ensuring that the agent could include the extend of inexplicability of a given plan as yet another metric to consider when generating its plans rather than holding it as hard constraint.
Revised: We can do this by ensuring that the agent could include the extend of inexplicability of a given plan as yet another metric to consider when generating its plans rather than holding it as hard constraint.

Original: Assuming that the degree of inexplicability of the plan is directly proportional the degree of suboptimality of the plan in the human model, we can deﬁne an optimal balanced solution as follows.
Revised: Assuming that the degree of inexplicability of the plan is directly proportional the degree of suboptimality of the plan in the human model, we can deﬁne an optimal balanced solution as follows.

Original: Deﬁnition 4.
Revised: Deﬁnition 4.

Original: For a problem Ψ , the tuple (cid:104)E Ψ , π Ψ (cid:105) is said to be the optimal balanced solution if,
Revised: For a problem Ψ , the tuple (cid:104)E Ψ , π Ψ (cid:105) is said to be the optimal balanced solution if,

Original: π Ψ
Revised: π Ψ

Original: ( I R ) | =

Original: G R .

Original: (cid:64) (cid:104)

Original: ˆ E , ˆ π (cid:105) , such that the tuple satisfy (1.) and

Original: [Equation]
Revised: [Equation]

Original: [Equation] [Equation] [Equation] C Ψ T
Revised: G

Original: [Equation]
Revised: [Equation]

Original: M H [Equation] E ) ( π 
Revised: 2. (cid:64) (cid:104) ˆ E , ˆ π (cid:105) , such that the tuple satisfy (1.) and

Original: [Equation]
Revised: [Equation]

Original: where π ∗ 1 ∈ Π ∗T ( M H , ˆ E ) and π ∗ 1 ∈ Π ∗T ( M H , E Ψ ) and β captures the degree of penalty imposed by inexplicability of the given plan.
Revised: where π ∗ 1 ∈ Π ∗T ( M H , ˆ E ) and π ∗ 1 ∈ Π ∗T ( M H , E Ψ ) and β captures the degree of penalty imposed by inexplicability of the given plan.

Original: While the current work uses the difference of cost as the measure of inexplicability, one could have use other plan distances (or a combination) as standin for this measure.
Revised: While the current work uses the difference of cost as the measure of inexplicability, one could have use other plan distances (or a combination) as standin for this measure.

Original: This is especially helpful when the user’s model/preferences are not completely known.
Revised: This is especially helpful when the user’s model/preferences are not completely known.

Original:  Additional Considerations
Revised:  Additional Considerations

Original: Now that we have deﬁned some of the problems and solution concepts that have been covered in earlier works, we will now cover some additional problem considerations related to multi-model planning that has been previously ignored.
Revised: Now that we have deﬁned some of the problems and solution concepts that have been covered in earlier works, we will now cover some additional problem considerations related to multi-model planning that has been previously ignored.

Original: 3. The Stage of Interaction
Revised: 5. The Stage of Interaction

Original: An important factor we have more or less ignored in previous sections (and for that matter most of the previous works in model reconciliation) is whether the system is providing an explanation for a plan that has it has proposed or is the system providing post-hoc explanation for some plan that it has just executed (we will refer to such scenarios as explaining behaviors).
Revised: An important factor we have more or less ignored in previous sections (and for that matter most of the previous works in model reconciliation) is whether the system is providing an explanation for a plan that has it has proposed or is the system providing post-hoc explanation for some plan that it has just executed (we will refer to such scenarios as explaining behaviors).

Original: One may be tempted to argue that this distinction is unnecessary if we are limiting our attention to completely speciﬁed sequential plans in deterministic domains, but the fact that the user is viewing a plan being executed gives us opportunities to simplify explanation that may just not be available when the user and the system are deliberating over a proposed plan.
Revised: One may be tempted to argue that this distinction is unnecessary if we are limiting our attention to completely speciﬁed sequential plans in deterministic domains, but the fact that the user is viewing a plan being executed gives us opportunities to simplify explanation that may just not be available when the user and the system are deliberating over a proposed plan.

Original: In particular, we could use the agents actions (the fact they were successfully executed and their effects) to help shape the user’s perception about the task and the agent’s capabilities.
Revised: In particular, we could use the agents actions (the fact they were successfully executed and their effects) to help shape the user’s perception about the task and the agent’s capabilities.

Original: For example, the robot opening a door is enough to inform the human that the door was not locked in the ﬁrst place and does not require a separate communication action or the robot could go through a passage to show to the human that the passage was not blocked.
Revised: For example, the robot opening a door is enough to inform the human that the door was not locked in the ﬁrst place and does not require a separate communication action or the robot could go through a passage to show to the human that the passage was not blocked.

Original: Thus in these multi-model settings, we need to allow for the fact that these actions not only have an effect on the task level state (i.e ontic effects) but may also have epistemic effects on the user’s mental models.
Revised: Thus in these multi-model settings, we need to allow for the fact that these actions not only have an effect on the task level state (i.e ontic effects) but may also have epistemic effects on the user’s mental models.

Original: We could go one further step and treat all explanations as executing actions with epistemic effects.
Revised: We could go one further step and treat all explanations as executing actions with epistemic effects.

Original: This means that model-reconciliation planning is in fact an epistemic planning problem.
Revised: This means that model-reconciliation planning is in fact an epistemic planning problem.

Original: Previous works such as [ Muise et al., 2015 ] have shown how certain subsets of epistemic planning problem can be compiled to classical planning problems.
Revised: Previous works such as [Muise et al., 2015] have shown how certain subsets of epistemic planning problem can be compiled to classical planning problems.

Original: In the following sections, we will discuss how we can leverage similar compilation schemes to capture multi-model planning problems and show how such methods provide us with computational advantages over methods that separate reasoning about explanations and the task level plans.
Revised: In the following sections, we will discuss how we can leverage similar compilation schemes to capture multi-model planning problems and show how such methods provide us with computational advantages over methods that separate reasoning about explanations and the task level plans.

Original: 3. User’s Attention
Revised: 5. User’s Attention

Original: An assumption made by many of the earlier works is the fact that the human observer is a perfect listener.
Revised: An assumption made by many of the earlier works is the fact that the human observer is a perfect listener.

Original: Which means that once the explanation is provided she will deﬁnitely include it in her reasoning.
Revised: Which means that once the explanation is provided she will deﬁnitely include it in her reasoning.

Original: Unfortunately, this is not true in most cases.
Revised: Unfortunately, this is not true in most cases.

Original: The human’s ability to understand the explanation may depend on factors like the hardness of the concept being explained (for eg: explaining the robot’s reach in terms of motion constraints vs the ability for the robot to pick up heavy objects) and the mode of explanation (a simple visualization vs natural language) and even the cognitive load of the listener.
Revised: The human’s ability to understand the explanation may depend on factors like the hardness of the concept being explained (for eg: explaining the robot’s reach in terms of motion constraints vs the ability for the robot to pick up heavy objects) and the mode of explanation (a simple visualization vs natural language) and even the cognitive load of the listener.

Original: A more realistic approach would require us to model the fact that the user may not necessarily consider some information even when it is presented to them.
Revised: A more realistic approach would require us to model the fact that the user may not necessarily consider some information even when it is presented to them.

Original: The system may need to repeat its explanatory messages or use simpler explanations.
Revised: The system may need to repeat its explanatory messages or use simpler explanations.

Original: Computing such strategies would require us to move to formulations that allows for non-deterministic or stochastic transitions for explanatory messages.
Revised: Computing such strategies would require us to move to formulations that allows for non-deterministic or stochastic transitions for explanatory messages.

Original: In subsection 5.4, we discuss how our new formulation can be extended to support such scenarios.
Revised: In subsection 6.4, we discuss how our new formulation can be extended to support such scenarios.

Original: Before we discuss the actual compilation of model reconciliation planning problem, we will look at the urban search and rescue (USAR) domain that will act as the running example for the rest of the paper.

Original: Urban Search and Rescue

Original: USAR presents an ideal testbed for research on explainable planning as it looks at cases where the decision to follow suboptimal or in-executable plan can be potentially disastrous, yet limitations in communications capability could prevent the agents from providing detailed explanations.

Original: The basic scenario consists of an autonomous agent that has been deployed to the disaster scene and an external commander who is monitoring the activities of the robot.

Original: Both agents start with the same model of the world (i.e the map of the building before the disaster) but the models diverge over time owing to the fact that robot has access to more accurate information about the current status of the building.

Original: This model divergence could lead to the commander incorrectly evaluating valid robot plans as sub-optimal or unsafe.

Original: One way to satisfy the commander would be to point out possible changes to it’s model that led the robot to come up with the plan in the ﬁrst place.

Original: Figure 2 illustrates a typical scenario where the robot needs to travel from P1 to its goal at P15.

Original: Here the human believes the robot should be moving to waypoint P6 and follow that corridor to go to P15, while the robot knows it should be moving to P7.

Original: This disagreement rises from the fact that the human incorrectly believes that the path from P6 to P is clear while that from P8 to P12 is blocked.

Original: If the robot were to follow the explanation scheme that was established in [ Chakraborti et al., 2017 ] then the robot would stick to its own plan and provide the following explanation – > remove-( c l e a r p6 p5 )-from-I ( i .

Original: e . , Path from P6 to P5 i s blocked )

Original: >

Original: add-( c l e a r p8 p12 )-to-I ( i .

Original: e . , Path from P8 to P12 i s c l e a r )

Original: If communicating these explanation were expensive, the robot could also follow the approach detailed in [Chakraborti,

Original: Sreedharan, and Kambhampati, 2018b] and follow the path through P9 that also involves clearing the movable rubble between P9 and P10.

Original: In this case the robot only needs to provide a single explanation.

Original: > remove-

Original: ( c l e a r p6 p5 )-from-I ( i .

Original: e . , Path from P6 to P5 i s blocked )

Original:  Compilation
Revised:  Compilation

Original: To support planning with explanatory actions, we will adopt a formulation that is similar to the one introduced in [Muise et al., 2015] to compile reasoning about epistemic states into a classical planning problem.
Revised: To support planning with explanatory actions, we will adopt a formulation that is similar to the one introduced in [ Muise et al., 2015 ] to compile reasoning about epistemic states into a classical planning problem.

Original: In our setting, each explanatory actions can be viewed as an action with an epistemic effect.
Revised: In our setting, each explanatory actions can be viewed as an action with an epistemic effect.

Original: One interesting distinction to make here is the fact that the human’s belief state now not only includes their belief about the task state but also their belief about the robot’s model.
Revised: One interesting distinction to make here is the fact that the human’s belief state now not only includes their belief about the task state but also their belief about the robot’s model.

Original: This means that the planning model will need to separately keep track of (1) the current robot state, (2) the human’s belief regarding the current state, (3) how actions would effect each of these (as humans may have differing expectations about the effects of each action) and (4) how those expectations change with explanations.
Revised: This means that the planning model will need to separately keep track of (1) the current robot state, (2) the human’s belief regarding the current state, (3) how actions would effect each of these (as humans may have differing expectations about the effects of each action) and (4) how those expectations change with explanations.

Original: Given the model reconciliation planning problem Ψ =
Revised: Given the model reconciliation planning problem Ψ =

Original: (cid:104)M R , M H (cid:105) , we will generate a new planning model M Ψ
Revised: (cid:104)M R , M H (cid:105) , we will generate a new planning model M Ψ

Original: = (cid:104) F Ψ , A Ψ , IΨ , G Ψ (cid:105) as follows F Ψ =
Revised: [Equation] F Ψ , A [Equation] I

Original: [Equation]
Revised: [Equation]

Revised: G Ψ

Original:  where F B is a set of new ﬂuents that will be used to capture the human’s belief about the task state and F µ is a set of meta ﬂuents that we will use to capture the effects of explanatory actions and G
Revised: [Equation] as follows [Equation] where F B is a set of new ﬂuents that will be used to capture the human’s belief about the task state and F µ is a set of meta ﬂuents that we will use to capture the effects of explanatory actions and G

Original: and I are special goal and initial state propositions.
Revised: and I are special goal and initial state propositions.

Original: We will use the notation B ( p ) to capture the human’s belief about the ﬂuent p .
Revised: We will use the notation B ( p ) to capture the human’s belief about the ﬂuent p .

Original: We are able to use a single ﬂuent to capture the human belief as we are speciﬁcally dealing with a scenario where the human’s belief about the robot model is fully known.
Revised: We are able to use a single ﬂuent to capture the human belief as we are speciﬁcally dealing with a scenario where the human’s belief about the robot model is fully known.

Original: In this case, we also do not require any of the additional rules that were employed in [ Muise et al., 2015 ] to ensure that the state captures the deductive closure of the agent beliefs.
Revised: In this case, we also do not require any of the additional rules that were employed in [Muise et al., 2015] to ensure that the state captures the deductive closure of the agent beliefs.

Original: F µ will contain an element for every part of the human model that can be changed by the robot through explanations.
Revised: F µ will contain an element for every part of the human model that can be changed by the robot through explanations.

Original: A meta ﬂuent corresponding to a literal φ from the precondition of an action a takes the form of µ + ( φ prec a ) , where the superscript + refers to the fact that the clause φ is part the precondition of the action a in the robot model (for cases where the ﬂuent represents an incorrect human belief we will be using the superscript − ).
Revised: A meta ﬂuent corresponding to a literal φ from the precondition of an action a takes the form of µ + ( φ prec a ) , where the superscript + refers to the fact that the clause φ is part the precondition of the action a in the robot model (for cases where the ﬂuent represents an incorrect human belief we will be using the superscript − ).

Original: For every action a = (cid:104) prec a , adds a , dels a (cid:105) ∈
Revised: For every action a = (cid:104) prec a , adds a , dels a (cid:105) ∈

Original: A R and its human counterpart a h = prec a h , adds a h , dels a h ∈
Revised: A R and its human counterpart a h = prec a h , adds a h , dels a h ∈

Original: A H , we deﬁne a new action a Ψ =
Revised: A H , we deﬁne a new action a Ψ =

Original: (cid:104) prec a Ψ , adds a Ψ , dels a Ψ (cid:105) ∈
Revised: (cid:104) prec a Ψ , adds a Ψ , dels a Ψ (cid:105) ∈

Original: M Ψ whose precondition is given as –
Revised: M Ψ whose precondition is given as –

Original: [Equation]
Revised: [Equation]

Original: The important point to note here is that at any given state, an action in the augmented model is only applicable if the action is executable in robot model and the human believes the action to be executable.
Revised: The important point to note here is that at any given state, an action in the augmented model is only applicable if the action is executable in robot model and the human believes the action to be executable.

Original: Unlike the executability of the action in the robot model (captured through unconditional preconditions) the human’s beliefs about the action executability can be manipulated by turning the meta ﬂuents on and off.
Revised: Unlike the executability of the action in the robot model (captured through unconditional preconditions) the human’s beliefs about the action executability can be manipulated by turning the meta ﬂuents on and off.

Original: The effects of these actions can also be deﬁned similarly.
Revised: The effects of these actions can also be deﬁned similarly.

Original: In addition to these task level actions (represented by the set
Revised: In addition to these task level actions (represented by the set

Original: A τ ), we can also deﬁne explanatory actions ( A µ ) that either add µ + ( ∗ ) ﬂuents or delete µ − ( ∗ ) .
Revised: A τ ), we can also deﬁne explanatory actions ( A µ ) that either add µ + ( ∗ ) ﬂuents or delete µ − ( ∗ ) .

Original: Special actions a 0 and a ∞ that are responsible for setting all the initial state conditions true and checking the goal conditions are also added into the domain model.
Revised: Special actions a 0 and a ∞ that are responsible for setting all the initial state conditions true and checking the goal conditions are also added into the domain model.

Original: a 0 has a single precondition that checks for I and has the following add and delete effects –
Revised: a 0 has a single precondition that checks for I and has the following add and delete effects –

Original: [Equation]
Revised: [Equation]

Original: where F µ − is the subset of F µ that consists of all the ﬂuents of the form µ − ( ∗ ) .
Revised: where F µ − is the subset of F µ that consists of all the ﬂuents of the form µ − ( ∗ ) .

Original: Similarly, the precondition of action a ∞ is set using the original goal and adds the proposition G .
Revised: Similarly, the precondition of action a ∞ is set using the original goal and adds the proposition G .

Original: [Equation]
Revised: [Equation]

Original: Finally the new initial state and the goal speciﬁcation becomes I E
Revised: Finally the new initial state and the goal speciﬁcation becomes I E

Original: = {I} and G E
Revised: = {I} and G E

Original: = {G} respectively.
Revised: = {G} respectively.

Original: To see how such a compilation would look in practice, consider an action (move from p1 p2) that allows the robot to move from point p1 to p2 only if the path is clear.
Revised: To see how such a compilation would look in practice, consider an action (move from p1 p2) that allows the robot to move from point p1 to p2 only if the path is clear.

Original: The action is deﬁned as follows in the robot model.
Revised: The action is deﬁned as follows in the robot model.

Original: [Equation]
Revised: [Equation]

Revised: r p 1 p 2 ) ) :

Revised: e f f e c t ( and ( not ( a t p 1 ) )

Revised: ( a t p 2 ) ) )

Original: Let’s assume the human is aware of this action but doesn’t know that they need to care about the status of the path (as they assume the robot can move through any debris ﬁlled path).
Revised: Let’s assume the human is aware of this action but doesn’t know that they need to care about the status of the path (as they assume the robot can move through any debris ﬁlled path).

Original: In this case, the corresponding action in the augmented model and the relevant explanatory action will be ( : a c t i o n move from p1 p : p r e c o n d i t i o n ( and ( a t p 1 )
Revised: In this case, the corresponding action in the augmented model and the relevant explanatory action will be ( : a c t i o n move from p1 p : p r e c o n d i t i o n ( and ( a t p 1 )

Original: ( B ( ( a t p 1 ) ) )
Revised: ( B ( ( a t p 1 ) ) )

Original: ( c l e a r p 1 p 2 ) ( i m p l i e s ( µ + prec ( move from p1 p2 , ( c l e a r p 1 p 2 ) ) )
Revised: ( c l e a r p 1 p 2 ) ( i m p l i e s ( µ + prec ( move from p1 p2 , ( c l e a r p 1 p 2 ) ) )

Original: ( B ( ( c l e a r p 1 p 2 ) ) ) ) ) :
Revised: ( B ( ( c l e a r p 1 p 2 ) ) ) ) ) :

Original: e f f e c t ( and ( not ( a t p 1 ) )
Revised: e f f e c t ( and ( not ( a t p 1 ) )

Original: ( a t p 2 ) ( not B ( a t p 1 ) )
Revised: ( a t p 2 ) ( not B ( a t p 1 ) )

Original: B ( a t p 2 ) ) ) )
Revised: B ( a t p 2 ) ) ) )

Original: ( : a c t i o n e x p l a i n µ + prec move from clear : p r e c o n d i t i o n ( and ) :
Revised: ( : a c t i o n e x p l a i n µ + prec move from clear : p r e c o n d i t i o n ( and ) :

Original: e f f e c t ( and µ + prec ( move from p1 p2 , ( c l e a r p 1 p 2 ) ) ) )
Revised: e f f e c t ( and µ + prec ( move from p1 p2 , ( c l e a r p 1 p 2 ) ) ) )

Original: We will refer to an augmented model that contains an explanatory action for each possible model update and has no actions with effects on both the human’s mental model and the task level state as the canonical augmented model .
Revised: We will refer to an augmented model that contains an explanatory action for each possible model update and has no actions with effects on both the human’s mental model and the task level state as the canonical augmented model .

Original: Given an augmented model, let π E be some plan that is valid for this model ( π E ( I Ψ ) ⊆
Revised: Given an augmented model, let π E be some plan that is valid for this model ( π E ( I Ψ ) ⊆

Original: G Ψ ).
Revised: G Ψ ).

Original: From π E , we extract two types of information – the model updates induced by the actions in the plan (represented as E ( π E ) ) and the sequence of actions that have some effect of the task state (henceforth referred to as actions with ontic effects) represented as D ( π E ) .
Revised: From π E , we extract two types of information – the model updates induced by the actions in the plan (represented as E ( π E ) ) and the sequence of actions that have some effect of the task state (henceforth referred to as actions with ontic effects) represented as D ( π E ) .

Original: Note that E ( π E ) may contain effects from action in D ( π E ) .
Revised: Note that E ( π E ) may contain effects from action in D ( π E ) .

Original: This brings us to the following proposition –
Revised: This brings us to the following proposition –

Original: Proposition 1.
Revised: Proposition 1.

Original: Given a multi-model planning problem Ψ and the corresponding augmented model M Ψ , then for any plan [Equation]
Revised: Given a multi-model planning problem Ψ and the corresponding augmented model M Ψ , then for any plan π ( I Ψ ) | =

Original: Ψ G Ψ , the tuple (cid:104)E ( π ) , D ( π ) (cid:105) is a valid solution for Ψ .
Revised: Ψ G Ψ , the tuple (cid:104)E ( π ) , D ( π ) (cid:105) is a valid solution for Ψ .

Original: This result can be trivially shown to be true given the above formulation.
Revised: This result can be trivially shown to be true given the above formulation.

Original: Unfortunately, the compilation on it’s own only takes care of generating plans along with the justiﬁcations for correctness of the plan.
Revised: Unfortunately, the compilation on it’s own only takes care of generating plans along with the justiﬁcations for correctness of the plan.

Original: In many cases, the user would also be interested in understanding why the given plan is optimal.
Revised: In many cases, the user would also be interested in understanding why the given plan is optimal.

Original: Claim 1.
Revised: Claim 1.

Original: Given a multi-model planning problem Ψ and the corresponding augmented model M Ψ , then there exists a plan π ( I Ψ ) | =
Revised: Given a multi-model planning problem Ψ and the corresponding augmented model M Ψ , then there exists a plan π ( I Ψ ) | =

Original: Ψ G Ψ , such that the tuple (cid:104)E ( π ) , D ( π ) (cid:105) is a complete solution for Ψ but π may not be optimal for M Ψ .
Revised: Ψ G Ψ , such that the tuple (cid:104)E ( π ) , D ( π ) (cid:105) is a complete solution for Ψ but π may not be optimal for M Ψ .

Original: The ﬁrst part of the claim comes from the fact that the space of valid plans for M Ψ spans the entire set of valid plans for the robot model and the set of all possible model updates.
Revised: The ﬁrst part of the claim comes from the fact that the space of valid plans for M Ψ spans the entire set of valid plans for the robot model and the set of all possible model updates.

Original: On the other hand, due to the structure of the preconditions, for a given a set of model updates, there may be plans that are valid in the human model that never gets expanded.
Revised: On the other hand, due to the structure of the preconditions, for a given a set of model updates, there may be plans that are valid in the human model that never gets expanded.

Original: This means when the search comes up with a plan π ∗ that is optimal for
Revised: This means when the search comes up with a plan π ∗ that is optimal for

Original: M Ψ , it is possible that M H + E ( π ∗ ) could have plans cheaper than D ( π ∗ ) , that were not expanded as they were invalid in the robot model.
Revised: M Ψ , it is possible that M H + E ( π ∗ ) could have plans cheaper than D ( π ∗ ) , that were not expanded as they were invalid in the robot model.

Original: 5. Planning For Complete Solutions Using Augmented Model
Revised: 6. Planning For Complete Solutions Using Augmented Model

Original: The simplest way to generate complete solutions would be by updating the goal test.
Revised: A way to generate complete solutions would be by updating the goal test used by the search.

Original: Now for each valid plan in M Ψ , we would  need to check that the plan is in fact optimal in the updated model.
Revised: In addition to checking if the goal facts are indeed met in the resultant state, we would now also need to check that the plan is in fact optimal in the updated model.

Original: Note that this is an inversion of the search in [Chakraborti et al., 2017] with the added advantage that we are explicitly reasoning with explanatory actions and only check for plan optimality in human models that are guaranteed to support at least one robot executable plan.
Revised: Note that this is an inversion of the search in [ Chakraborti et al., 2017 ] with the added advantage that we are explicitly reasoning with explanatory actions and only check for plan optimality in human models that are guaranteed to support at least one robot executable plan.

Original: Furthermore, if we memoize the results of each secondary search with respect to E ( π ) , we can guarantee that the number of optimality tests will be less than or equal to the number of tests required by the earlier approach.
Revised: Furthermore, if we memoize the results of each secondary search with respect to E ( π ) , we can guarantee that the number of optimality tests will be less than or equal to the number of tests required by the earlier approach.

Original: Given the nature of this suboptimality test, it should be possible to leverage methods like search space reuse to speed up search, but since our focus here is on establishing the properties of the simplest formulation we will focus on cases that use a simple optimality test.
Revised: Given the nature of this suboptimality test, it should be possible to leverage methods like search space reuse to speed up search, but since our focus here is on establishing the properties of the simplest formulation we will focus on cases that use a simple optimality test.

Original: The next question to ask would be, under what conditions can this new encoding be guaranteed to generate explanations that are minimally complete.
Revised: The next question to ask would be, under what conditions can this new encoding be guaranteed to generate explanations that are minimally complete.

Original: Before we formally state the conditions, let us deﬁne a new concept called optimality gap (denoted as ∆ π M ) for a planning model, which captures the cost difference between the optimal plan and the second most optimal plan.
Revised: Before we formally state the conditions, let us deﬁne a new concept called optimality gap (denoted as ∆ π M ) for a planning model, which captures the cost difference between the optimal plan and the second most optimal plan.

Original: ∆ π M can be deﬁned as –
Revised: ∆ π M can be deﬁned as –

Original: [Equation]
Revised: [Equation]

Original: Theorem 1.
Revised: Theorem 1.

Original: Given a canonical augmented model M Ψ for a multi-model planning problem Ψ
Revised: Given a canonical augmented model M Ψ for a multi-model planning problem Ψ

Original: =
Revised: =

Original: (cid:104)M R , M H , T (cid:105) , if the sum of costs of all explanatory actions is less than or equal to ∆ π M R and if π is the cheapest valid plan for M Ψ such that
Revised: (cid:104)M R , M H , T (cid:105) , if the sum of costs of all explanatory actions is less than or equal to ∆ π M R and if π is the cheapest valid plan for M Ψ such that

Original: D ( π ) ∈ Π ∗T ( M Ψ , D ( π )) , then (1) D ( π ) is optimal for M R (2) E ( π ) is the MCE for D ( π ) (3) There exists no plan ˆ π ∈ Π ∗ R such that MCE for D (ˆ π ) is cheaper than E ( π ) ,i.e, the search will ﬁnd an optimal explicable plan if one exists.
Revised: D ( π ) ∈ Π ∗T ( M Ψ , D ( π )) , then (1) D ( π ) is optimal for M R (2) E ( π ) is the MCE for D ( π ) (3) There exists no plan ˆ π ∈ Π ∗ R such that MCE for D (ˆ π ) is cheaper than E ( π ) ,i.e, the search will ﬁnd an optimal explicable plan if one exists.

Original: Proof Sketch.
Revised: Proof Sketch.

Original: We observe that there exists no valid plan π (cid:48) for the augmented model ( M Ψ ) with a cost lower than that of π and where the ontic fragment ( D ( π (cid:48) ) ) is optimal for the human model.
Revised: We observe that there exists no valid plan π (cid:48) for the augmented model ( M Ψ ) with a cost lower than that of π and where the ontic fragment ( D ( π (cid:48) ) ) is optimal for the human model.

Original: Let’s assume D ( π ) (cid:54)∈ Π ∗R (i.e current plan’s ontic fragment is not optimal in robot model) and let ˆ π ∈ Π ∗R .
Revised: Let’s assume D ( π ) (cid:54)∈ Π ∗R (i.e current plan’s ontic fragment is not optimal in robot model) and let ˆ π ∈ Π ∗R .

Original: Now let’s consider the augmented plan corresponding to ˆ π , ˆ π E , i.e, E (ˆ π E ) is the MCE for the plan
Revised: Now let’s consider the augmented plan corresponding to ˆ π , ˆ π E , i.e, E (ˆ π E ) is the MCE for the plan

Original: ˆ π ) and D (ˆ π E )
Revised: ˆ π ) and D (ˆ π E )

Original: =
Revised: =

Original: ˆ π .
Revised: ˆ π .

Original: Then the given augmented plan ˆ π E is a valid solution for our augmented planning problem M Ψ (since the ˆ π E consists of the MCE for ˆ π , the plan must be valid and optimal in the human model), moreover the cost of ˆ π E must be lower than π .
Revised: Then the given augmented plan ˆ π E is a valid solution for our augmented planning problem M Ψ (since the ˆ π E consists of the MCE for ˆ π , the plan must be valid and optimal in the human model), moreover the cost of ˆ π E must be lower than π .

Original: This contradicts our earlier assumption hence we can show that D ( π ) is in fact optimal for the robot model.
Revised: This contradicts our earlier assumption hence we can show that D ( π ) is in fact optimal for the robot model.

Original: Using a similar approach we can also show that no cheaper explanation exists for π E and there exists no other plan with a cheaper explanation.
Revised: Using a similar approach we can also show that no cheaper explanation exists for π E and there exists no other plan with a cheaper explanation.

Original: Also note, that while it is hard to ﬁnd the exact value for the optimality gap, we are guaranteed that the optimality gap is greater than or equal to one for domains with only unit cost actions or is guaranteed to be greater than or equal to ( C 2 − C 1 ) , where C 1 is the cost of the cheapest action and Cis the cost of the second cheapest action (i.e ∀ a, ( C M ( a ) <
Revised: Also note, that while it is hard to ﬁnd the exact value for the optimality gap, we are guaranteed that the optimality gap is greater than or equal to one for domains with only unit cost actions or is guaranteed to be greater than or equal to ( C 2 − C 1 ) , where C 1 is the cost of the cheapest action and Cis the cost of the second cheapest action (i.e ∀ a, ( C M ( a ) <

Original: C 2 → C M ( a ) =
Revised: C 2 → C M ( a ) =

Original: C 1 ) )
Revised: C 1 ) )

Original: 5. Balanced Plans
Revised: 6. Balanced Plans

Original: We can use the above formulation directly to obtain balanced explicable plans, we just no longer need to put any speciﬁc restriction on the cost of explanatory action.
Revised: We can use the above formulation directly to obtain balanced explicable plans, we just no longer need to put any speciﬁc restriction on the cost of explanatory action.

Original: To generate optimal balanced plans, we need to relax the requirement that the ﬁnal plan is optimal in the human model.
Revised: To generate optimal balanced plans, we need to relax the requirement that the ﬁnal plan is optimal in the human model.

Original: Instead we can incorporate the inexplicability penalty into the reasoning about the plan, by assigning the cost of a ∞ (the goal action) to be β times the difference between the optimal plan in the human model and the current plan.
Revised: Instead we can incorporate the inexplicability penalty into the reasoning about the plan, by assigning the cost of a ∞ (the goal action) to be β times the difference between the optimal plan in the human model and the current plan.

Original: When β is set to zero the problem would just identify the optimal plan corresponding to the original robot model and when β is set high enough the formulation just generates explicable balanced plans.
Revised: When β is set to zero the problem would just identify the optimal plan corresponding to the original robot model and when β is set high enough the formulation just generates explicable balanced plans.

Original: This can be guaranteed if beta is set higher that κ , where κ is some upper bound on plan length for the robot (that includes explanatory actions).
Revised: This can be guaranteed if beta is set higher that κ , where κ is some upper bound on plan length for the robot (that includes explanatory actions).

Original: On the other hand, if the cost of any explanatory actions is also higher than κ then the formulation will try to ﬁnd the plan that is closest to an optimal plan in the original human model and is still executable in the robot model.
Revised: On the other hand, if the cost of any explanatory actions is also higher than κ then the formulation will try to ﬁnd the plan that is closest to an optimal plan in the original human model and is still executable in the robot model.

Original: To see the use of balanced plans, lets revisit the urban search and rescue case.
Revised: To see the use of balanced plans, lets revisit the urban search and rescue case.

Original: Here, the optimal path for the robot to follow would be to go through P7, P8 and P12.
Revised: Here, the optimal path for the robot to follow would be to go through P7, P8 and P12.

Original: The human thinks the path should be the one through P6 and P5.
Revised: The human thinks the path should be the one through P6 and P5.

Original: Explaining the optimality of the path requires explaining that the path from P6 to P5 is blocked (which can be explained through the action explain obstructed P6 P5 ) and the path from P8 to P is clear (explained by explain away obstructed P8 P12 ).
Revised: Explaining the optimality of the path requires explaining that the path from P6 to P5 is blocked (which can be explained through the action explain obstructed P6 P5 ) and the path from P8 to P is clear (explained by explain away obstructed P8 P12 ).

Original: Let us assume that the application of each of these explanatory actions increases the total cost by ten.
Revised: Let us assume that the application of each of these explanatory actions increases the total cost by ten.

Original: The balanced explicable plan in this setting would be
Revised: The balanced explicable plan in this setting would be

Original: [Equation]
Revised: [Equation]

Original: →
Revised: move from p1 p

Original: [Equation]
Revised: [Equation]

Original:  → [Equation] →[Equation]
Revised: move from p9 p10 → move from p10 p11 → move from p11 p16 → move from p16 p15 →

Original: → [Table]
Revised: GOAL ACT

Original: If we try to identify an optimal balanced plan for β = 1 and a cost of 5 for the clear passage action, then the plan that would be generated would be –
Revised: If we try to identify an optimal balanced plan for β = 1 and a cost of 5 for the clear passage action, then the plan that would be generated would be –

Original: INIT ACT → move from p1 p9 → c l e a r p a s s a g e p 9 p 1 0 → move from p9 p10 → move from p10 p11 → move from p11 p16 → move from p16 p15 →
Revised: INIT ACT → move from p1 p9 → c l e a r p a s s a g e p 9 p 1 0 → move from p9 p10 → move from p10 p11 → move from p11 p16 → move from p16 p15 →

Original: GOAL ACT 5. Plans with Epistemic Side-effects
Revised: GOAL ACT 6. Plans with Epistemic Side-effects

Original: As mentioned earlier, in cases where the user is observing a plan being executed, even the agent’s non-explanatory actions could have effects on user’s mental model.
Revised: As mentioned earlier, in cases where the user is observing a plan being executed, even the agent’s non-explanatory actions could have effects on user’s mental model.

Original: We can easily incorporate this requirement by associating effects involving meta-ﬂuents into our task speciﬁc actions.
Revised: We can easily incorporate this requirement by associating effects involving meta-ﬂuents into our task speciﬁc actions.

Original: Such effects may be speciﬁed by domain experts or could be generated using heuristic rules.
Revised: Such effects may be speciﬁed by domain experts or could be generated using heuristic rules.

Original: For example, if an action is executed in a state where a precondition believed by the user is not met then that precondition should be removed from the human’s perceived model.
Revised: For example, if an action is executed in a state where a precondition believed by the user is not met then that precondition should be removed from the human’s perceived model.

Original: To illustrate the use of such explanatory actions in our encoding let us visit the USAR scenario and assume that the human thinks that the path from P8 to P12 is blocked and the one from P6 to P5 is free.
Revised: To illustrate the use of such explanatory actions in our encoding let us visit the USAR scenario and assume that the human thinks that the path from P8 to P12 is blocked and the one from P6 to P5 is free.

Original: Also in this setting, for explaining the status of passages (whether they are blocked or not) the robot can now use two actions, one a rather expensive explicit communication action, that sends the updated map information to the human or the robot can just visit the blocked passage and the human who is watching a video feed of robot actions will learn that the passage is blocked or clear.
Revised: Also in this setting, for explaining the status of passages (whether they are blocked or not) the robot can now use two actions, one a rather expensive explicit communication action, that sends the updated map information to the human or the robot can just visit the blocked passage and the human who is watching a video feed of robot actions will learn that the passage is blocked or clear.

Original: Thus the action descriptions for the move action will be – ( : a c t i o n move from P7 P : p r e c o n d i t i o n ( and ( r o b o t a t P 7 ) . . .
Revised: Thus the action descriptions for the move action will be – ( : a c t i o n move from P7 P : p r e c o n d i t i o n ( and ( r o b o t a t P 7 ) . . .

Original: ( B ( r o b o t a t P 7 ) ) ) :
Revised: ( B ( r o b o t a t P 7 ) ) ) :

Original: e f f e c t ( and ( r o b o t a t P 8 ) . . .
Revised: e f f e c t ( and ( r o b o t a t P 8 ) . . .

Original: ( B ( clear P8 P12 ) )
Revised: ( B ( clear P8 P12 ) )

Original: ( i n c r e a s e ( t o t a l − c o s t ) 1 ) ) )
Revised: ( i n c r e a s e ( t o t a l − c o s t ) 1 ) ) )

Original: With this new action the robot knows that as soon as it reaches P8 the human would know that the path from P8 to
Revised: With this new action the robot knows that as soon as it reaches P8 the human would know that the path from P8 to

Original: P12 is clear so it can continue on that path.
Revised: P12 is clear so it can continue on that path.

Original: So the new robot plan will be –
Revised: So the new robot plan will be –

Original: INIT ACT → move from p1 p7 → move from p7 p8 → move from p8 p12 → move from p12 p13 → e x p l a i n o b s t r u c t e d p 6 p 5 → move from p12 p13 → move from p13 p14 → move from p14 p15 → GOAL ACT
Revised: INIT ACT → move from p1 p7 → move from p7 p8 → move from p8 p12 → move from p12 p13 → e x p l a i n o b s t r u c t e d p 6 p 5 → move from p12 p13 → move from p13 p14 → move from p14 p15 → GOAL ACT

Original: 5. Plans for Inattentive Users
Revised: 6. Plans for Inattentive Users

Original: In this scenario, we can no longer assume that explanatory actions would have deterministic effects on user’s model and that means considering planning models that allow for nondeterministic or stochastic effects.
Revised: In this scenario, we can no longer assume that explanatory actions would have deterministic effects on user’s model and that means considering planning models that allow for nondeterministic or stochastic effects.

Original: Domain
Revised: Domain

Original: New Compilation
Revised: New Compilation

Original: Model Space Search cov.
Revised: Model Space Search cov.

Original: runtime cov.
Revised: runtime cov.

Original: runtime
Revised: runtime

Original: Blocksworld 13/ 569.
Revised: Blocksworld 13/ 569.

Original: Elevator 15/ 59.
Revised: Elevator 15/ 59.

Original: Gripper 5/ 2301.
Revised: Gripper 5/ 2301.

Original: Driverlog 4/ 2740.
Revised: Driverlog 4/ 2740.

Original: Satellite 2/ 3186.
Revised: Satellite 2/ 3186.

Original: To see a simple example of how this would look, consider the USAR domain and look at the ability of the move action to inform the commander about the status of the passage from P8 to P12.
Revised: To see a simple example of how this would look, consider the USAR domain and look at the ability of the move action to inform the commander about the status of the passage from P8 to P12.

Original: The robot can not always guarantee that the commander would be looking at the screen and there is a chance that the commander won’t be looking at the screen when the path is presented.
Revised: The robot can not always guarantee that the commander would be looking at the screen and there is a chance that the commander won’t be looking at the screen when the path is presented.

Original: Thus in the new deﬁnition of action (move from P7 P8) we will replace the effect that adds
Revised: Thus in the new deﬁnition of action (move from P7 P8) we will replace the effect that adds

Original: B (clear P8 P12) with the effect (one-off ( B (clear P8 P12)) (and))
Revised: B (clear P8 P12) with the effect (one-off ( B (clear P8 P12)) (and))

Original: Which means that the action’s ability to update the human model is a non-deterministic effect.
Revised: Which means that the action’s ability to update the human model is a non-deterministic effect.

Original: In this case, we can look for a conformant plan by converting the earlier search into a search over belief space.
Revised: In this case, we can look for a conformant plan by converting the earlier search into a search over belief space.

Original: A search node only passes the goal test if the goal condition are met in every state in the belief space.
Revised: A search node only passes the goal test if the goal condition are met in every state in the belief space.

Original: Thankfully, this particular problem does have a conformant solution – e x p l a i n o b s t r u c t e d p 6 p 5 → e x p l a i n c l e a r p 8 p 1 2 → INIT ACT → move from p1 p7 → move from p7 p8 → move from p8 p12 → move from p12 p13 → move from p13 p14 → move from p14 p15 →
Revised: Thankfully, this particular problem does have a conformant solution – e x p l a i n o b s t r u c t e d p 6 p 5 → e x p l a i n c l e a r p 8 p 1 2 → INIT ACT → move from p1 p7 → move from p7 p8 → move from p8 p12 → move from p12 p13 → move from p13 p14 → move from p14 p15 →

Original: GOAL 
Revised: GOAL ACT

Original: ACT  Empirical Comparison of Model-Space Search and Planning with Explanatory Actions
Revised:  Empirical Comparison of Model-Space Search and Planning with Explanatory Actions

Original: The focus of this section is to see how our compilation compares with the approaches that separate the reasoning about explanations and plan generation.
Revised: The focus of this section is to see how our compilation compares with the approaches that separate the reasoning about explanations and plan generation.

Original: In particular, we will consider the approaches discussed in [ Chakraborti, Sreedharan, and Kambhampati, 2018b] as a point of comparison.
Revised: In particular, we will consider the approaches discussed in [ Chakraborti, Sreedharan, and Kambhampati, 2018b] as a point of comparison.

Original: Note that in order for the model space search to always identify the optimal balanced explicable plan, generating an optimal plan at each possible model is not enough.
Revised: Note that in order for the model space search to always identify the optimal balanced explicable plan, generating an optimal plan at each possible model is not enough.

Original: The approach would require iterating over the space of all possible optimal plans at a given node to ﬁnd one that is executable in the robot model or require involved compilations that only produce optimal plans that are executable in robot model.
Revised: The approach would require iterating over the space of all possible optimal plans at a given node to ﬁnd one that is executable in the robot model or require involved compilations that only produce optimal plans that are executable in robot model.

Original: To avoid changing the method too much, we used an optimistic version of the model space search that only identiﬁes one optimal plan per search node and the search ends as soon as it ﬁnd a node where the optimal plan produced has the same cost as the robot’s plan and is executable in the robot model.
Revised: To avoid changing the method too much, we used an optimistic version of the model space search that only identiﬁes one optimal plan per search node and the search ends as soon as it ﬁnd a node where the optimal plan produced has the same cost as the robot’s plan and is executable in the robot model.

Original: For comparison, we selected ﬁve IPC domains and for each domain, we created three unique models by introducing random updates in the model (except in the case of gripper and driverlog where only 5 were removed).
Revised: For comparison, we selected ﬁve IPC domains and for each domain, we created three unique models by introducing random updates in the model (except in the case of gripper and driverlog where only 5 were removed).

Original: Each of these three domains were paired with ﬁve problem instances and then tested on each of the possible conﬁgurations.
Revised: Each of these three domains were paired with ﬁve problem instances and then tested on each of the possible conﬁgurations.

Original: Each instance was run with a limit of 30 minutes, all explanatory actions were restricted to the beginning of the plan and the cost of explanatory actions were set to be twice the cost of original action.
Revised: Each instance was run with a limit of 30 minutes, all explanatory actions were restricted to the beginning of the plan and the cost of explanatory actions were set to be twice the cost of original action.

Original: Table 3 lists the time taken to solve each of these problems.
Revised: Table 3 lists the time taken to solve each of these problems.

Original: For calculating the average runtime, we used 1800 secs as the stand in for the runtime of all the instances that timed out.
Revised: For calculating the average runtime, we used 1800 secs as the stand in for the runtime of all the instances that timed out.

Original: We used h max as the heuristic for all the conﬁgurations.
Revised: We used h max as the heuristic for all the conﬁgurations.

Original: As clearly apparent from the table, the new approach does better than the original method for generating balanced plans for most of the domains.
Revised: As clearly apparent from the table, the new approach does better than the original method for generating balanced plans for most of the domains.

Original: Gripper seems to be the only domain, where model search seem to be doing better but this is also a domain that had the smallest number of model differences.
Revised: Gripper seems to be the only domain, where model search seem to be doing better but this is also a domain that had the smallest number of model differences.

Original: This points to the fact that the ability to leverage planning heuristics seems to make a marked difference in domains with a large number of possible explanatory actions.
Revised: This points to the fact that the ability to leverage planning heuristics seems to make a marked difference in domains with a large number of possible explanatory actions.

Original:  Related Work
Revised:  Related Work

Original: It’s widely accepted in social sciences literature that explanations must be generated while keeping in mind the beliefs of the agent receiving the explanation [ Miller, 2018; Slugoski et al., 1993 ] .
Revised: It’s widely accepted in social sciences literature that explanations must be generated while keeping in mind the beliefs of the agent receiving the explanation [ Miller, 2018; Slugoski et al., 1993 ] .

Original: As such, epistemic planning makes for an excellent framework for studying the problem of generating these explanations.
Revised: As such, epistemic planning makes for an excellent framework for studying the problem of generating these explanations.

Original: While the most general formulation of epistemic planning has been shown to be undecidable, many simpler fragments has been identiﬁed [ Bolander, Jensen, and
Revised: While the most general formulation of epistemic planning has been shown to be undecidable, many simpler fragments have been identiﬁed [ Bolander, Jensen, and

Original: Schwarzentruber, 2015 ] .
Revised: Schwarzentruber, 2015 ] .

Original: Recently, there have been a lot of interest in developing efﬁcient methods for planning in such domains [ Muise et al., 2015; Kominis and Geffner, 2015; 2017;
Revised: Recently, there have been a lot of interest in developing efﬁcient methods for planning in such domains [ Muise et al., 2015; Kominis and Geffner, 2015; 2017;

Original: Le et al., 2018; Huang et al., 2018 ] .
Revised: Le et al., 2018; Huang et al., 2018 ] .

Original: In our base scenario, we will assume (1) a ﬁnite nesting of beliefs, (2) the human is merely an observer, and (3) all actions are public.
Revised: In our base scenario, we will assume (1) a ﬁnite nesting of beliefs, (2) the human is merely an observer, and (3) all actions are public.

Original: The speciﬁc problems discussed in our paper hardly exercises most of the capabilities provided by epistemic planning.
Revised: The speciﬁc problems discussed in our paper hardly exercises most of the capabilities provided by epistemic planning.

Original: It’s important to note that given the epistemic nature of the explanatory actions, solving the general model reconciliation problem would require leveraging all those capabilities.
Revised: It’s important to note that given the epistemic nature of the explanatory actions, solving the general model reconciliation problem would require leveraging all those capabilities.

Original: Our hope is that by presenting model reconciliation in this more general setting, the community would be motivated to start looking at more general and complex versions of these problems.
Revised: Our hope is that by presenting model reconciliation in this more general setting, the community would be motivated to start looking at more general and complex versions of these problems.

Original: Our work also looks at the use of explanatory actions as a means of communicating information to the human observer.
Revised: Our work also looks at the use of explanatory actions as a means of communicating information to the human observer.

Original: The most obvious types of such explanatory action includes purely communicative actions such as speech [Tellex et al., 2014] or the use of mixed reality projections [Chakraborti, Sreedharan, and Kambhampati, 2018a;
Revised: The most obvious types of such explanatory action includes purely communicative actions such as speech [Tellex et al., 2014] or the use of mixed reality projections [Chakraborti, Sreedharan, and Kambhampati, 2018a;

Original: Ganesan, 2017], but recent works have shown that physical agents could also use movements to relay information such as intention [MacNally et al., 2018; Dragan, Lee, and Srinivasa, 2013] and incapability [Kwon, Huang, and Dragan, 2018].
Revised: Ganesan, 2017], but recent works have shown that physical agents could also use movements to relay information such as intention [MacNally et al., 2018; Dragan, Lee, and Srinivasa, 2013] and incapability [Kwon, Huang, and Dragan, 2018].

Original: Our framework could be easily adopted to any of these explanatory actions and would naturally allow for a trade-off between these different types of communication.
Revised: Our framework could be easily adopted to any of these explanatory actions and would naturally allow for a trade-off between these different types of communication.

Original: Many recent works dealing with explanation generation for planning, have looked at characterizing explanation in terms of the types of questions they answer (c.f [Fox, Long, and
Revised: Many recent works dealing with explanation generation for planning, have looked at characterizing explanation in terms of the types of questions they answer (c.f [Fox, Long, and

Original: Magazzeni, 2017; Smith, 2012] and contrastive explanations in general).
Revised: Magazzeni, 2017; Smith, 2012] and contrastive explanations in general).

Original: This characterization is orthogonal to the question of what type of information constitutes valid explanations.
Revised: This characterization is orthogonal to the question of what type of information constitutes valid explanations.

Original: Putting aside questions regarding observability, the reason why a user requests an explanation is either due  knowledge asymmetry (incomplete or incorrect knowledge of the task) or due to limitations of their inferential capabilities.
Revised: Putting aside questions regarding observability, the reason why a user requests an explanation is either due to knowledge asymmetry (incomplete or incorrect knowledge of the task) or due to limitations of their inferential capabilities.

Original: Depending on the context, the answer to any of the questions described in these papers would require correcting human’s model of the task and/or providing inferential assistance.
Revised: Depending on the context, the answer to any of the questions described in these papers would require correcting human’s model of the task and/or providing inferential assistance.

Original: Works that have looked at model reconciliation explanations have mostly focused on the former.
Revised: Works that have looked at model reconciliation explanations have mostly focused on the former.

Original: Explanations discussed in this paper can be viewed as an answer to the question “Why this plan?”
Revised: Explanations discussed in this paper can be viewed as an answer to the question “Why this plan?”

Original: (which can also be viewed as a contrastive question of the form “Why this plan and not any other plan?”).
Revised: (which can also be viewed as a contrastive question of the form “Why this plan and not any other plan?”).

Original: This is not to say that in complex scenarios just the model reconciliation information would sufﬁce
Revised: This is not to say that in complex scenarios just the model reconciliation information would sufﬁce

Original: but it would need to be supplemented with information that can bridge the differences in inferential capabilities.
Revised: but it would need to be supplemented with information that can bridge the differences in inferential capabilities.

Original: Use of abstractions [Sreedharan, Srivastava, and
Revised: Use of abstractions [Sreedharan, Srivastava, and

Original: Kambhampati, 2018 ] , providing refutation of speciﬁc foils [ Sreedharan, Srivastava, and Kambhampati, 2018 ] and providing causal explanations [ Seegebarth et al., 2012 ] could all be used to augment model reconciliation explanations.
Revised: Kambhampati, 2018 ] , providing refutation of speciﬁc foils [ Sreedharan, Srivastava, and Kambhampati, 2018 ] and providing causal explanations [ Seegebarth et al., 2012 ] could all be used to augment model reconciliation explanations.

Original:  Conclusion and Discussion
Revised:  Conclusion and Discussion

Original: The paper presents a more general formulation for the problem of planning with users in the loop with asymmetric models than any of the previous works.
Revised: The paper presents a more general formulation for the problem of planning with users in the loop with asymmetric models than any of the previous works.

Original: We discuss how this formulation can be extended to capture novel explanatory behaviors and can be solved using approaches that are computationally more efﬁcient than methods that rely on direct model space search.
Revised: We discuss how this formulation can be extended to capture novel explanatory behaviors and can be solved using approaches that are computationally more efﬁcient than methods that rely on direct model space search.

Original: One possible avenue for future work would be investigating and implementing planning compilations that capture extensions of the model reconciliation problem that have previously been investigated like speciﬁc foils, uncertain human models, state abstractions, differences in action costs, disjunctive preconditions, etc...
Revised: One possible avenue for future work would be investigating and implementing planning compilations that capture extensions of the model reconciliation problem that have previously been investigated like speciﬁc foils, uncertain human models, state abstractions, differences in action costs, disjunctive preconditions, etc...

Original: It would also be worth investigating if there are any speciﬁc considerations to be made when choosing heuristics for such planning models.
Revised: It would also be worth investigating if there are any speciﬁc considerations to be made when choosing heuristics for such planning models.

Original: . Explicability versus explanations in human-aware
Revised: . Explicability versus explanations in human-