A Preliminary Logic-based Approach for Explanation Generation
Abstract
In an explanation generation problem, an agent needs to identify and explain the reasons for its decisions to another agent.
Existing work in this area is mostly conﬁned to planning-based systems that use automated planning approaches to solve the problem.
In this paper, we ap- proach this problem from a new perspective, where we propose a general logic-based framework for explana- tion generation.
In particular, given a knowledge base
KB 1 that entails a formula φ and a second knowledge base KB 2 that does not entail φ , we seek to identify an explanation (cid:15) that is a subset of KB 1 such that the union of KB 2 and (cid:15) entails φ .
We deﬁne two types of explana- tions, model- and proof-theoretic explanations, and use cost functions to reﬂect preferences between explana- tions.
Further, we present our algorithm implemented for propositional logic that compute such explanations and empirically evaluate it in random knowledge bases and a planning domain.
Introduction
With increasing proliferation and integration of AI systems in our daily life, there is a surge of interest in explainable AI , which includes the development of AI systems whose actions can be easily understood by humans.
Driven by this goal, machine learning (ML) researchers have begun to classify commonly used ML algorithms according to different dimensions of explainability (Guidotti et al. 2018); improved the explainability of existing ML algorithms (Vaughan et al. 2018; Alvarez Melis and Jaakkola 2018;
Petkovic et al. 2018); as well as proposed new ML algorithms that trade off accuracy for increasing explainability (Dong et al. 2017;
Gilpin et al. 2018).
In contrast, researchers in the automated planning community have mostly taken a complementary approach.
While there is some work on adapting planning algorithms to ﬁnd easily explainable plans 2 (i.e., plans that are easily understood and accepted by a human user)
(Zhang et al. 2017), most work has focused on the explanation generation problem (i.e., the problem of identifying explanations of plans found by planning agents that when presented to users, will allow them to understand and accept the proposed plan) (Langley 2016; Kambhampati 1990).
Within this context, researchers have tackled the problem where the model of the human user may be (1) inconsistent with the model of the planning agent (Chakraborti et al. 2017b); (2) must be learned (Zhang et al. 2017); and (3) a different form or abstraction than that of the planning agent (Sreedharan et al. 2018; Tian et al. 2016).
However, a common thread across most of these works is that they, not surprisingly, employ mostly automated planning approaches.
For example, they often assume that the models of both the agent and human are encoded in PDDL format.
In this paper, we approach the explanation generation problem from a different perspective – one based on knowledge representation and reasoning (KR).
We propose a general logic-based framework for explanation generation, where given a knowledge base KB(of an agent) that entails a formula φ and a knowledge base KB 2 (of a human user) that does not entail φ , the goal is to identify an explanation (cid:15) ⊆
KB 1 such that
KB 2 ∪ (cid:15) entails φ .
We deﬁne two types of explanations, model- and proof-theoretic explanations, and use cost functions to reﬂect preferences between explanations.
Further, we present an algorithm, implemented for propositional logic, that computes such explanations and evaluate its performance experimentally in random knowledge bases as well as in a planning domain.
In addition to providing an alternative approach to solve the same explanation generation problem tackled thus far by the automated planning community, our approach has the merit of being more generalizable to other problems beyond planning problems as long as they can be modeled using a logical KR language.
Preliminaries Logic
A logic L is a tuple ( KB L , BS L , ACC L ) where KB L is the set of well-formed knowledge bases (or theories) of L – each being a set of formulae.
BS L is the set of possible belief sets; each element of BS L is a set of syntactic elements representing the beliefs L may adopt.
ACC L : KB L → 2 BS L describes the “semantics” of L by assigning to each element of KB L a set of acceptable sets of beliefs.
For each KB ∈ KB L and B ∈ ACC L ( KB ) , we say that B is a model of
KB .
A logic is monotonic if KB ⊆ KB (cid:48) implies
ACC L ( KB (cid:48) )
⊆
ACC L ( KB ) .
Example 1 Assume that L refers to the propositional logic over an alphabet P .
Then, KB L is the set of propositional theories over P , BS L = 2 P , and ACC L maps each theory KB into the set of its models in the usual sense.
We say that a KB is consistent if ACC L ( KB ) (cid:54) = ∅ .
A formula ϕ in the logic L is entailed by KB , denoted by KB | =
L ϕ , if ACC L ( KB ) (cid:54) = ∅ and ϕ ∈ B for every B ∈ ACC L ( KB ) .
For our later use, we will assume that a negation operator ¬ over formulas exists; and ϕ and ¬ ϕ are contradictory with each other in the sense that for any KB and B ∈ ACC L ( KB ) , if ϕ ∈ B then ¬ ϕ (cid:54)∈ B ; and if ¬ ϕ ∈ B then ϕ (cid:54)∈ B .
(cid:15) ⊆
KB is called a sub-theory of KB .
A theory KB subsumes a theory KB (cid:48) , denoted by KB (cid:1) KB (cid:48) if ACC L ( KB ) ⊂ ACC L ( KB (cid:48) ) .
Conclusions of a knowledge base can also be derived using rules.
A rule system Σ L of a logic L is a set of rules of the form
[Equation]
where ϕ i are formulas.
The left hand side could be empty.
For a rule r of the form (1), body ( r ) (resp.
head ( r ) ) denotes the left (resp.
right) side of r .
Intuitively, a rule r states that if the body is true then the head is also true.
Given a knowledge base KB and a rule system Σ L , we say KB (cid:96) Σ L ϕ if either ϕ ∈ KB or there exists a sequence of rules r 1 , . . .
, r n in Σ L such that body ( r 1 ) ⊆
KB , head ( r n ) = ϕ , head ( r i ) ∈ KB for i
= 1 , . .
. , n − 1 , and body ( r i ) ⊆
KB or body ( r i ) ⊆ body ( r 1 ) ∪ { head ( r j ) |
j
= 1 , . .
, i − 1 } for every i = 2 , . . . , n .
We call the sequence (cid:15) =
(cid:104) r 1 ; . . .
; r n (cid:105) as a proof from KB for ϕ w.r.t.
Σ L
and say that the proof (cid:15) has the length n .
Σ L is said to be sound if for every ϕ , KB (cid:96) Σ L ϕ implies KB | =
L ϕ .
It is complete if for every ϕ ,
KB
| =
L ϕ implies KB (cid:96) Σ L ϕ .
Classical Planning as Boolean Satisﬁability
A classical planning problem (Russell and Norvig 2009) can be naturally encoded as an instance of propositional satisﬁability (Kautz et al. 1992).
The basic idea is the following: Given a planning problem P , ﬁnd a solution for P of length n by creating a propositional formula that represent the initial state, goal, and the action dynamics for n time steps. This is referred to as the bounded planning problem ( P, n ) , and we deﬁne the formula for ( P, n ) such that: any model of the formula represents a solution to ( P, n ) and if ( P, n ) has a solution, then the formula is satisﬁable. We encode ( P, n ) as a formula Φ such that (cid:104) a 0 , a 1 , . . .
, a n − 1 (cid:105) is a solution for ( P, n ) if and only if Φ can be satisﬁed in a way that makes the ﬂuents a 0 , a 1 , . .
. , a n − 1 true.
The formula Φ is a conjunction of the following formulae:
• Initial State: Let F be the set of possible facts in the planning problem:
[Equation]
• Goal: Let G be the set of goal facts:
[Equation]
∈ G
[Equation]
• Action Scheme: For every action a i at time step i :
[Equation]
(cid:94)
[Equation]
f i
[Equation]
f
[Equation]
Precondition(a)
[Equation]
[Equation]
f i
[Equation]
[Equation]
f ∈
[Equation]
[Equation]
(cid:94)
[Equation]
¬ f i +
[Equation]
f ∈ Delete(a) } • Explanatory Frame Axioms: Formulae describing what does not change between steps i and i + 1 :
[Equation]
} • Complete Exclusion Axioms: Only one action can occur at each time step:
[Equation]
Finally, we can
extract a plan by
ﬁnding
an
assignment of
truth
values
that
satisﬁes Φ (i.
e., for i = n − 1 , there will be
exactly one action a such that a i = True ). This could be easily done by using a satisﬁability algorithm, such as the well-known DPLL algorithm (Davis et al. 1962).
In this paper, we will mostly use examples from propositional logic.
We make use of the fact that the resolution rule is sound and complete in ﬁrst-order logic (Robinson 1965), and hence, in propositional logic.
This allows us to utilize the DPLL algorithm in computing proofs for a formula given a knowledge base.
Two Accounts of Explanations
In this section, we introduce the notion of an explanation in the following setting:
Explanation Generation Problem: Given two knowledge bases KB 1 and KB 2 and a formula ϕ in a logic L .
Assume that KB 1 | =
L ϕ and
KB 2 (cid:54)| = L ϕ .
The goal is to identify an explanation (i.e., a set of formulas) (cid:15) ⊆
KB 1 such that
KB 2 ∪ (cid:15) | = ϕ .
We ﬁrst deﬁne the notion of a support of a formula w.r.t.
a knowledge base.
Deﬁnition 1 (Support) Assume that KB | =
L ϕ .
We say that (cid:15) ⊆
KB is a support of ϕ w.r.t.
KB if (cid:15) | =
L ϕ .
Assume that (cid:15) is a support of ϕ w.r.t.
KB .
We say that (cid:15) ⊆
KB is a ⊆ -minimal support of ϕ if no proper subtheory of (cid:15) is a support of ϕ .
Furthermore, (cid:15) is a (cid:1) - general support of ϕ if there is no support (cid:15) (cid:48) of ϕ w.r.t.
KB such that (cid:15) subsumes (cid:15) (cid:48) .
We now deﬁne below two types of explanations – model-theoretic and proof-theoretic explanations.
Model-Theoretic Explanations
Deﬁnition 2 ( m -Explanation)
Given two knowledge bases KB 1 and KB 2 in logic L and a formula ϕ .
Assume that KB 1 | =
L ϕ and KB 2 (cid:54)| =
L ϕ .
A model-theoretic explanation (or m -explanation) for ϕ from KB 1 for KB 2 is a support (cid:15) w.r.t.
KB 1 for ϕ such that KB 2 ∪ (cid:15) | =
L ϕ .
Example 2 Consider proposition logic theories over the set of propositions { a, b, c } with the usual deﬁnition of models, satisfaction, etc.
Assume KB
= { a, b, a → c, a ∧ b → c } and KB 2 = { a } .
We have that (cid:15)
= { a, a → c } and (cid:15)
= { a, b, a ∧ b → c } are two ⊆ -minimal supports of c w.r.t.
KB 1 .
Only (cid:15) 1 is a (cid:1) - general support of c w.r.t.
KB 1 since (cid:15) 2 (cid:1) (cid:15) 1 .
Both (cid:15) 1 and (cid:15) 2 can serve as m -explanations for c from KB 1 for KB 2 .
Of course, KB 1 is itself an mexplanation for c from KB 1 for KB 2 .
Consider KB 3 = { a, ¬ b } .
In this case, we have that only (cid:15) 1 is an m -explanation for c from KB 1 for KB 3 .
Now, consider KB 4 = {¬ a } .
In this case, we have no m -explanation for c from KB 1 for KB 4 .
Proposition 1 For two knowledge bases KB 1 and
KB 2 in a monotonic logic L , if KB 1 | =
L ϕ and
KB 2 | =
L ¬ ϕ , then there exists no m -explanation for ϕ from KB 1 for KB 2 .
The KB 4 in Example 2 and Proposition 1 show that m -explanations alone might be insufﬁcient.
Sometimes, we also need to persuade the other agent that its knowledge base is not correct.
We leave this for the future.
In this paper, we assume that KB 2 (cid:54)| =
L ¬ ϕ
and
KB 2 (cid:54)| =
L ϕ and, thus, an m -explanation always exists.
Proof-Theoretic Explanations
Deﬁnition 3 ( p -explanation)
Given a logic L with a sound and complete rule system Σ L and two knowledge bases KB 1 and KB 2 in logic L and a formula ϕ .
Assume that KB 1 (cid:96) L ϕ and KB 2 (cid:54)(cid:96) L ϕ .
A proof-theoretic explanation (or p -explanation) for ϕ from KB 1 for KB 2 is a proof (cid:104) r 1 ; . . .
; r n (cid:105) from KBfor ϕ such that KB 2 ∪ ( (cid:83) ni
=1 body ( r i ) ∩ KB 1 ) (cid:96) Σ L ϕ and KB 2 ∪ ( (cid:83)
ni =1 body ( r i ) ∩ KB 1 ) is consistent.
Example 3 Consider the theories KB 1 = { a, b, a → c, a ∧ b → c } and KB 2 = { a } from Example 2.
Let us assume that Σ L is the set of rules of the form l (cid:96) L l and l, ¬ l ∨ p (cid:96) p for any literals l, p in the language of KBand KB 2 .
Then, (cid:104) a, ¬ a ∨ c (cid:96) L c (cid:105) is a proof from KBfor c , which is also a p -explanation for ϕ from KB 1 for
KB 2 .
Likewise, (cid:104) a (cid:96) L a ; b (cid:96) L b ; a, ¬ a ∨ ¬ b ∨ c (cid:96) L ¬ b ∨ c ; b, ¬ b ∨ c (cid:96) L c (cid:105) is a p-explanation for c from KB 1 for
KB 2 .
Proposition 2 Assume that Σ L is a sound and complete rule system of a logic L , KB 1 is a knowledge base, and ϕ is a formula in L .
For each proof (cid:104) r 1 ; . . .
; r n (cid:105) from
KB 1 for ϕ w.r.t.
Σ L , (cid:83) ni =1 body ( r i ) ∩ KB 1 is a support of ϕ w.r.t.
KB 1 .
Proposition 2 implies that each proof from KB 1 for ϕ could be identiﬁed as a p -explanation for ϕ from
KB 1 if Σ L is sound and complete.
This provides the following relationship between m -explanations and pexplanations.
Proposition 3 Assume that Σ L is a sound and complete rule system of a logic L , KB 1 and KB 2 are two knowledge bases in L , and ϕ is a formula in L .
Then,
• for each m -explanation (cid:15) for ϕ from KB 1 for KB 2 , there exists a p -explanation (cid:104) r 1 ; . . .
; r n (cid:105) for ϕ from KB 1 for KB 2 such that (cid:83) ni =1 body ( r i ) ∩ KB 1 ⊆ (cid:15) ; and • for each p -explanation (cid:104) r 1 ; . . .
; r n (cid:105) for ϕ from KBfor KB 2 , (cid:83) ni =1 body ( r i ) ∩ KB 1 is an m -explanation for ϕ from KB 1 for KB 2 .
Preferred Explanations
Given KB 1 and KB 2 and a formula ϕ , there might be several ( m - or p -) explanations for ϕ from KBfor KB 2 .
For brevity, we will now use the term xexplanation for x ∈ { m, p } to refer to an x -explanation for ϕ from KB 1 for KB 2 .
Obviously, not all explanations are equal.
One might preferred a subset minimal m -explanation or a shortest length p -explanation over others.
We will next deﬁne a general preferred relation among explanations.
We assume a cost function C xL that maps pairs of knowledge bases and sets of explanations to nonnegative real values, i.e.,
[Equation]
where Ω is the set of x -explanations and R ≥ 0 denotes the set of non-negative real numbers.
Intuitively, this function can be used to characterize different complexity measurements of an explanation.
A cost function C mL is monotonic if for any two mexplanations (cid:15) 1 ⊆ (cid:15) 2 , C mL ( KB, (cid:15) 1 ) ≤ C mL ( KB, (cid:15) 2 ) .
A cost function C pL is monotonic if for any two pexplanations (cid:15) 1 and (cid:15) 2 such that (cid:15) 1 is a subsequence of (cid:15) 2 , C pL ( KB, (cid:15) 1 ) ≤ C mL ( KB, (cid:15) 2 ) .
C xL induces a preference relation ≺ KB over explanations as follows.
Deﬁnition 4 (Preferred Explanation)
Given a cost function C xL , a knowledge base KB 2 , and two xexplanations (cid:15) 1 and (cid:15) 2 for KB 2 , we say (cid:15) 1 is preferred over (cid:15) 2 w.r.t.
KB 2 (denoted by (cid:15) 1 (cid:22) xKB 2 (cid:15) 2 ) iff
[Equation] and (cid:15) 1 is strictly preferred over (cid:15) 2 w.r.t.
KB 2 (denoted by (cid:15) 1 ≺ xKB 2 (cid:15) 2 ) if
[Equation]
[Equation]
This allows us to compare explanations as follows.
Deﬁnition 5 (Most Preferred Explanation)
Given a cost function C xL and a knowledge base KB 2 , an explanation (cid:15) is a most preferred x -explanation w.r.t.
KB 2 if there exists no other explanation (cid:15) (cid:48) such that (cid:15) (cid:48) ≺ xKB 2 (cid:15) .
Proposition 4 If C xL is monotonic then the relation (cid:22) xKB 2 over x -explanations is transitive, antisymmetric, and reﬂexive; and the relation ≺ xKB 2 over x -explanations is transitive and anti-symmetric.
There are several natural monotonic cost functions.
Examples for cost functions for m -explanations include:
• c 1 L ( KB 2 , (cid:15) ) =
| (cid:15) | , the cardinality of (cid:15) , indicates the number of formulas that need to be explained; • c 2 L ( KB 2 , (cid:15) )
=
| (cid:15) \ KB 2 | , the cardinality of (cid:15) \ KB 2 , indicates the number of new formulas that need to be explained; • c 3 L ( KB 2 , (cid:15) )
=
| new vars ( KB 2 , (cid:15) ) | indicates the number of new symbols occurring in (cid:15) that are not in KB 2 and need to be explained; • c 4 L ( KB 2 , (cid:15) )
=
length ( (cid:15) ) indicates the number of literals in (cid:15) that need to be explained
Naturally, some of these cost functions can also be combined (e.g., c 2 L + c 3 L will measure the number of new formulas and new symbols that must be explained).
Observe that the three functions c 1 L and c 4 L are independent from KB 2 while c 2 L and c 3 L depend on KB 2 .
A potential advantage of a cost function that is independent from KB 2 is that it helps simplify the computation of most preferred explanations.
Example 4 Continuing with Example 2, if we use c 1 L as the cost function, then we have that (cid:15) 1 ≺ m KB(cid:15) 2 ≺ mKB 2 KB 1 .
Furthermore, (cid:15) 1 is the most preferred m -explanation from KB 1 to KB 2 . Input: Logic L
Computing Preferred Explanations
At a high level, Algorithms 1 and 2 can be used for computing most-preferred explanations given a formula ϕ and two knowledge bases KB 1 and KB 2 of a logic L with the cost function C xL .
We assume that when computing for p -explanations, a sound and complete rule system is available.
Our algorithms rely on the existence of an algorithm for checking entailment between knowledge bases and formulas (Lines 1 and 3 in Algorithm 1 and Line 4 in Algorithm 2) and an algorithm for computing a potential explanation that is minimal with respect to a cost function and a knowledge base (Lines 2-3 in Algorithm 2).
These two algorithms depend on the logic L and the cost function C xL and need to be implemented for speciﬁc logic L and function C xL .
In the rest of this section, we discuss the implementation of our algorithms for propositional logic and different cost functions.
With propositional logic, it is easy to see that checking for entailment can be done by a SAT solver (e.g., MiniSat (E´en and S¨orensson 2003)).
We next discuss two algorithm implementations, one for mexplanations and one for p -explanations, that ﬁnd an explanation that is minimal with respect to a cost function and a knowledge base.
Most-Preferred m -Explanations
Given a cost function C mL such as c 1 L , c 2 L , c 3 L , or c 4 L as deﬁned in Section , Algorithm 3 computes a most pre- Input: Formula ϕ , KBs KB 1 and KB 2 , cost function C mL Output: most m t
C considered 3 repeat (cid:15) checked if then return (cid:15) else for a ∈ KB 1 do if
∪ a checked then v = C
) q = v key ferred m -explanations w.r.t.
C mL from KB 1 to KB 2 for ϕ or returns nil if none exists.
The key data structures in the algorithm is a priority queue q , initialized to only include the empty set, of potential explanations ordered by their costs (Line 1) and a set checked of invalid explanations that have been considered thus far (line 2).
The algorithm repeatedly loops the following steps: ( i ) move the explanation with the smallest cost from the priority queue q to checked (Lines 4-5); ( ii ) check if it is a valid m -explanation and return if it is (Lines 6-7); ( iii ) if not, extend the explanation by 1 (with each clause from KB 1 ) and insert the extended explanations into the priority queue q (Lines 8-12).
If all potential explanations are exhausted, which means that there are no valid m -explanations, then the algorithm returns nil (Line 14).
It is straightforward to see that the following proposition holds.
Proposition 5 For two propositional theories KBand KB 2 and a formula ϕ , Algorithm 3 returns a most preferred m -explanation w.r.t.
C mL for ϕ from KB 1 to
KB 2 if one exists.
Most-Preferred p -Explanations
Given a cost function C pL on p -explanations, Algorithm 4 computes a most-preferred p -explanation w.r.t.
C pL from KB 1 to KB 2 for ϕ or returns nil if none exists.
We use the following notations in the pseudocode:
For a proof (cid:104) (cid:15) (cid:105) , where (cid:15) is the sequence (cid:104) r 1 ; . . .
; r n (cid:105) , we write c ( (cid:15) )
=
head ( r n ) and b ( (cid:15) )
= (cid:83) ni =1 body ( r i ) .
We also write ϕ 1 ,ϕ 2 ϕ to indicate that ϕ is the result of 7 repeat do if c and c ( (cid:15) (cid:48) ) contain complementary literals and c ( (cid:15) ) ,c
(cid:15)
) φ holds then then key applying the resolution rule on ϕ 1 and ϕ 2 .
And we use ◦ to denote the concatenation of two sequences.
The algorithm uses the same two data structures – priority queue q and set checked – as in Algorithm 3.
The algorithm ﬁrst populates the queue q with singlerule proofs consist of single clauses in KB 1 (Lines 2-4).
Then, it repeatedly loops the following steps: ( i ) move the proof with the smallest cost from the priority queue q to checked (Lines 8-9); ( ii ) check if it is a valid pexplanation and return if it is (Lines 10-11); ( iii ) if not, extend the proof by 1 and insert the extended proofs into the priority queue q (Lines 12-17).
If all potential proofs are exhausted, which means that there are no valid pexplanations, then the algorithm returns nil (Line 19).
It is straightforward to see that the following proposition holds.
Proposition 6 For two propositional theories KBand KB 2 and a formula ϕ , Algorithm 4 returns a most preferred p -explanation w.r.t.
C pL for ϕ from KB 1 to
KB 2 if one exists.
Plan Explanation Generation
As presented in the preliminaries, we can model a planning problem using the propositional logic language and thus utilize the proposed framework to generate explanations. Particularly, we form the knowledge base of [Table]
[Table]
[Table]
the agent, namely KB , by adding the encoded formula Φ (represented in CNF clauses) as well as the optimal plan of the speciﬁc planning problem.
Then, we deﬁne the explanation in terms of KB and plan optimality as follows:
Deﬁnition 6 (Optimal Plan Explanation)
Given a knowledge base
KB and a plan π n = (cid:104) a 0 , a 1 , . .
. , a n − 1 (cid:105) , we say that π n is optimal in KB if and only if KB | = φ , where ∀ t = 1 , . . .
n − 1 : φ = ¬ goal t .
In other words, the formula φ that we seek to explain is that no plan of lengths 1 to n − 1 exists, and that a plan of length n exists.
Therefore, combined, that plan must be an optimal plan. Now, given a second knowledge base KB 2 (i.e that of a human user), where KB 2 (cid:54)| = φ , we can compute a model- or proof-theoretic explanation as deﬁned in Deﬁnitions 2 and 3. Experimental Results We empirically evaluate our implementation of Algorithm 3 to ﬁnd m -explanations on two synthetically generated benchmarks – random knowledge bases and a planning domain called B LOCKSWORLD – both encoded in propositional logic.
3 We evaluated our algorithm using the four cost functions described in Section .
Our algorithm was implemented in Python and experiments were performed on a machine with an Intel i 2.6GHz processor and 16GB of RAM.
We report both the cost of the optimal m -explanation found as well as the runtime of the algorithm.
Random Knowledge Bases
We ﬁrst evaluated our algorithm on random knowledge bases with clauses in Horn form, where we varied the 3 For random knowledge bases, we used an optimized version that uses a version of backward chaining that ﬁnds the set of all possible explanations.
This approach works only when the clauses in the knowledge base are in Horn form and is sound and complete for such a case (Russell and Norvig 2009).
cardinality of KB 1 (the KB of the agent providing the explanation) from 20 to 1000.
To construct KB 2 (the
KB of the agent receiving the explanation), we randomly chose 25% of the clauses from KB 1 .
To construct each KB 1 , we ﬁrst generated | KB 1 |random symbols, which will be used in the KB.
Then, we iteratively generated clauses of increasing length l from 2 to 7.
For each length l , we generated (cid:98) | KB 1 | 2 · l (cid:99) clauses using the symbols we previously generated such that each symbol is used at most once in these clauses of length l .
Each clause is a conjunction of l − 1 elements as the premise and the ﬁnal l th element as the conclusion.
For example, a KB with a cardinality of 20,symbols are ﬁrst generated.
Then, 5 clauses of length 2, 3 clauses of length 3, 2 clauses of lengths 4 and 5, and 1 clause of lengths 6 and 7 are generated.
Finally, to complete the KB, we add all the symbols that are exclusively in the premise of the clauses generated as facts in the KB.
The formula ϕ that we seek to explain is one of the randomly chosen conclusions in the clauses generated, which we ensure is entailed by KB 1 .
Table 1(a) tabulates our results.
We make the following observations:
• As expected, the runtimes increase as | KB 1 | increases since the algorithm will need to search over a larger search space.
• As expected, the costs of explanations also increase as | KB 1 | increases since the explanations are presumably longer and more complex.
• Finally, the runtimes for cost functions c 1 L and c 4 L are smaller than that of c 2 L and c 3 L .
The reason is the computation of the costs of possible explanations is faster with the former two cost functions since they are not dependent on KB 2 while the computation for the latter two cost functions are dependent on KB 2 .
Planning Domain
As we were motivated by the explanation generation problem studied in the automated planning community, we also conducted experiments on B LOCKSWORLD , a planning domain where multiple blocks must be stacked in a particular order on a table.
For these planning problems, we ﬁrst used F ASTD OWNWARD (Helmert 2006) to ﬁnd optimal solutions to the planning problem.
Then, we translate the planning problem into a SAT problem with horizon h
(Kautz et al. 1992), where h is the length of the optimal plan.
These CNF clauses then form our KB 1 (the KB of the agent providing the explanation).
Similar to random knowledge bases, we construct KB 2 (the KB of the agent receiving the explanation) by randomly choosing 25% of the clauses from KB 1 .
The formula ϕ that we seek to explain is then that no plan of lengths 1 to h − 1 exists, and that a plan of length h (i.e., the plan found by F AST D OWNWARD ) exists.
Therefore, combined, that plan must be an optimal plan.
Table 1(b) tabulates our results, where we observe similar trends as in the experiment on random knowledge bases.
The key difference is that the runtimes for all four cost functions here are a lot closer to each other, and the reason is because there was only one valid explanation in each problem instance.
Thus, regardless of the choice of cost function, that explanation had to be found.
Our experiments for larger problems are omitted as they timed out after 6 hours.
Related Work and Discussions
There is a very large body of work related to the very broad area of explainable AI.
We have brieﬂy discussed some of them from the ML literature in Section .
We refer readers to surveys by (Adadi and Berrada 2018) and (Dosilovic et al. for more in-depth discussions of this area.
We focus below on related work from the KR and planning literature only since we employ KR techniques to solve explainable planning problems in this paper.
Related Work from the KR Literature: We note that the notion of an explanation proposed in this paper might appear similar to the notion of a diagnosis that has been studied extensively in the last several decades (e.g., (Reiter 1987)) as both aim at explaining something to an agent.
Diagnosis focuses on identifying the reason for the inconsistency of a theory whereas an mor p -explanation aims at identifying the support for a formula.
The difference lies in that a diagnosis is made with respect to the same theory and m - or p -explanation is sought for the second theory.
Another earlier research direction that is closely related to the proposed notion of explanation is that of developing explanation capabilities of knowledge-based systems and decision support systems, which resulted in different notions of explanation such as trace, strategic, deep, or reasoning explanations (see review by (Moulin et al. 2002) for a discussion of these notions).
All of these types of explanations focus on answering why certain rules in a knowledge base are used and how a conclusion is derived.
This is not our focus in this paper.
The present development differs from earlier proposals in that m - or p -explanations are identiﬁed with the aim of explaining a given formula to a second theory.
Furthermore, the notion of an optimal explanation with respect to the second theory is proposed.
There have been attempts to using argumentation for explanation (Cyras et al. 2017;
Cyras et al. 2019)
because of the close relation between argumentation and explanation.
For example, argumentation was used by (Cyras et al. 2019) to answer questions such as why a schedule does (does not) satisfy a criteria (e.g., feasibility, efﬁciency, etc.); the approach was to develop for each type of inquiry, an abstract argumentation framework (AF) that helps explain the situation by extracting the attacks (non-attacks) from the corresponding AF.
Our work differs from these works in that it is more general and does not focus on a speciﬁc question.
It is worth to pointing out that the problem of computing a most preferred explanation for ϕ from KB 1 to
KB 2 might look similar to the problem of computing a weakest sufﬁcient condition of ϕ on KB 1 under KBas described by (Lin 2001).
As it turns out, the two notions are quite different.
Given that KB 1 = { p, q } and
KB 2 = { p } .
It is easy to see that q is the unique explanation for q from KB 1 to KB 2 .
On the other hand, the weakest sufﬁcient condition of q on KB 1 under KBis ⊥ (Proposition 8, (Lin 2001)).
Related Work from the Planning Literature: In human-aware planning, the (planning) agent must have knowledge of the human model in order to be able to contemplate the goals of the humans as well as foresee how its plan will be perceived by them.
This is of the highest importance in the context of explainable planning since an explanation of a plan cannot be onesided (i.e., it must incorporate the human’s beliefs of the planner).
In a plan generation process, a planner performs argumentation over a set of different models (Chakraborti et al. 2017a); these models usually are the model of the agent incorporating the planner, the model of the human in the loop, the model the agent thinks the human has, the model the human thinks the agent has, and the agent’s approximation of the latter.
Therefore, the necessity for plan explanations arises when the model of the agent and the model the human thinks the agent has diverge so that the optimal plans in the agent’s model are inexplicable to the human.
During a collaborative activity, an explainable planning agent (Fox et al. 2017) must be able to account for such model differences and maintain an explanatory dialogue with the human so that both of them agree on the same plan. This forms the nucleus of explanation generation of an explainable planning agent, and is referred to as model reconciliation (Chakraborti et al. 2017b). In this approach, the agent computes the optimal plan in terms of his model and provides an explanation of that plan in terms of model differences. Essentially, these explanations can be viewed as the agent’s attempt to move the human’s model to be in agreement with its own.
Further, for computing explanations using this approach the following four requirements are considered:
• Completeness – No better solution exists.
This is achieved by enforcing that the plan being explained is optimal in the updated human model.
• Conciseness – Explanations should be easily understandable to the human.
• Monotonicity – The remaining model differences cannot change the completeness of an explanation.
• Computability – Explanations should be easy to compute (from the agent’s perspective).
As our work is motivated by these ideas, we now identify some similarities and connections with our proposed approach.
First, it is easy to see that we implicitly enforce the ﬁrst three requirements when computing an explanation – the notions of completeness and conciseness are captured through the use of our cost functions.
We do not claim to satisfy the computability requirement as it is more subjective and is more domain dependent.
Inanutshell, the model reconciliation approach works by providing a model update (cid:15) such that the optimal plan is feasible and optimal in the updated model of the human.
This is similar to our deﬁnition of the explanation generation problem where we want to identify an explanation (cid:15) ⊆
KB 1 (i.e., a set of formulae) such that KB 2 ∪ (cid:15) | = φ .
In addition, the ⊆ -minimal support in Deﬁnition 1 is equivalent to minimally complete explanations (MCEs) (the shortest explanation).
The (cid:67) -general support can be viewed as similar to the minimally monotonic explanations (MMEs) (the shortest explanation such that no further model updates invalidate it), with the only difference being that in the general support scenario, the explanations are such that all subsuming (cid:15) are also valid supports.
In contrast, model patch explanations (MPEs) (includes all the model updates) are trivial explanations and are equivalent to our deﬁnition that KB 1 itself serves as an m -explanation for KB 2 .
Note that, in our approach, we do not allow for explanations on “mistaken” expectations in the human model, as it can be inferred from Proposition 1 (monotonic language L ).
From the model reconciliation perspective, such restriction is relaxed and allowed.
However, a similar property can be seen if the mental model is not known and, therefore, by taking an “empty” model as starting point explanations can only add to the human’s understanding but not mend mistaken ones.
Conclusions and Future Work
Explanation generation is an important problem within the larger explainable AI thrust.
Existing work on this problem has been done in the context of automated planning domains, where researchers have primarily employed, unsurprisingly, automated planning approaches.
In this paper, we approach the problem from the perspective of KR, where we propose a general logic-based framework for explanation generation.
We further deﬁne two types of explanations, model- and proof-theoretic explanations, and use cost functions to reﬂect preferences between explanations.
Our empirical results with algorithms implemented for propositional logic on both random knowledge bases as well as a planning domain demonstrate the generality of our approach beyond planning problems.
Future work includes investigating more complex scenarios, such as one where an agent needs to persuade another that its knowledge base is incorrect. Acknowledgment
This research is partially supported by NSF grants 1345232, 1757207, and 1812628.
The views and conclusions contained in this document are those of the authors and should not be interpreted as representing the ofﬁcial policies, either expressed or implied, of the sponsoring organizations, agencies, or the U.S. government.