Generating Explanations of Action Failures in a Cognitive Robotic Architecture (2020)

Abstract: We describe an approach to generating explanations about why robot actions fail, focusing on the considerations of robots that are run by cognitive robotic architectures. We define a set of Failure Types and Explanation Templates, motivating them by the needs and constraints of cognitive architectures that use action scripts and interpretable belief states, and describe content realization and surface realization in this context. We then describe an evaluation that can be extended to further study the effects of varying the explanation templates.

## introduction robots that can explain why their behavior deviates from user expectations will likely benefit by better retaining human trust @xcite . robots that are driven by a cognitive architecture such as soar @xcite , @xcite , or diarc @xcite have additional requirements in terms of connecting to the architecture's representations such as its belief structures and action scripts. if properly designed, these robots can build on the interpretability of such architectures to produce explanations of action failures. there are various types of cognitive architectures, which may be defined as "abstract models of cognition in natural and artificial agents and the software instantiations of such models" @xcite but in this effort we focus on the type that uses action scripts, belief states, and natural language to interact with humans as embodied robots in a situated environment. in section 2 we describe an approach to explaining action failures, in which a person gives a command to a robot but the robot is unable to complete the action. this approach was implemented in a physical robot with a cognitive architecture, and tested with a preliminary evaluation as described in section 3. after comparing our effort to related work in section 4, we finish by discussing future work. ## an approach to action failure explanation our approach is made up of a set of failure types, a set of explanation templates, algorithms for content realization, and algorithms for surface realization. ## evaluation to validate our system, we conducted a user study. besides testing the components all working together, we were also interested in understanding the effect of the different types of explanation templates on human perceptions of the explanations given. this study was conducted under the oversight of an institutional review board. ## related work human-robot interaction (hri) research on explaining the actions of robots @xcite is related to research on explaining planning decisions @xcite , on generating language that describes the pre-and post-conditions of actions in planners @xcite , and on generating natural language explanations from various types of meaning representations @xcite . in hri work that focuses on error reporting, briggs and scheutz (2015) defined a set of felicity conditions that must hold for a robot to accept a command. they outlined an architecture that reasons about whether each felicity condition holds, and they provided example interactions, although they did not evaluate an implementation of their approach. similarly, @xcite used a logicbased approach to identify whether a command can be done, and provided example situations, but no evaluation. our approach is similar in that we define a set of failure types for action commands, but we implement and evaluate our approach with a user study. other recent hri work has included communicating errors using non-verbal actions to have a robot express its inability to perform an action @xcite , which does not focus on more complex system problems using natural language communications as we do. there has also been recent work on user modeling and tailoring responses to users in robots @xcite @xcite . in one effort worth building upon, chiyah garcia et al. ( 2018 ) used a human expert to develop explanations for unmanned vehicle decisions. these explanations followed @xcite in being characterized in terms of soundness, relating the depth of details, and completeness, relating to the number of details. chiyah garcia et al. found links between the "low soundness and high completeness" condition and intelligibility and value of explanations. ## conclusions we have described an approach to generating action failure explanations in robots, focusing on the needs and strengths of a subset of cognitive robotic architectures. this approach takes advantage of the interpretability of action scripts and belief representations, and is guided by recent directions in hri research. importantly, the explanation of this approach is not a post-hoc interpretation of a blackbox system, but is an accurate representation of the robot's operation. various aspects of the approach are being continually refined. currently, new failure types are being investigated, and the content realization and surface realization algorithms are being revised and tested. finally, the evaluation in section 2.2 describes a preliminary approach to comparing the relative impact of the various explanation templates. we are pursuing additional studies focusing on varying the explanations produced. initial studies would be video-based, after which follow-up studies would be conducted in the context of a task being performed either in person, or via a virtual interface that we have constructed, and the goal would be to examine the ways that context features such as user model, physical setting, and task state affect the type of explanation required.