Toward Domain-Neutral Human-Level Metacognition

We have found that implementing a metacognitive loop (MCL), which gives intelligent systems the ability to selfmonitor their ongoing performance and make targeted changes to their various action-determining components, can play an important role in helping systems cope with the unexpected problems and events that are the inevitable result of real-world deployment. In this paper, we discuss our work with MCL-enhanced intelligent systems, and describe the ontologies that allow MCL to reason about, appropriately classify and respond to the perfomance anomalies it detects. Introduction: The Hardest AI Problem and the Metacognitive Loop It is widely agreed that AI has been extremely successful in the narrow sense: given any of a large variety of well-defined AI problems, there is (or, so goes the common wisdom, there could easily be) an implemented solution using reasonably well-understood techniques. Or at the very least, there are solutions to similar problems, so that there is much reason to expect solutions to the problem at hand. But in the wide sense, AI is a disappointment. Although many AI systems are at or beyond human-level competence at the individual tasks for which they were designed (from chess playing to medical diagnosis), AI systems are nowhere near human-level competence across the board. This is not simply a matter of building a system that has numerous subsystems, one for each of hundreds or thousands of individual tasks. The combined skills of a doctor, lawyer, chess-master, linguist, etc., does not a human make. For humans have, in addition to whatever number of individual skills, the ability to do reasonably well when faced with situations for which they have not been specifically trained. This general competence, the ability to muddle through when faced with the unexpected or unfamiliar, we consider to be the the core of human-level common sense. Indeed, this ability may well be the key to how we are able to train: we recognize when we lack an ability, make a decision to gain it, and recognize when it is time to stop learning—when training is complete; or complete enough for our purposes; or when it is ineffective, too slow, too expensive, or no longer important. We Copyright c © 2007, Association for the Advancement of Artificial Intelligence (www.aaai.org). All rights reserved. also (sometimes! ) recognize when we are in over our heads, and then we ask for help or wisely give up. It is this general sort of ability that current AI systems lack, with the result that they tend to be brittle, easily breaking when confronted with situations beyond their set of narrow competencies. Why is this so? And why has the brittleness problem been so hard to solve? Our hypothesis is this: while there are well-established and sohpisticated AI technologies for dealing with change or novelty e.g., machine learning (ML) and commonsense reasoning (CSR), neither alone is adequate to the task in question, since each needs the other to realize its true potential. Thus ML needs reasoned guidance (a form of CSR) as to when, why, what, and how to learn; and CSR needs trainable modules (to which ML can be applied) so that the fruits of such reasoning can be applied to bring about better future performance. Proper melding of ML and CSR should, in our view, go a long way toward solving the brittleness problem and bringing us closer to the ultimate goal of AI. What will it take to meld ML and CSR in such a way as to solve the brittleness problem, i.e., to build a system that (under a wide range of unforeseen circumstances) adjusts itself instead of breaking? We have isolated four capacities. First of all, a system will need to know enough about what it is doing (its goals, its actions, and their results), so that it can detect—and then assess—when those goals are not being achieved (Brachman 2002). This is a key place where CSR comes into play, in reasoning after the fact about such goal failures. Other relevant approaches include case-based reasoning and meta-level planning (Cox 2005; Anderson & Oates 2007). Second, it will need to be able to make targeted changes to itself when there are indications of current inadequacies in what it is doing. This is where ML can come into play: if the desired changes involve any sort of training or adapting, the relevant trainable modules and their associated training processes must be invoked. Third, it will need to be able to apply the same first two capacities to monitor how a given self-change is working out (thus there is a form of recursion involved). This again involves CSR in order to assess whether the targeted change is occurring properly. And fourth, it will need to do this in real time. This is a tall order, but the elements are clear enough to allow progress. This paper reports on our work to date and where we see it headed. The rest of the paper is organized as follows: We outline our specific approach to these four capacities (which we call MCL: the metacognitive loop); we discuss some early pilot studies; we present three ontologies that allow MCL to reason about, and appropriately classify and respond to the perfomance anomolies it detects; we show how these ontologies have been applied to a simulated autonomous vehicle domain; and we conclude with some general considerations and a discussion of future work.