P R O D I G Y : An Integrated Archi tec ture for P lanning and Learning

Artificial intelligence has progressed to the point where multiple cognitive capabilities are being integrated into computational architectures, such as SOAR, PRODIGY~ THEO, and ICARUS. This paper reports on the PRODIGY architecture, describing its planning and problem solving capabilities and touching upon its multiple learning methods. Learning in PRODIGY Occurs at all decision points and integration in PRODIGY is at the knowledge level; the learning and reasoning modules produce mutually interpretable knowledge structures. Issues in architectural design are discussed, providing a context to examine the underlying tenets of the PRODIGY architecture. 1 I n t r o d u c t i o n A common dream for many AI researches, present authors included, is the construction of a general purpose learning and reasoning system that given basic axiomatic knowledge of a domain is capable of becoming an expert problem solver. Our machine learning approach, implemented in PRODIGY [2], starts with a general problem-solving engine based on a possibly incomplete domain theory. The problem solver improves its performance through experience by refining the initial domain knowledge and learning knowledge to control the search process. The paper is divided into two parts. The first part describes the basic architecture, including the problem solver and the various learning modules. The second part discusses the design issues in building an integrated architecture. 2 T h e PRODIGY A r c h i t e c t u r e 2.1 T h e P r o b l e m S o l v e r PRODIGY'$ basic reasoning engine is a general-purpose problem solver and planner [10] that searches for sequences of operators (i.e., plans) to accomplish a set of goals from a specified initial state description. Search in PRODIGY is guided by a set of control rules that apply at each decision point. Search control rules may be general or domain specific, handcoded or automatically acquired, and may consist of heuristic preferences or definitive selections. In the absence of any search control, PRODIGY defaults to depth-first means-ends analysis. But, with appropriate search control knowledge it can emulate other search disciplines, including breath-first search, depth-first iterative-deepening, best-first search, and knowledge-based plan instantiation. 2 . 2 K n o w l e d g e R e p r e s e n t a t i o n Each operator has a precondition expression that must be satisfied before the operator can be applied, and a list of effects that describe how the application of the operator changes the world. Precondition expressions are well-formed formulas in a form of predicate logic encompassing negation, conjunction, disjunction, and existential and universal quantification. The effects are atomic formulas that describe the facts that are added or deleted from the current state when the operator is applied. Operators may also contain conditional effects, which represent changes to the world that are dependent on the state in which the operator is applied. 2.3 P r o b l e m D e f i n i t i o n a n d P r o b l e m S o l v i n g A problem consists of an initial state and a goal expression. To solve a problem, PRODIGY must find a sequence of operators that, if applied to the initial state, produces a final state satisfying the goal expression. The search tree initially starts out as a single node containing the initial state and goal expression. The tree is expanded by repeating the following two steps: 1. Dec i s ion phase : There are four types of decisions that PRODIGY makes during problem solving. First, it must decide what node in the search tree to expand next, defaulting to a depth-first expansion. Each node consists of a set of goals and a state describing the world. After a node has been selected, one of the node's goals must be selected, and then an operator relevant to this goal must be chosen. Finally, a set of bindings for the parameters of that operator must be decided upon. 2. E x p a n s i o n phase : If the instantiated operator's preconditions are satisfied, the operator is applied. Otherwise, PRODIGY subgoals on the unmatched preconditions. In either case, a new node is created with updated information about the state or the subgoals. The search terminates after creating a node whose state satisfies the top-level goal expression. 2 . 4 C o n t r o l R u l e s As PRODIGY attempts to solve a problem, it must make decisions about which node to expand, which goal to work on, which operator to apply, and which objects to use. These decisions can be influenced by control rules to increase the efficiency of the problem solver's search and to improve the quality of the solutions that are found. PRODIGY's reliance on explicit control rules, which can be learned for specific domains, distinguishes it from most domain independent problem solvers. Instead of using a leastcommitment search strategy, for example, PRODIGY expects that any important decisions will be guided by the presence of appropriate control knowledge. If no control rules are relevant to a decision, then PRODIGY makes a quick, arbitrary choice. If in fact the wrong choice is made, and costly backtracking proves necessary, an attempt will be made to learn the control knowledge that must be missing. The rationale for PRODIGY's casual commi tment strategy is that for any 51 S I G A R T Bullet in , Vol. 2, No. 4 decision with significant ramifications, control rules should be present; if they are not, the problem solver should not attempt to be clever without knowledge, rather, the cleverness should come about as a result of learning. Thus, our emphasis is on an elegant and simple problem-solving architecture which can produce sophisticated behavior by learning the appropriate, domain-specific control knowledge. Control rules can be employed to guide the four decisions described in Section 2.3. Each control rule has a left-hand side condition testing applicability and a right-hand side indicating whether to SELECT, REJECT, or PREFER a particular candidate. To make a control decision, given a default set of candidates (nodes, goals, operators, or bindings, depending on the decision), PRODIGY first applies the applicable selection rules to select a subset of the candidates. If no selection rules are applicable, all the candidates are included. Next rejection rules further filter this set by explicit elimination of particular remaining candidates, and finally preference rules are used to find the most preferred alternative. If backtracking is necessary, the next most preferred candidate is attempted, and so on, until a global solution is found, or until all selected and non-rejected candidates are exhausted. 2.5 T h e L e a r n i n g M o d u l e s PRODIGY~s general problem solver is combined with several learning modules. The PRODIGY architecture was designed both as a unified testbed for different learning methods and as a general architecture to solve interesting problems in complex task domains. Let us now focus on the architecture itself, as diagrammed in Figure 1. The problem solver produces a complete search tree, encapsulating all decisions right ones and wrong ones as well as the final solution. This information is used by each learning component in different ways. In addition to the central problem solver, PRODIGY has the following learning components: A P P R E N T I C E : A user interface that can participate in an apprentice-like dialogue [5], enabling the user to evaluate and guide the system's problem solving and learning. The interface is graphic-based and tied directly to the problem solver, so that it can both acquire domain knowledge or accept advice as it is solving a problem. EBL: An explanation-based learning facility [9] for acquiring control rules from a problem-solving trace. Explanations are constructed from an axiomatized theory describing both the domain and relevant aspects of the problem solver's architecture. Then the resulting descriptions are expressed in control rule form, and control rules whose utility in search reduction outweighs their application overhead are retained. S T A T I C : A method for learning control rules by analyzing PRODIGY's domain descriptions prior to problem solving. The STATIC program produces control rules without utilizing any training examples. STATIC can be viewed as a compiler for PRODIGY's domains [4]. In experimental tests, STATIC produced control knowledge that was superior to that of EBL and did so one to two orders of magnitude faster. However, not all problem spaces permit purely static learning, requiring EBL's dynamic capabilities. STATIC's design is based on a detailed predictive theory of EBL, which is described in [3]. A N A L O G Y : A derivational analogy engine [13] that uses similar previously solved problems to solve new problems. The problem solver records the justifications for each decision during its search process. These justifications are then used to guide the reconstruction of the solution for subsequent problem solving situations where equivalent justifications hold true. Both analogy and EBL are independent mechanisms to acquire domain-specific control knowledge. A L P I N E : An abstraction learning and planning module [6]. The axiomatized domain knowledge is divided into multiple abstraction levels based on an analysis of the domain. Then, during problem solving, PRODIGY first finds a solution in an abstract space and then uses the abstract solution to guide the search for solutions in more detailed problem spaces. This method is orthogonal to analogy and EBL, in that both can apply at each level of abstraction. E X P E R I M E N T : A learning-by-experimentation module for refining domain knowledge that is incompletely or incorrectly specified [1]. Experimentation is triggered when plan execution monitoring detects a divergence between internal expectations and external observations. The main focus of experimentation is to refine the factual domain knowledge,