Metareasoning and Bounded Rationality

What role does metareasoning play in models of bounded rationality? We examine the various existing computational approaches to bounded rationality and divide them into three classes. Only one of these classes significantly relies on a metareasoning component. We explore the characteristics of this class of models and argue that it offers desirable properties. In fact, many of the effective approaches to bounded rationality that have been developed since the early 1980’s match this particular paradigm. We conclude with some open research problems and challenges. Computational models of bounded rationality In the pursuit of building decision-making machines, artificial intelligence researchers often turn to theories of “rationality” in decision theory and economics. Rationality is a desired property of intelligent agents since it provides well-defined normative evaluation criteria and since it establishes formal frameworks to analyze agents (Doyle 1990; Russell and Wefald 1991). But in general, rationality requires making optimal choices with respect to one’s desires and goals. As early as 1947, Herbert Simon observed that optimal decision making is impractical in complex domains since it requires one to perform intractable computations within a limited amount of time (Simon 1947; 1982). Moreover, the vast computational resources required to select optimal actions often reduce the utility of the result. Simon suggested that some criterion must be used to determine that an adequate, or satisfactory, decision has been found. He used the Scottish word “satisficing,” which means satisfying, to denote decision making that searches until an alternative is found that is satisfactory by the agent’s aspiration level criterion. Simon’s notion of satisficing has inspired much work within the social sciences and within artificial intelligence in the areas of problem solving, planning and search. In the social sciences, much of the work has focused on developing descriptive theories of human decision making (Gigerenzer 2000). These theories attempt to explain how people make decisions in the real-world, coping with complex situations, uncertainty, and limited amount of time. The answer is often based on a variety of heuristic methods that Copyright c © 2008, Association for the Advancement of Artificial Intelligence (www.aaai.org). All rights reserved. are used by people to operate effectively in these situations. Work within the AI community–which is the focus of this paper–has produced a variety of computational models that can take into account the computational cost of decision making (Dean and Boddy 1988; Horvitz 1987; Russell et al. 1993; Wellman 1990; Zilberstein 1993). The idea that the cost of decision making must be taken into account was introduced by Simon and later by the statistician Irving Good who used the term Type II Rationality to describe it (Good 1971). Good said that “when the expected time and effort taken to think and do calculations is allowed for in the costs, then one is using the principle of rationality of type II. ” But neither Simon nor Good presented any effective computational framework to implement “satisficing” or “type II rationality”. It is by now widely accepted that in most cases the ideal decision-theoretic notion of rationality is beyond our reach. However, the concept of satisficing offers only a vague design principle that needs a good deal of formalization before it can be used in practice. In particular, one must define the required properties of a satisficing criterion and the quality of behavior that is expected when these properties are achieved. AI researchers have introduced over the years a variety of computational models that can be seen as forms of bounded rationality. We start by dividing these models into three broad classes. We are particularly interested in the role that metareasoning plays in these theories. Approximate reasoning One of the early computational approaches to bounded rationality has been based on heuristic search. In fact, Simon has initially identified satisficing with heuristic search. In this context, heuristic search represents a form of approximate reasoning. It uses some domain knowledge to guide the search process, which continues until a satisfactory solution is found. This should be distinguished from admissible heuristic techniques such as A∗ that are designed to always return the optimal answer. Admissible heuristic search is an important part of AI, but it has little to do with bounded rationality. The focus on optimal, rather than satisfying, solutions makes this type of heuristic search simply a more efficient way to find exact answers. Simon refers to another type of heuristic functions in which heuristics are used to select “adequate” solutions. Such heuristic functions are rarely admissible and the corresponding search processes are not optimal in any formal sense. Systems based on non-admissible heuristic functions are often harder to evaluate, especially when optimal decisions are not available. Although it is often assumed that approximate reasoning is aimed at finding approximate answers to a given problem, it can take different forms. For example, the initial problem can be reformulated in such a way that reduces its complexity. The reformulation process could be approximate, yielding a new problem that is easier to solve because it does not retain all the details of the original problem. The resulting problem could then be solved efficiently and perhaps optimally and the obtained solution can then be used as an approximate solution for the original problem. One example of such a process–also referred to as approximate modeling–is when deterministic action models are used in planning, ignoring the uncertainty about action failures. Combined with suitable runtime execution monitoring, such an approach could be beneficial. In fact, the winner of a recent probabilistic planning competition was a planner based on these principles. Regardless of the form of approximation, approximate reasoning techniques can be complemented by some form of explicit or implicit metareasoning. Metareasoning in this context is a mechanism to make certain runtime decisions by reasoning about the problem solving–or object-level–reasoning process. This can be done either explicitly, by introducing another level of reasoning as shown in Figure 1, or implicitly, by pre-compiling metareasoning decisions into the object-level reasoning process at design time. For example, metareasoning has been used to develop search control strategies–both explicitly and implicitly. In some cases, the goal is specifically to optimize the tradeoff between search effort and quality of results (Russell and Wefald 1991). Thus, metareasoning could play a useful role in certain forms of approximate reasoning, but it is not–by definition–a required component. While it is clear that any form of bounded rationality essentially implies that the agent performs approximate reasoning, the opposite is not necessarily true. Generally, frameworks for approximate reasoning do not provide any formal guarantees about the overall performance of the agent. Such guarantees are necessary to offer a satisfactory definition of bounded rationality and thus restore some sort of qualified optimality. So, we do not view a heuristic rule that proves useful in practice to be in itself a framework for bounded rationality if it does not have additional formal properties. The rest of this section describes two additional approaches that offer such precise properties. Optimal metaraesoning Since metareasoning is a component that monitors and controls object-level deliberation, one could pose the question of whether the metareasoning process itself is optimal. Optimality here is with respect to the overall agent performance, given its fixed object-level deliberation capabilities. This is a well-defined question that sometimes has a simple answer. For example, metareasoning may focus on the single question of when to stop deliberation and take action. Depending on how the base-level component is structured, the answer may or may not be straightforward. Optimal metareasoning has been also referred to as rational metareasoning (Horvitz 1989) and metalevel rationality (Russell 1995) to distinguish it from perfect rationality. This offers one precise form of bounded rationality that we will examine in further details in the next section. It should be noted that optimal metareasoning can result in arbitrary poor agent performance. This is true because we do not impose upfront any constraints on the object-level deliberation process, in terms of either efficiency or correctness. Nevertheless, we will see later that this presents an attractive framework for bounded rationality and that performance guarantees can be established once additional constraints are imposed on the overall architecture.