Instance-Based Decision Making in Dynamic Environments : Modeling the Learning Process

This paper presents a cognitive model of how people learn to make better decisions in dynamic environments. I argue that, in dynamic environments, the main learning structure is an “instance” representing the knowledge of the situation (decision context), the decision (one among the possible actions), and an evaluation of the decision results (utility). Decision making behavior in dynamic environments turns from rule-based to instance-based as decision makers become familiar and confront similar situations. A cognitive model has been designed to illustrate this form of learning in a dynamic resource management task. Results from the cognitive model support the instance-based ideas and suggest that instances guide the recognition to relevant task cues. Decision makers become more selective in their judgments without a prescribing rule. Learning in Dynamic Decision Making Dynamic Decision Making (DDM) is characterized by multiple, interdependent decisions occurring in real-time in a continuously changing environment (Brehmer, 1990). DDM involves making decisions under uncertainty in situations in which not all possible eventualities are available nor it is possible to evaluate them within a time frame. Classic theories of judgment and choice such as the Expected Utility Theory often ignore the complexity and dynamic characteristics of the real world, and the cognitive limitations of decision makers, assuming that decision makers always choose the option with the maximum utility. Optimal decision making is impossible in DDM. Optimality requires assessing all the alternatives and predicting the likelihood of their success, which is impossible given limited information, high uncertainty and limited time. Evidence suggests that the rationality of human decision making is bounded (Simon, 1957). Bounded rationality results from both, cognitive limitations and environment constraints. A general question that drives my research is: How do decision makers improve their performance in dynamic environments? The traditional approach to improving decision effectiveness suggests the use of heuristics or general rules of action (Newell & Simon, 1972). Research has shown that a wide variety of decision rules are generally superior to unaided human decision making in some situations. However, the application of decision rules usually proves too restrictive for real world tasks (Einhorn & Hogarth, 1981; Gilboa & Schmeidler, 1995; Klein, 1997). In contrast to rule-based approaches, instance-based learning theories argue that skill development can be explained in terms of the storage of specific solutions to specific problems (Logan, 1988; Gilboa & Schmeidler, 1995). In a recently proposed Instance-Based Theory of Learning in DDM (Gonzalez, Lerch and Lebiere, 2001) we argue that, in dynamic environments, the main learning structure is an “instance” representing the knowledge of the situation (decision context), the decision (one among the possible actions), and evaluation of the decision results (utility) (SDU instance). Learning occurs by a gradual movement from rule-based to instance-based decisions, as decision makers confront and recognize similar situations. A cognitive model has been designed to illustrate this form of learning in a dynamic resource management task. The model is summarized next. Instance-Based Decision Making in the WPP task The Water Purification Plant (WPP) is a resource allocation task, isomorph of a real-world mail sorting task. WPP is a dynamic, real-time simulation in which decision makers distribute water to multiple factories by activating and de-activating pumps. Figure 1 shows a snapshot of the simulation. Water is assigned to different tanks based on a scenario unknown to the decision maker (a scenario defines the amount of water and the time of assignment). Due to electricity constraints, decision makers may only activate 5 out of 44 pumps in the system. Each factory has a deadline by which it needs to receive all the water from the system. After each deadline, the gallons of water not distributed on time to each factory are calculated and reported to the decision maker as a form of feedback. The best performance in this task is zero gallons missed. Figure 1. WPP snapshot WPP does not have one “optimal” solution. There are many sequences and combinations of decisions that may lead to the best performance. A heuristic that has proved to be better than random assignment in this task is the Time rule (Lerch, Gonzalez, Harter, 2001). The Time decision rule prescribes to activate the pump (s) for the tank with the most immediate deadline. This rule is basically the earliest time rule in operations management. The utility of the decision is calculated by subtracting the deadline time of the tank associated with the activated pump minus the time when the decision was made. The ACT-R cognitive model: CogWPP ACT-R (Anderson and Lebiere, 1998) is a cognitive modeling architecture that has been used in a wide range of tasks since its introduction in 1993. In ACT-R it is possible to represent knowledge in two forms: procedural (If-Then rules or productions) and declarative (chunks). It is also possible to retrieve this knowledge according to a set of performance/learning methods. A cognitive model of the WPP task (CogWPP) was implemented in ACT-R. The model assumes the user knows the Time rule, prior to the availability of instances. SDU instances are implemented into a chunk structure, where Situations are described by a tank and its characteristics (e.g., amount of water, chain value, deadline, etc.), Decisions are activation and deactivation of pumps and Utility is implemented in time units (minutes left to a deadline). SDU instances are generated for every decision-making situation confronted. When presented with a decision situation, the CogWPP evaluates it using any of two production rules: Instance-Based-Judgment or Rule-Based-Judgment. Figure 2 shows the Englishlike version of the Instance-BasedJudgment production. Making judgments by Figure 2. Instance -Based-Judgment production instance or by rule is determined by the similarity between the current situation and situations experienced in the past. CogWPP attempts to use stored knowledge by firing the Instance-BasedJudgment production first. If no similar situations have been experienced in the past, then CogWPP chooses to fire the Rule-Based-Judgment production. In the Instance-Based-Judgment production, CogWPP uses a scaled linear difference between two numbers to determine chunk similarity. When at least one instance in the past is found to be similar enough to the current situation, CogWPP performs a (P Instance-Based-Judgment IF The goal is to evaluate a tank AND there is at least one instance in the past with judgments for the same decision on similar tanks, in a similar time and with similar amounts of water as in the current situation THEN Determine the Utility of the current situation by blending the Utility values of similar instances from the past. Store this situation as a new SDU