A COGNITIVE AGENT ARCHITECTURE OPTIMIZED FOR ADAPTIVITY

The Engine for Composable Logical Agents with Intuitive Reorganization (ECLAIR) is a cognitive agent architecture that allows an agent to quickly adapt its behavior to new environments. ECLAIR addresses two problems in agent learning: generalizing the process of adaptation and detecting when adaptation is required. ECLAIR incorporates the main mechanisms from Piaget’s Cognitive-Stage Theory of Development [10], and it uses concepts from Damasio’s Somatic Marker Hypothesis [4] for discovery of what should be learned. ECLAIR has modules for stimuli, awareness, plan behavior, reflex behavior, control/decision making, and adaptivity. ECLAIR agents take a hybrid approach to action. In normal situations they act logically, using plans, when there is a known strategy to accomplish a task. However, when quick reaction is needed, motivation for action is intuitive or emotional. The agent fires reflexes triggered by changes in the agent's perception of personal well being. Adaptive learning extends both cognitive and emotional behavior in our architecture. We demonstrated the cognitive architecture and reflexive adaptation using a simulation for network-centric warfare logistics. 1. COGNITIVE ARCHITECTURES Researchers have strived to develop cognitive architectures since the 1980s. The main premise of these types of architectures is to emulate the human, how a person makes decisions, represents information, and learns. The two most widely known cognitive architectures with a psychological basis are SOAR [9] and ACT-R [1]. Both are hypotheses for answering Newell’s concept of a United Theory of Cognition [12]. Newell saw that in a person, there are many interacting components that must be integrated into a single comprehensive system, and believed that the single system is the source of all behavior. Thus, the goal of a cognitive architecture is to have one system that gives purpose to the many components that make up a thinking person. ACT-R is a cognitive architecture designed as an integration of components discovered in psychology research. This model is primarily meant to accurately simulate human behavior. Given a specific cognitive theory, ACT-R can be used to model the components of the theory. Once the model has been created, experiments can be made that get results very similar to human experimental results. In addition, the model can be used to extend previous theories by creating a novel experiment for the model. ACT-R also has a set of modules that represent different functional aspects of the brain. The modules interact when each module exposes part of its activity into a public buffer. The central production system uses the data in the buffers for its processing. ACT-R has primarily been used for psychological research [1]. SOAR is a cognitive architecture focused on the functional requirements of human level intelligence. SOAR attempts to recreate goal driven behavior, continuous learning from experience, and real time cognition. The goal is to have a system where memory can be directly used for action. A production system is at the heart of the architecture. The decision cycle has seven steps: Input, State Elaboration, Propose Operator, Compare Operators, Select Operator, Apply Operator, and Output [11]. SOAR’s mechanism for learning, called “chunking,” has proven to cause unexpected results and experiments have been done to replace the chunking mechanism by reinforcement learning [11]. The paper is organized as follows: Section 2 describes the theoretical background; Section 3 is the architecture description; Section 4 talks about adaptivity in detail; Section 5 describes the logistics application, and Section 6 concludes. 2. MOTIVATION AND THEORETICAL BACKGROUND FOR ECLAIR The Engine for Composable Logical Agents with Intuitive Reorganization (ECLAIR) was designed as a cognitive agent architecture amenable to adaptivity and learning. Agent adaptivity and learning are important for real world domains, where the situation in the environment dynamically changes and the agents have to adapt quickly. One domain of special interest is Sense and Respond Logistics (SRL). SRL aims to deliver precise, agile support through adaptive and responsive demand and support networks [13]. Automation technologies supporting SRL should respond to events that occur, as well as aid in the perception and anticipation of short-term needs. The cognitive agents working in NCL domains must adapt by reorganizing themselves to suit the environment, or by modifying their behavior to improve their effectiveness. ECLAIR is a cognitive model based on developmental cognitive psychology research and neuropsychological research. Though many developmental theories contributed to the ECLAIR model, the two most prominent are Piaget’s adaptation theory [10], and Damasio’s Somatic Marker Hypothesis [4]. Piaget's adaptation theory consists of three main concepts: Assimilation, Accommodation, and Equilibration. Assimilation processes unfamiliar input in the same way that one would process the most similar familiar input. Accommodation changes the processes to deal with unfamiliar input. Equilibration balances the aforementioned processes. The Somatic Marker Hypothesis stems from Damasio’s belief that gut feelings are a basis for decisions in addition to reasoning, which is often thought of as the only basis for decision. Damasio defines a somatic marker as a trigger that recalls feelings related to previously made decisions. The decision is made based on the best expected feeling given the available actions for the current circumstance. Each memory of feelings becomes a somatic marker that is used as a mapping from circumstance to action. ECLAIR agents contain modules for stimuli, awareness, plan behavior, reflex behavior, control/decision making, and adaptivity. The interaction between awareness, behavior and adaptivity allows agents to modify their behavior based on their perception of world and self-states. Self-states are represented by homeostatic vectors (HVs), where the comfortable level for the agent is a range, not a threshold. Agent well being is an emotional state that is computed as a function of the agent’s HVs. 3. ECLAIR ARCHITECTURE The main elements of ECLAIR architecture (Fig. 1) include the following modules: Stimulus, Awareness, Reflex Behavior, Plan Behavior, Adaptivity, and Action/Decision. Reflex Behavior Module