Above are papers related to the following paper: [Paper_ID] Paper #1105033. 	[Title] Learning to take risks [Abstract] Agents that learn about other agents and can exploit this information possess a distinct advantage in competitive situations. Games provide stylized adversarial environments to study agent learning strategies. Researchers have developed game playing programs that learn to play better from experience. We have developed a learning program that does not learn to play better, but learns to identify and exploit the weaknesses of a particular opponent by repeatedly playing it over several games. We propose a scheme for learning opponent action probabilities and a utility maximization framework that exploits this learned opponent model. We show that the proposed expected utility maximization strategy generalizes the traditional maximin strategy, and allows players to benefit by taking calculated risks that are avoided by the max-imin strategy. Experiments in the popular board game of Connect-4 show that a learning player consistently outperforms a non-learning player when pitted against another automated player using a weaker heuristic. Though our proposed mechanism does not improve the skill level of a computer player, it does improve its ability to play more effectively against a weaker opponent. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #114308. 	[Title] The Role of Forgetting in Learning [Abstract] This paper is a discussion of the relationship between learning and forgetting. An analysis of the economics of learning is carried out and it is argued that knowledge can sometimes have a negative value. A series of experiments involving a program which learns to traverse state spaces is described. It is shown that most of the knowledge acquired is of negative value even though it is correct and was acquired solving similar problems. It is shown that the value of the knowledge depends on what else is known and that random forgetting can sometimes lead to substantial improvements in performance. It is concluded that research into knowledge acquisition should take seriously the possibility that knowledge may sometimes be harmful. The view is taken that learning and forgetting are complementary processes which construct and maintain useful representations of experience. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #64519. 	[Title] Utilization Filtering a method for reducing the inherent harmfulness of deductively learned knowledge field of [Abstract] This paper highlights a phenomenon that causes deductively learned knowledge to be harmful when used for problem solving. The problem occurs when deductive problem solvers encounter a failure branch of the search tree. The backtracking mechanism of such problem solvers will force the program to traverse the whole subtree thus visiting many nodes twice - once by using the deductively learned rule and once by using the rules that generated the learned rule in the first place. We suggest an approach called utilization filtering to solve that problem. Learners that use this approach submit to the problem solver a filter function together with the knowledge that was acquired. The function decides for each problem whether to use the learned knowledge and what part of it to use. We have tested the idea in the context of a lemma learning system, where the filter uses the probability of a subgoal failing to decide whether to turn lemma usage off. Experiments show an improvement of performance by a factor of 3. This paper is concerned with a particular type of harmful redundancy that occurs in deductive problem solvers that employ backtracking in their search procedure, and use deductively learned knowledge to accelerate the search. The problem is that in failure branches of the search tree, the backtracking mechanism of the problem solver forces exploration of the whole subtree. Thus, the search procedure will visit many states twice - once by using the deductively learned rule, and once by using the search path that produced the rule in the first place. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #1111304. 	[Title] Using Temporal-Difference Reinforcement Learning to Improve Decision-Theoretic Utilities for Diagnosis [Abstract] Probability theory represents and manipulates uncertainties, but cannot tell us how to behave. For that we need utility theory which assigns values to the usefulness of different states, and decision theory which concerns optimal rational decisions. There are many methods for probability modeling, but few for learning utility and decision models. We use reinforcement learning to find the optimal sequence of questions in a diagnosis situation while maintaining a high accuracy. Automated diagnosis on a heart-disease domain is used to demonstrate that temporal-difference learning can improve diagnosis. On the Cleveland heart-disease database our results are better than those reported from all previous methods. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #6151. 	[Title] Strategy Learning with Multilayer Connectionist Representations 1 [Abstract] Results are presented that demonstrate the learning and fine-tuning of search strategies using connectionist mechanisms. Previous studies of strategy learning within the symbolic, production-rule formalism have not addressed fine-tuning behavior. Here a two-layer connectionist system is presented that develops its search from a weak to a task-specific strategy and fine-tunes its performance. The system is applied to a simulated, real-time, balance-control task. We compare the performance of one-layer and two-layer networks, showing that the ability of the two-layer network to discover new features and thus enhance the original representation is critical to solving the balancing task. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #1113614. 	[Title] NeuroDraughts: the role of representation, search, training regime and architecture in a TD draughts player [Abstract] NeuroDraughts is a draughts playing program similar in approach to NeuroGammon and NeuroChess [Tesauro, 1992, Thrun, 1995]. It uses an artificial neural network trained by the method of temporal difference learning to learn by self-play how to play the game of draughts. This paper discusses the relative contribution of board representation, search depth, training regime, architecture and run time parameters to the strength of the TDplayer produced by the system. Keywords: Temporal Difference Learning, Input representation, Search, Draughts. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #12576. 	[Title] Learning Sequential Decision Rules Using Simulation Models and Competition [Abstract] The problem of learning decision rules for sequential tasks is addressed, focusing on the problem of learning tactical decision rules from a simple flight simulator. The learning method relies on the notion of competition and employs genetic algorithms to search the space of decision policies. Several experiments are presented that address issues arising from differences between the simulation model on which learning occurs and the target environment on which the decision rules are ultimately tested. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #35490. 	[Title] A Model-Based Approach to Blame-Assignment in Design [Abstract] We analyze the blame-assignment task in the context of experience-based design and redesign of physical devices. We identify three types of blame-assignment tasks that differ in the types of information they take as input: the design does not achieve a desired behavior of the device, the design results in an undesirable behavior, a specific structural element in the design misbehaves. We then describe a model-based approach for solving the blame-assignment task. This approach uses structure-behavior-function models that capture a designer's comprehension of the way a device works in terms of causal explanations of how its structure results in its behaviors. We also address the issue of indexing the models in memory. We discuss how the three types of blame-assignment tasks require different types of indices for accessing the models. Finally we describe the KRITIK2 system that implements and evaluates this model-based approach to blame assignment. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #35. 	[Title] 4 Implementing Application Specific Routines  Genetic algorithms in search, optimization, and machine learning. Reading, MA: Addison-Wesley. [Abstract] To implement a specific application, you should only have to change the file app.c. Section 2 describes the routines in app.c in detail. If you use additional variables for your specific problem, the easiest method of making them available to other program units is to declare them in sga.h and external.h. However, take care that you do not redeclare existing variables. Two example applications files are included in the SGA-C distribution. The file app1.c performs the simple example problem included with the Pascal version; finding the maximum of x 10 , where x is an integer interpretation of a chromosome. A slightly more complex application is include in app2.c. This application illustrates two features that have been added to SGA-C. The first of these is the ithruj2int function, which converts bits i through j in a chromosome to an integer. The second new feature is the utility pointer that is associated with each population member. The example application interprets each chromosome as a set of concatenated integers in binary form. The lengths of these integer fields is determined by the user-specified value of field size, which is read in by the function app data(). The field size must be less than the smallest of the chromosome length and the length of an unsigned integer. An integer array for storing the interpreted form of each chromosome is dynamically allocated and assigned to the chromosome's utility pointer in app malloc(). The ithruj2int routine (see utility.c) is used to translate each chromosome into its associated vector. The fitness for each chromosome is simply the sum of the squares of these integers. This example application will function for any chromosome length. SGA-C is intended to be a simple program for first-time GA experimentation. It is not intended to be definitive in terms of its efficiency or the grace of its implementation. The authors are interested in the comments, criticisms, and bug reports from SGA-C users, so that the code can be refined for easier use in subsequent versions. Please email your comments to rob@galab2.mh.ua.edu, or write to TCGA: The authors gratefully acknowledge support provided by NASA under Grant NGT-50224 and support provided by the National Science Foundation under Grant CTS-8451610. We also thank Hillol Kargupta for donating his tournament selection implementation. Booker, L. B. (1982). Intelligent behavior as an adaptation to the task environment (Doctoral dissertation, Technical Report No. 243. Ann Arbor: University of Michigan, Logic of Computers Group). Dissertations Abstracts International, 43(2), 469B. (University Microfilms No. 8214966) Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #28456. 	[Title] A Competitive Approach to Game Learning [Abstract] Machine learning of game strategies has often depended on competitive methods that continually develop new strategies capable of defeating previous ones. We use a very inclusive definition of game and consider a framework within which a competitive algorithm makes repeated use of a strategy learning component that can learn strategies which defeat a given set of opponents. We describe game learning in terms of sets H and X of first and second player strategies, and connect the model with more familiar models of concept learning. We show the importance of the ideas of teaching set [20] and specification number [19] k in this new context. The performance of several competitive algorithms is investigated, using both worst-case and randomized strategy learning algorithms. Our central result (Theorem 4) is a competitive algorithm that solves games in a total number of strategies polynomial in lg(jHj), lg(jX j), and k. Its use is demonstrated, including an application in concept learning with a new kind of counterexample oracle. We conclude with a complexity analysis of game learning, and list a number of new questions arising from this work. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #682666. 	[Title] Multiagent Reinforcement Learning: Theoretical Framework and an Algorithm [Abstract] In this paper, we adopt general-sum stochastic games as a framework for multiagent reinforcement learning. Our work extends previous work by Littman on zero-sum stochastic games to a broader framework. We design a multiagent Q-learning method under this framework, and prove that it converges to a Nash equilibrium under specified conditions. This algorithm is useful for finding the optimal strategy when there exists a unique Nash equilibrium in the game. When there exist multiple Nash equilibria in the game, this algorithm should be combined with other learning techniques to find optimal strategies. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #10796. 	[Title] An Improved Algorithm for Incremental Induction of Decision Trees [Abstract] Technical Report 94-07 February 7, 1994 (updated April 25, 1994) This paper will appear in Proceedings of the Eleventh International Conference on Machine Learning. Abstract This paper presents an algorithm for incremental induction of decision trees that is able to handle both numeric and symbolic variables. In order to handle numeric variables, a new tree revision operator called `slewing' is introduced. Finally, a non-incremental method is given for finding a decision tree based on a direct metric of a candidate tree. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #170798. 	[Title] Learning Problem-Solving Concepts by Reflecting on Problem Solving [Abstract] Learning and problem solving are intimately related: problem solving determines the knowledge requirements of the reasoner which learning must fulfill, and learning enables improved problem-solving performance. Different models of problem solving, however, recognize different knowledge needs, and, as a result, set up different learning tasks. Some recent models analyze problem solving in terms of generic tasks, methods, and subtasks. These models require the learning of problem-solving concepts such as new tasks and new task decompositions. We view reflection as a core process for learning these problem-solving concepts. In this paper, we identify the learning issues raised by the task-structure framework of problem solving. We view the problem solver as an abstract device, and represent how it works in terms of a structure-behavior-function model which specifies how the knowledge and reasoning of the problem solver results in the accomplishment of its tasks. We describe how this model enables reflection, and how model-based reflection enables the reasoner to adapt its task structure to produce solutions of better quality. The Autognostic system illustrates this reflection process. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #210871. 	[Title] Coevolving High-Level Representations [Abstract]  Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #1103162. 	[Title] Team-Partitioned, Opaque-Transition Reinforcement Learning [Abstract] In this paper, we present a novel multi-agent learning paradigm called team-partitioned, opaque-transition reinforcement learning (TPOT-RL). TPOT-RL introduces the concept of using action-dependent features to generalize the state space. In our work, we use a learned action-dependent feature space. TPOT-RL is an effective technique to allow a team of agents to learn to cooperate towards the achievement of a specific goal. It is an adaptation of traditional RL methods that is applicable in complex, non-Markovian, multi-agent domains with large state spaces and limited training opportunities. Multi-agent scenarios are opaque-transition, as team members are not always in full communication with one another and adversaries may affect the environment. Hence, each learner cannot rely on having knowledge of future state transitions after acting in the world. TPOT-RL enables teams of agents to learn effective policies with very few training examples even in the face of a large state space with large amounts of hidden state. The main responsible features are: dividing the learning task among team members, using a very coarse, action-dependent feature space, and allowing agents to gather reinforcement directly from observation of the environment. TPOT-RL is fully implemented and has been tested in the robotic soccer domain, a complex, multi-agent framework. This paper presents the algorithmic details of TPOT-RL as well as empirical results demonstrating the effectiveness of the developed multi-agent learning approach with learned features. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #299195. 	[Title] Learning to Play Games From Experience: An Application of Artificial Neural Networks and Temporal Difference Learning [Abstract]  Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #38205. 	[Title] Competitive Environments Evolve Better Solutions for Complex Tasks [Abstract]  Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #299197. 	[Title] TD Learning of Game Evaluation Functions with Hierarchical Neural Architectures [Abstract]  Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #87363. 	[Title] Computation. Automated Synthesis of Analog Electrical Circuits by Means of Genetic Programming [Abstract] The design (synthesis) of analog electrical circuits starts with a high-level statement of the circuit's desired behavior and requires creating a circuit that satisfies the specified design goals. Analog circuit synthesis entails the creation of both the topology and the sizing (numerical values) of all of the circuit's components. The difficulty of the problem of analog circuit synthesis is well known and there is no previously known general automated technique for synthesizing an analog circuit from a high-level statement of the circuit's desired behavior. This paper presents a single uniform approach using genetic programming for the automatic synthesis of both the topology and sizing of a suite of eight different prototypical analog circuits, including a lowpass filter, a crossover (woofer and tweeter) filter, a source identification circuit, an amplifier, a computational circuit, a time-optimal controller circuit, a temperaturesensing circuit, and a voltage reference circuit. The problemspecific information required for each of the eight problems is minimal and consists primarily of the number of inputs and outputs of the desired circuit, the types of available components, and a fitness measure that restates the high-level statement of the circuit's desired behavior as a measurable mathematical quantity. The eight genetically evolved circuits constitute an instance of an evolutionary computation technique producing results on a task that is usually thought of as requiring human intelligence. The fact that a single uniform approach yielded a satisfactory design for each of the eight circuits as well as the fact that a satisfactory design was created on the first or second run of each problem are evidence for the general applicability of genetic programming for solving the problem of automatic synthesis of analog electrical circuits. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #17476. 	[Title] Markov games as a framework for multi-agent reinforcement learning [Abstract] In the Markov decision process (MDP) formalization of reinforcement learning, a single adaptive agent interacts with an environment defined by a probabilistic transition function. In this solipsistic view, secondary agents can only be part of the environment and are therefore fixed in their behavior. The framework of Markov games allows us to widen this view to include multiple adaptive agents with interacting or competing goals. This paper considers a step in this direction in which exactly two agents with diametrically opposed goals share an environment. It describes a Q-learning-like algorithm for finding optimal policies and demonstrates its application to a simple two-player game in which the optimal policy is probabilistic. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #161221. 	[Title] A Local Learning Algorithm for Dynamic Feedforward and Recurrent Networks [Abstract] Most known learning algorithms for dynamic neural networks in non-stationary environments need global computations to perform credit assignment. These algorithms either are not local in time or not local in space. Those algorithms which are local in both time and space usually can not deal sensibly with `hidden units'. In contrast, as far as we can judge by now, learning rules in biological systems with many `hidden units' are local in both space and time. In this paper we propose a parallel on-line learning algorithm which performs local computations only, yet still is designed to deal with hidden units and with units whose past activations are `hidden in time'. The approach is inspired by Holland's idea of the bucket brigade for classifier systems, which is transformed to run on a neural network with fixed topology. The result is a feedforward or recurrent `neural' dissipative system which is consuming `weight-substance' and permanently trying to distribute this substance onto its connections in an appropriate way. Simple experiments demonstrating the feasability of the algorithm are reported. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #6213. 	[Title] Machine Learning Learning to Predict by the Methods of Temporal Differences Keywords: Incremental learning, prediction, [Abstract] This article introduces a class of incremental learning procedures specialized for prediction|that is, for using past experience with an incompletely known system to predict its future behavior. Whereas conventional prediction-learning methods assign credit by means of the difference between predicted and actual outcomes, the new methods assign credit by means of the difference between temporally successive predictions. Although such temporal-difference methods have been used in Samuel's checker player, Holland's bucket brigade, and the author's Adaptive Heuristic Critic, they have remained poorly understood. Here we prove their convergence and optimality for special cases and relate them to supervised-learning methods. For most real-world prediction problems, temporal-difference methods require less memory and less peak computation than conventional methods; and they produce more accurate predictions. We argue that most problems to which supervised learning is currently applied are really prediction problems of the sort to which temporal-difference methods can be applied to advantage. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #6215. 	[Title] Planning by Incremental Dynamic Programming [Abstract] This paper presents the basic results and ideas of dynamic programming as they relate most directly to the concerns of planning in AI. These form the theoretical basis for the incremental planning methods used in the integrated architecture Dyna. These incremental planning methods are based on continually updating an evaluation function and the situation-action mapping of a reactive system. Actions are generated by the reactive system and thus involve minimal delay, while the incremental planning process guarantees that the actions and evaluation function will eventually be optimal|no matter how extensive a search is required. These methods are well suited to stochastic tasks and to tasks in which a complete and accurate model is not available. For tasks too large to implement the situation-action mapping as a table, supervised-learning methods must be used, and their capabilities remain a significant limitation of the approach. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #334153. 	[Title] Machine Learning by Function Decomposition [Abstract] We present a new machine learning method that, given a set of training examples, induces a definition of the target concept in terms of a hierarchy of intermediate concepts and their definitions. This effectively decomposes the problem into smaller, less complex problems. The method is inspired by the Boolean function decomposition approach to the design of digital circuits. To cope with high time complexity of finding an optimal decomposition, we propose a suboptimal heuristic algorithm. The method, implemented in program HINT (HIerarchy Induction Tool), is experimentally evaluated using a set of artificial and real-world learning problems. It is shown that the method performs well both in terms of classification accuracy and discovery of meaningful concept hierarchies. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #976334. 	[Title] Applying Online Search Techniques to Continuous-State Reinforcement Learning key to the success of the local [Abstract] In this paper, we describe methods for efficiently computing better solutions to control problems in continuous state spaces. We provide algorithms that exploit online search to boost the power of very approximate value functions discovered by traditional reinforcement learning techniques. We examine local searches, where the agent performs a finite-depth lookahead search, and global searches, where the agent performs a search for a trajectory all the way from the current state to a goal state. The key to the success of the global methods lies in using aggressive state-space search techniques such as uniform-cost search and A fl , tamed into a tractable form by exploiting neighborhood relations and trajectory constraints that arise from continuous-space dynamic control. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #1112911. 	[Title] Evolution of Iteration in Genetic Programming D a v d A The solution to many [Abstract] This paper introduces the new operation of restricted iteration creation that automatically Genetic programming extends Holland's genetic algorithm to the task of automatic programming. Early work on genetic programming demonstrated that it is possible to evolve a sequence of work-performing steps in a single result-producing branch (that is, a one-part "main" program). The book Genetic Programming: On the Programming of Computers by Means of Natural Selection (Koza 1992) describes an extension of Holland's genetic algorithm in which the genetic population consists of computer programs (that is, compositions of primitive functions and terminals). See also Koza and Rice (1992). In the most basic form of genetic programming (where only a single result-producing branch is evolved), genetic programming demonstrated the capability to discover a sequence (as to both its length and its content) of work-performing steps that is sufficient to produce a satisfactory solution to several problems, including many problems that have been used over the years as benchmarks in machine learning and artificial intelligence. Before applying genetic programming to a problem, the user must perform five major preparatory steps, namely identifying the terminals (inputs) of the to-be-evolved programs, identifying the primitive functions (operations) contained in the to-be-evolved programs, creating the fitness measure for evaluating how well a given program does at solving the problem at hand, choosing certain control parameters (notably population size and number of generations to be run), and determining the termination criterion and method of result designation (typically the best-so-far individual from the populations produced during the run). creates a restricted iteration-performing Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #13269. 	[Title] database [Abstract] MIT Computational Cognitive Science Technical Report 9701 Abstract We describe variational approximation methods for efficient probabilistic reasoning, applying these methods to the problem of diagnostic inference in the QMR-DT database. The QMR-DT database is a large-scale belief network based on statistical and expert knowledge in internal medicine. The size and complexity of this network render exact probabilistic diagnosis infeasible for all but a small set of cases. This has hindered the development of the QMR- DT network as a practical diagnostic tool and has hindered researchers from exploring and critiquing the diagnostic behavior of QMR. In this paper we describe how variational approximation methods can be applied to the QMR network, resulting in fast diagnostic inference. We evaluate the accuracy of our methods on a set of standard diagnostic cases and compare to stochastic sampling methods. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #170338. 	[Title] Learning Team Strategies With Multiple Policy-Sharing Agents: A Soccer Case Study [Abstract] We use simulated soccer to study multiagent learning. Each team's players (agents) share action set and policy but may behave differently due to position-dependent inputs. All agents making up a team are rewarded or punished collectively in case of goals. We conduct simulations with varying team sizes, and compare two learning algorithms: TD-Q learning with linear neural networks (TD-Q) and Probabilistic Incremental Program Evolution (PIPE). TD-Q is based on evaluation functions (EFs) mapping input/action pairs to expected reward, while PIPE searches policy space directly. PIPE uses adaptive "probabilistic prototype trees" to synthesize programs that calculate action probabilities from current inputs. Our results show that TD-Q encounters several difficulties in learning appropriate shared EFs. PIPE, however, does not depend on EFs and can find good policies faster and more reliably. This suggests that in multiagent learning scenarios direct search through policy space can offer advantages over EF-based approaches. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #19045. 	[Title] AUTOMATED TOPOLOGY AND SIZING OF ANALOG CIRCUITS AUTOMATED DESIGN OF BOTH THE TOPOLOGY AND SIZING [Abstract] This paper describes an automated process for designing analog electrical circuits based on the principles of natural selection, sexual recombination, and developmental biology. The design process starts with the random creation of a large population of program trees composed of circuit-constructing functions. Each program tree specifies the steps by which a fully developed circuit is to be progressively developed from a common embryonic circuit appropriate for the type of circuit that the user wishes to design. Each fully developed circuit is translated into a netlist, simulated using a modified version of SPICE, and evaluated as to how well it satisfies the user's design requirements. The fitness measure is a user-written computer program that may incorporate any calculable characteristic or combination of characteristics of the circuit, including the circuit's behavior in the time domain, its behavior in the frequency domain, its power consumption, the number of components, cost of components, or surface area occupied by its components. The population of program trees is genetically bred over a series of many generations using genetic programming. Genetic programming is driven by a fitness measure and employs genetic operations such as Darwinian reproduction, sexual recombination (crossover), and occasional mutation to create offspring. This automated evolutionary process produces both the topology of the circuit and the numerical values for each component. This paper describes how genetic programming can evolve the circuit for a difficult-to-design low-pass filter. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #892139. 	[Title] Using Communication to Reduce Locality in Distributed Multi-Agent Learning [Abstract] This paper attempts to bridge the fields of machine learning, robotics, and distributed AI. It discusses the use of communication in reducing the undesirable effects of locality in fully distributed multi-agent systems with multiple agents/robots learning in parallel while interacting with each other. Two key problems, hidden state and credit assignment, are addressed by applying local undirected broadcast communication in a dual role: as sensing and as reinforcement. The methodology is demonstrated on two multi-robot learning experiments. The first describes learning a tightly-coupled coordination task with two robots, the second a loosely-coupled task with four robots learning social rules. Communication is used to 1) share sensory data to overcome hidden state and 2) share reinforcement to overcome the credit assignment problem between the agents and bridge the gap between local/individual and global/group payoff. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #20972. 	[Title] Exploratory Learning in the Game of GO [Abstract] This paper considers the importance of exploration to game-playing programs which learn by playing against opponents. The central question is whether a learning program should play the move which offers the best chance of winning the present game, or if it should play the move which has the best chance of providing useful information for future games. An approach to addressing this question is developed using probability theory, and then implemented in two different learning methods. Initial experiments in the game of Go suggest that a program which takes exploration into account can learn better against a knowledgeable opponent than a program which does not. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #134128. 	[Title] Learning To Play the Game of Chess [Abstract] This paper presents NeuroChess, a program which learns to play chess from the final outcome of games. NeuroChess learns chess board evaluation functions, represented by artificial neural networks. It integrates inductive neural network learning, temporal differencing, and a variant of explanation-based learning. Performance results illustrate some of the strengths and weaknesses of this approach. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #6385. 	[Title] Multi-Agent Reinforcement Learning: Independent vs. Cooperative Agents [Abstract] Intelligent human agents exist in a cooperative social environment that facilitates learning. They learn not only by trial-and-error, but also through cooperation by sharing instantaneous information, episodic experience, and learned knowledge. The key investigations of this paper are, "Given the same number of reinforcement learning agents, will cooperative agents outperform independent agents who do not communicate during learning?" and "What is the price for such cooperation?" Using independent agents as a benchmark, cooperative agents are studied in following ways: (1) sharing sensation, (2) sharing episodes, and (3) sharing learned policies. This paper shows that (a) additional sensation from another agent is beneficial if it can be used efficiently, (b) sharing learned policies or episodes among agents speeds up learning at the cost of communication, and (c) for joint tasks, agents engaging in partnership can significantly outperform independent agents although they may learn slowly in the beginning. These tradeoffs are not just limited to multi-agent reinforcement learning. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #595056. 	[Title] Using Genetic Programming to Evolve Board Evaluation Functions [Abstract]  Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #265203. 	[Title] CFS-C: A Package of Domain Independent Subroutines for Implementing Classifier Systems in Arbitrary, User-Defined Environments. [Abstract]  Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #117. 	[Title] Generalized Markov Decision Processes: Dynamic-programming and Reinforcement-learning Algorithms [Abstract] The problem of maximizing the expected total discounted reward in a completely observable Markovian environment, i.e., a Markov decision process (mdp), models a particular class of sequential decision problems. Algorithms have been developed for making optimal decisions in mdps given either an mdp specification or the opportunity to interact with the mdp over time. Recently, other sequential decision-making problems have been studied prompting the development of new algorithms and analyses. We describe a new generalized model that subsumes mdps as well as many of the recent variations. We prove some basic results concerning this model and develop generalizations of value iteration, policy iteration, model-based reinforcement-learning, and Q-learning that can be used to make optimal decisions in the generalized model under various assumptions. Applications of the theory to particular models are described, including risk-averse mdps, exploration-sensitive mdps, sarsa, Q-learning with spreading, two-player games, and approximate max picking via sampling. Central to the results are the contraction property of the value operator and a stochastic-approximation theorem that reduces asynchronous convergence to synchronous convergence. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #33013. 	[Title] Learning Conventions in Multiagent Stochastic Domains using Likelihood Estimates [Abstract] Fully cooperative multiagent systemsthose in which agents share a joint utility modelis of special interest in AI. A key problem is that of ensuring that the actions of individual agents are coordinated, especially in settings where the agents are autonomous decision makers. We investigate approaches to learning coordinated strategies in stochastic domains where an agent's actions are not directly observable by others. Much recent work in game theory has adopted a Bayesian learning perspective to the more general problem of equilibrium selection, but tends to assume that actions can be observed. We discuss the special problems that arise when actions are not observable, including effects on rates of convergence, and the effect of action failure probabilities and asymmetries. We also use likelihood estimates as a means of generalizing fictitious play learning models in our setting. Finally, we propose the use of maximum likelihood as a means of removing strategies from consideration, with the aim of convergence to a conventional equilibrium, at which point learning and deliberation can cease. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #887. 	[Title] Some studies in machine learning using the game of checkers. IBM Journal, 3(3):211-229, 1959. Some [Abstract]  Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #1129208. 	[Title] Figure 3: Average model size accepted from a ran-dom prefix-closed samples of various size, and [Abstract] that is based on Angluin's L fl algorithm. The algorithm maintains a model consistent with its past examples. When a new counterexample arrives it tries to extend the model in a minimal fashion. We conducted a set of experiments where random automata that represent different strategies were generated, and the algorithm tried to learn them based on prefix-closed samples of their behavior. The algorithm managed to learn very compact models that agree with the samples. The size of the sample had a small effect on the size of the model. The experimental results suggest that for random prefix-closed samples the algorithm behaves well. However, following Angluin's result on the difficulty of learning almost uniform complete samples [ An-gluin, 1978 ] , it is obvious that our algorithm does not solve the complexity issue of inferring a DFA from a general prefix-closed sample. We are currently looking for classes of prefix-closed samples in which US-L* behaves well. [ Carmel and Markovitch, 1994 ] D. Carmel and S. Markovitch. The M* algorithm: Incorporating opponent models into adversary search. Technical Report CIS report 9402, Technion, March 1994. [ Carmel and Markovitch, 1995 ] D. Carmel and S. Markovitch. Unsupervised learning of finite automata: A practical approach. Technical Report CIS report 9504, Technion, March 1995. [ Shoham and Tennenholtz, 1994 ] Y. Shoham and M. Tennenholtz. Co-Learning and the evolution of social activity. Technical Report STAN-CS-TR-94-1511, Stanford Univrsity, Department of Computer Science, 1994. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #1122425. 	[Title] Reinforcement Learning with Imitation in Heterogeneous Multi-Agent Systems [Abstract] The application of decision making and learning algorithms to multi-agent systems presents many interestingresearch challenges and opportunities. Among these is the ability for agents to learn how to act by observing or imitating other agents. We describe an algorithm, the IQ-algorithm, that integrates imitation with Q-learning. Roughly, a Q-learner uses the observations it has made of an expert agent to bias its exploration in promising directions. This algorithm goes beyond previous work in this direction by relaxing the oft-made assumptions that the learner (observer) and the expert (observed agent) share the same objectives and abilities. Our preliminary experiments demonstrate significant transfer between agents using the IQ-model and in many cases reductions in training time. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #198653. 	[Title] Information filtering: Selection mechanisms in learning systems. Machine Learning, 10:113-151, 1993. Generalization as search. Artificial [Abstract]  Please summarise all the given information.

Paper #1105033 has the title `Learning to take risks  
` and the abstract `Agents that learn about other agents and can exploit this information possess a distinct advantage in competitive situations. Games provide stylized adversarial environments to study agent learning strategies. Researchers have developed game playing programs that learn to play better from experience. We have developed a learning program that does not learn to play better, but learns to identify and exploit the weaknesses of a particular opponent by repeatedly playing it over several games. We propose a scheme for learning opponent action probabilities and a utility maximization framework that exploits this learned opponent model. We show that the proposed expected utility maximization strategy generalizes the traditional maximin strategy, and allows players to benefit by taking calculated risks that are avoided by the max-imin strategy. Experiments in the popular board game of Connect-4 show that a learning player consistently outperforms a non-learning player when pitted against another automated player using a weaker heuristic. Though our proposed mechanism does not improve the skill level of a computer player, it does improve its ability to play more effectively against a weaker opponent. 
`.
Question: Which category should Paper #1105033 be classified as ?
You can select one from ['Neural_Networks', 'Case_Based', 'Theory', 'Reinforcement_Learning', 'Probabilistic_Methods', 'Rule_Learning', 'Genetic_Algorithms'].


Paper #1105033 has the title `Learning to take risks  
` and the abstract `Agents that learn about other agents and can exploit this information possess a distinct advantage in competitive situations. Games provide stylized adversarial environments to study agent learning strategies. Researchers have developed game playing programs that learn to play better from experience. We have developed a learning program that does not learn to play better, but learns to identify and exploit the weaknesses of a particular opponent by repeatedly playing it over several games. We propose a scheme for learning opponent action probabilities and a utility maximization framework that exploits this learned opponent model. We show that the proposed expected utility maximization strategy generalizes the traditional maximin strategy, and allows players to benefit by taking calculated risks that are avoided by the max-imin strategy. Experiments in the popular board game of Connect-4 show that a learning player consistently outperforms a non-learning player when pitted against another automated player using a weaker heuristic. Though our proposed mechanism does not improve the skill level of a computer player, it does improve its ability to play more effectively against a weaker opponent. 
`.
Question: Which category should Paper #1105033 be classified as ?
You can select one from ['Neural_Networks', 'Case_Based', 'Theory', 'Reinforcement_Learning', 'Probabilistic_Methods', 'Rule_Learning', 'Genetic_Algorithms'].

This paper should be classified into: Reinforcement_Learning