Above are papers related to the following paper: [Paper_ID] Paper #4584. 	[Title] References elements that can solve difficult learning control problems. on Simulation of Adaptive Behavior, pages [Abstract] Miller, G. (1956). The magical number seven, plus or minus two: Some limits on our capacity for processing information. The Psychological Review, 63(2):81-97. Schmidhuber, J. (1990b). Towards compositional learning with dynamic neural networks. Technical Report FKI-129-90, Technische Universitat Munchen, Institut fu Informatik. Servan-Schreiber, D., Cleermans, A., and McClelland, J. (1988). Encoding sequential structure in simple recurrent networks. Technical Report CMU-CS-88-183, Carnegie Mellon University, Computer Science Department. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #1026. 	[Title] Genes, Phenes and the Baldwin Effect: Learning and Evolution in a Simulated Population [Abstract] The Baldwin Effect, first proposed in the late nineteenth century, suggests that the course of evolutionary change can be influenced by individually learned behavior. The existence of this effect is still a hotly debated topic. In this paper clear evidence is presented that learning-based plasticity at the phenotypic level can and does produce directed changes at the genotypic level. This research confirms earlier experimental work done by others, notably Hinton & Nowlan (1987). Further, the amount of plasticity of the learned behavior is shown to be crucial to the size of the Baldwin Effect: either too little or too much and the effect disappears or is significantly reduced. Finally, for learnable traits, the case is made that over many generations it will become easier for the population as a whole to learn these traits (i.e. the phenotypic plasticity of these traits will increase). In this gradual transition from a genetically driven population to one driven by learning, the importance of the Baldwin Effect decreases. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #6151. 	[Title] Strategy Learning with Multilayer Connectionist Representations 1 [Abstract] Results are presented that demonstrate the learning and fine-tuning of search strategies using connectionist mechanisms. Previous studies of strategy learning within the symbolic, production-rule formalism have not addressed fine-tuning behavior. Here a two-layer connectionist system is presented that develops its search from a weak to a task-specific strategy and fine-tunes its performance. The system is applied to a simulated, real-time, balance-control task. We compare the performance of one-layer and two-layer networks, showing that the ability of the two-layer network to discover new features and thus enhance the original representation is critical to solving the balancing task. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #6152. 	[Title] Reinforcement Learning with Modular Neural Networks for Control [Abstract] Reinforcement learning methods can be applied to control problems with the objective of optimizing the value of a function over time. They have been used to train single neural networks that learn solutions to whole tasks. Jacobs and Jordan [5] have shown that a set of expert networks combined via a gating network can more quickly learn tasks that can be decomposed. Even the decomposition can be learned. Inspired by Boyan's work of modular neural networks for learning with temporal-difference methods [4], we modify the reinforcement learning algorithm called Q-Learning to train a modular neural network to solve a control problem. The resulting algorithm is demonstrated on the classical pole-balancing problem. The advantage of such a method is that it makes it possible to deal with complex dynamic control problem effectively by using task decomposition and competitive learning. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #6155. 	[Title] On the Computational Economics of Reinforcement Learning [Abstract] Following terminology used in adaptive control, we distinguish between indirect learning methods, which learn explicit models of the dynamic structure of the system to be controlled, and direct learning methods, which do not. We compare an existing indirect method, which uses a conventional dynamic programming algorithm, with a closely related direct reinforcement learning method by applying both methods to an infinite horizon Markov decision problem with unknown state-transition probabilities. The simulations show that although the direct method requires much less space and dramatically less computation per control action, its learning ability in this task is superior to, or compares favorably with, that of the more complex indirect method. Although these results do not address how the methods' performances compare as problems become more difficult, they suggest that given a fixed amount of computational power available per control action, it may be better to use a direct reinforcement learning method augmented with indirect techniques than to devote all available resources to a computation-ally costly indirect method. Comprehensive answers to the questions raised by this study depend on many factors making up the eco nomic context of the computation. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #1114125. 	[Title] H-learning: A Reinforcement Learning Method to Optimize Undiscounted Average Reward [Abstract] In this paper, we introduce a model-based reinforcement learning method called H-learning, which optimizes undiscounted average reward. We compare it with three other reinforcement learning methods in the domain of scheduling Automatic Guided Vehicles, transportation robots used in modern manufacturing plants and facilities. The four methods differ along two dimensions. They are either model-based or model-free, and optimize discounted total reward or undiscounted average reward. Our experimental results indicate that H-learning is more robust with respect to changes in the domain parameters, and in many cases, converges in fewer steps to better average reward per time step than all the other methods. An added advantage is that unlike the other methods it does not have any parameters to tune. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #6163. 	[Title] Generalization in Reinforcement Learning: Safely Approximating the Value Function [Abstract] To appear in: G. Tesauro, D. S. Touretzky and T. K. Leen, eds., Advances in Neural Information Processing Systems 7, MIT Press, Cambridge MA, 1995. A straightforward approach to the curse of dimensionality in reinforcement learning and dynamic programming is to replace the lookup table with a generalizing function approximator such as a neural net. Although this has been successful in the domain of backgammon, there is no guarantee of convergence. In this paper, we show that the combination of dynamic programming and function approximation is not robust, and in even very benign cases, may produce an entirely wrong policy. We then introduce Grow-Support, a new algorithm which is safe from divergence yet can still reap the benefits of successful generalization. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #1135125. 	[Title] Constructing Fuzzy Graphs from Examples [Abstract] Methods to build function approximators from example data have gained considerable interest in the past. Especially methodologies that build models that allow an interpretation have attracted attention. Most existing algorithms, however, are either complicated to use or infeasible for high-dimensional problems. This article presents an efficient and easy to use algorithm to construct fuzzy graphs from example data. The resulting fuzzy graphs are based on locally independent fuzzy rules that operate solely on selected, important attributes. This enables the application of these fuzzy graphs also to problems in high dimensional spaces. Using illustrative examples and a real world data set it is demonstrated how the resulting fuzzy graphs offer quick insights into the structure of the example data, that is, the underlying model. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #552469. 	[Title] On Learning More Concepts [Abstract] The coverage of a learning algorithm is the number of concepts that can be learned by that algorithm from samples of a given size. This paper asks whether good learning algorithms can be designed by maximizing their coverage. The paper extends a previous upper bound on the coverage of any Boolean concept learning algorithm and describes two algorithms|Multi-Balls and Large-Ball|whose coverage approaches this upper bound. Experimental measurement of the coverage of the ID3 and FRINGE algorithms shows that their coverage is far below this bound. Further analysis of Large-Ball shows that although it learns many concepts, these do not seem to be very interesting concepts. Hence, coverage maximization alone does not appear to yield practically-useful learning algorithms. The paper concludes with a definition of coverage within a bias, which suggests a way that coverage maximization could be applied to strengthen weak preference biases. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #6169. 	[Title] In  Improving Elevator Performance Using Reinforcement Learning [Abstract] This paper describes the application of reinforcement learning (RL) to the difficult real world problem of elevator dispatching. The elevator domain poses a combination of challenges not seen in most RL research to date. Elevator systems operate in continuous state spaces and in continuous time as discrete event dynamic systems. Their states are not fully observable and they are nonstationary due to changing passenger arrival rates. In addition, we use a team of RL agents, each of which is responsible for controlling one elevator car. The team receives a global reinforcement signal which appears noisy to each agent due to the effects of the actions of the other agents, the random nature of the arrivals and the incomplete observation of the state. In spite of these complications, we show results that in simulation surpass the best of the heuristic elevator control algorithms of which we are aware. These results demonstrate the power of RL on a very large scale stochastic dynamic optimization problem of practical utility. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #1123867. 	[Title] A Study of Maximal-Coverage Learning Algorithms [Abstract] The coverage of a learning algorithm is the number of concepts that can be learned by that algorithm from samples of a given size. This paper asks whether good learning algorithms can be designed by maximizing their coverage. The paper extends a previous upper bound on the coverage of any Boolean concept learning algorithm and describes two algorithms|Multi-Balls and Large-Ball|whose coverage approaches this upper bound. Experimental measurement of the coverage of the ID3 and FRINGE algorithms shows that their coverage is far below this bound. Further analysis of Large-Ball shows that although it learns many concepts, these do not seem to be very interesting concepts. Hence, coverage maximization alone does not appear to yield practically-useful learning algorithms. The paper concludes with a definition of coverage within a bias, which suggests a way that coverage maximization could be applied to strengthen weak preference biases. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #45599. 	[Title] Grounding Robotic Control with Genetic Neural Networks [Abstract] Technical Report AI94-223 May 1994 Abstract An important but often neglected problem in the field of Artificial Intelligence is that of grounding systems in their environment such that the representations they manipulate have inherent meaning for the system. Since humans rely so heavily on semantics, it seems likely that the grounding is crucial to the development of truly intelligent behavior. This study investigates the use of simulated robotic agents with neural network processors as part of a method to ensure grounding. Both the topology and weights of the neural networks are optimized through genetic algorithms. Although such comprehensive optimization is difficult, the empirical evidence gathered here shows that the method is not only tractable but quite fruitful. In the experiments, the agents evolved a wall-following control strategy and were able to transfer it to novel environments. Their behavior suggests that they were also learning to build cognitive maps. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #35. 	[Title] 4 Implementing Application Specific Routines  Genetic algorithms in search, optimization, and machine learning. Reading, MA: Addison-Wesley. [Abstract] To implement a specific application, you should only have to change the file app.c. Section 2 describes the routines in app.c in detail. If you use additional variables for your specific problem, the easiest method of making them available to other program units is to declare them in sga.h and external.h. However, take care that you do not redeclare existing variables. Two example applications files are included in the SGA-C distribution. The file app1.c performs the simple example problem included with the Pascal version; finding the maximum of x 10 , where x is an integer interpretation of a chromosome. A slightly more complex application is include in app2.c. This application illustrates two features that have been added to SGA-C. The first of these is the ithruj2int function, which converts bits i through j in a chromosome to an integer. The second new feature is the utility pointer that is associated with each population member. The example application interprets each chromosome as a set of concatenated integers in binary form. The lengths of these integer fields is determined by the user-specified value of field size, which is read in by the function app data(). The field size must be less than the smallest of the chromosome length and the length of an unsigned integer. An integer array for storing the interpreted form of each chromosome is dynamically allocated and assigned to the chromosome's utility pointer in app malloc(). The ithruj2int routine (see utility.c) is used to translate each chromosome into its associated vector. The fitness for each chromosome is simply the sum of the squares of these integers. This example application will function for any chromosome length. SGA-C is intended to be a simple program for first-time GA experimentation. It is not intended to be definitive in terms of its efficiency or the grace of its implementation. The authors are interested in the comments, criticisms, and bug reports from SGA-C users, so that the code can be refined for easier use in subsequent versions. Please email your comments to rob@galab2.mh.ua.edu, or write to TCGA: The authors gratefully acknowledge support provided by NASA under Grant NGT-50224 and support provided by the National Science Foundation under Grant CTS-8451610. We also thank Hillol Kargupta for donating his tournament selection implementation. Booker, L. B. (1982). Intelligent behavior as an adaptation to the task environment (Doctoral dissertation, Technical Report No. 243. Ann Arbor: University of Michigan, Logic of Computers Group). Dissertations Abstracts International, 43(2), 469B. (University Microfilms No. 8214966) Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #6184. 	[Title] Forward models: Supervised learning with a distal teacher [Abstract] Internal models of the environment have an important role to play in adaptive systems in general and are of particular importance for the supervised learning paradigm. In this paper we demonstrate that certain classical problems associated with the notion of the "teacher" in supervised learning can be solved by judicious use of learned internal models as components of the adaptive system. In particular, we show how supervised learning algorithms can be utilized in cases in which an unknown dynamical system intervenes between actions and desired outcomes. Our approach applies to any supervised learning algorithm that is capable of learning in multi-layer networks. *This paper is a revised version of MIT Center for Cognitive Science Occasional Paper #40. We wish to thank Michael Mozer, Andrew Barto, Robert Jacobs, Eric Loeb, and James McClelland for helpful comments on the manuscript. This project was supported in part by BRSG 2 S07 RR07047-23 awarded by the Biomedical Research Support Grant Program, Division of Research Resources, National Institutes of Health, by a grant from ATR Auditory and Visual Perception Research Laboratories, by a grant from Siemens Corporation, by a grant from the Human Frontier Science Program, and by grant N00014-90-J-1942 awarded by the Office of Naval Research. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #20526. 	[Title] Self-Improving Factory Simulation using Continuous-time Average-Reward Reinforcement Learning [Abstract] Many factory optimization problems, from inventory control to scheduling and reliability, can be formulated as continuous-time Markov decision processes. A primary goal in such problems is to find a gain-optimal policy that minimizes the long-run average cost. This paper describes a new average-reward algorithm called SMART for finding gain-optimal policies in continuous time semi-Markov decision processes. The paper presents a detailed experimental study of SMART on a large unreliable production inventory problem. SMART outperforms two well-known reliability heuristics from industrial engineering. A key feature of this study is the integration of the reinforcement learning algorithm directly into two commercial discrete-event simulation packages, ARENA and CSIM, paving the way for this approach to be applied to many other factory optimization problems for which there already exist simulation models. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #6196. 	[Title] The Parti-game Algorithm for Variable Resolution Reinforcement Learning in Multidimensional State-spaces [Abstract] Parti-game is a new algorithm for learning feasible trajectories to goal regions in high dimensional continuous state-spaces. In high dimensions it is essential that learning does not plan uniformly over a state-space. Parti-game maintains a decision-tree partitioning of state-space and applies techniques from game-theory and computational geometry to efficiently and adaptively concentrate high resolution only on critical areas. The current version of the algorithm is designed to find feasible paths or trajectories to goal regions in high dimensional spaces. Future versions will be designed to find a solution that optimizes a real-valued criterion. Many simulated problems have been tested, ranging from two-dimensional to nine-dimensional state-spaces, including mazes, path planning, non-linear dynamics, and planar snake robots in restricted spaces. In all cases, a good solution is found in less than ten trials and a few minutes. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #188471. 	[Title] Adaptive state space quantisation for reinforcement learning of collision-free navigation [Abstract] The paper describes a self-learning control system for a mobile robot. Based on sensor information the control system has to provide a steering signal in such a way that collisions are avoided. Since in our case no `examples' are available, the system learns on the basis of an external reinforcement signal which is negative in case of a collision and zero otherwise. Rules from Temporal Difference learning are used to find the correct mapping between the (discrete) sensor input space and the steering signal. We describe the algorithm for learning the correct mapping from the input (state) vector to the output (steering) signal, and the algorithm which is used for a discrete coding of the input state space. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #13885. 	[Title] Learning Controllers for Industrial Robots [Abstract] One of the most significant cost factors in robotics applications is the design and development of real-time robot control software. Control theory helps when linear controllers have to be developed, but it doesn't sufficiently support the generation of non-linear controllers, although in many cases (such as in compliance control), nonlinear control is essential for achieving high performance. This paper discusses how Machine Learning has been applied to the design of (non-)linear controllers. Several alternative function approximators, including Multilayer Perceptrons (MLP), Radial Basis Function Networks (RBFNs), and Fuzzy Controllers are analyzed and compared, leading to the definition of two major families: Open Field Function Function Approximators and Locally Receptive Field Function Approximators. It is shown that RBFNs and Fuzzy Controllers bear strong similarities, and that both have a symbolic interpretation. This characteristics allows for applying both symbolic and statistic learning algorithms to synthesize the network layout from a set of examples and, possibly, some background knowledge. Three integrated learning algorithms, two of which are original, are described and evaluated on experimental test cases. The first test case is provided by a robot KUKA IR-361 engaged into the "peg-into-hole" task, whereas the second is represented by a classical prediction task on the Mackey-Glass time series. From the experimental comparison, it appears that both Fuzzy Controllers and RBFNs synthesised from examples are excellent approximators, and that, in practice, they can be even more accurate than MLPs. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #6209. 	[Title] Transfer of Learning by Composing Solutions of Elemental Sequential Tasks [Abstract] Although building sophisticated learning agents that operate in complex environments will require learning to perform multiple tasks, most applications of reinforcement learning have focussed on single tasks. In this paper I consider a class of sequential decision tasks (SDTs), called composite sequential decision tasks, formed by temporally concatenating a number of elemental sequential decision tasks. Elemental SDTs cannot be decomposed into simpler SDTs. I consider a learning agent that has to learn to solve a set of elemental and composite SDTs. I assume that the structure of the composite tasks is unknown to the learning agent. The straightforward application of reinforcement learning to multiple tasks requires learning the tasks separately, which can waste computational resources, both memory and time. I present a new learning algorithm and a modular architecture that learns the decomposition of composite SDTs, and achieves transfer of learning by sharing the solutions of elemental SDTs across multiple composite SDTs. The solution of a composite SDT is constructed by computationally inexpensive modifications of the solutions of its constituent elemental SDTs. I provide a proof of one aspect of the learning algorithm. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #6210. 	[Title] Reinforcement Learning with Soft State Aggregation [Abstract] It is widely accepted that the use of more compact representations than lookup tables is crucial to scaling reinforcement learning (RL) algorithms to real-world problems. Unfortunately almost all of the theory of reinforcement learning assumes lookup table representations. In this paper we address the pressing issue of combining function approximation and RL, and present 1) a function approx-imator based on a simple extension to state aggregation (a commonly used form of compact representation), namely soft state aggregation, 2) a theory of convergence for RL with arbitrary, but fixed, soft state aggregation, 3) a novel intuitive understanding of the effect of state aggregation on online RL, and 4) a new heuristic adaptive state aggregation algorithm that finds improved compact representations by exploiting the non-discrete nature of soft state aggregation. Preliminary empirical results are also presented. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #6213. 	[Title] Machine Learning Learning to Predict by the Methods of Temporal Differences Keywords: Incremental learning, prediction, [Abstract] This article introduces a class of incremental learning procedures specialized for prediction|that is, for using past experience with an incompletely known system to predict its future behavior. Whereas conventional prediction-learning methods assign credit by means of the difference between predicted and actual outcomes, the new methods assign credit by means of the difference between temporally successive predictions. Although such temporal-difference methods have been used in Samuel's checker player, Holland's bucket brigade, and the author's Adaptive Heuristic Critic, they have remained poorly understood. Here we prove their convergence and optimality for special cases and relate them to supervised-learning methods. For most real-world prediction problems, temporal-difference methods require less memory and less peak computation than conventional methods; and they produce more accurate predictions. We argue that most problems to which supervised learning is currently applied are really prediction problems of the sort to which temporal-difference methods can be applied to advantage. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #6214. 	[Title] Integrated Architectures for Learning, Planning, and Reacting Based on Approximating Dynamic Programming [Abstract] This paper extends previous work with Dyna, a class of architectures for intelligent systems based on approximating dynamic programming methods. Dyna architectures integrate trial-and-error (reinforcement) learning and execution-time planning into a single process operating alternately on the world and on a learned model of the world. In this paper, I present and show results for two Dyna architectures. The Dyna-PI architecture is based on dynamic programming's policy iteration method and can be related to existing AI ideas such as evaluation functions and universal plans (reactive systems). Using a navigation task, results are shown for a simple Dyna-PI system that simultaneously learns by trial and error, learns a world model, and plans optimal routes using the evolving world model. The Dyna-Q architecture is based on Watkins's Q-learning, a new kind of reinforcement learning. Dyna-Q uses a less familiar set of data structures than does Dyna-PI, but is arguably simpler to implement and use. We show that Dyna-Q architectures are easy to adapt for use in changing environments. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #1130567. 	[Title] NEUROCONTROL BY REINFORCEMENT LEARNING [Abstract] Reinforcement learning (RL) is a model-free tuning and adaptation method for control of dynamic systems. Contrary to supervised learning, based usually on gradient descent techniques, RL does not require any model or sensitivity function of the process. Hence, RL can be applied to systems that are poorly understood, uncertain, nonlinear or for other reasons untractable with conventional methods. In reinforcement learning, the overall controller performance is evaluated by a scalar measure, called reinforcement. Depending on the type of the control task, reinforcement may represent an evaluation of the most recent control action or, more often, of an entire sequence of past control moves. In the latter case, the RL system learns how to predict the outcome of each individual control action. This prediction is then used to adjust the parameters of the controller. The mathematical background of RL is closely related to optimal control and dynamic programming. This paper gives a comprehensive overview of the RL methods and presents an application to the attitude control of a satellite. Some well known applications from the literature are reviewed as well. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #6216. 	[Title] Generalization in Reinforcement Learning: Successful Examples Using Sparse Coarse Coding [Abstract] On large problems, reinforcement learning systems must use parameterized function approximators such as neural networks in order to generalize between similar situations and actions. In these cases there are no strong theoretical results on the accuracy of convergence, and computational results have been mixed. In particular, Boyan and Moore reported at last year's meeting a series of negative results in attempting to apply dynamic programming together with function approximation to simple control problems with continuous state spaces. In this paper, we present positive results for all the control tasks they attempted, and for one that is significantly larger. The most important differences are that we used sparse-coarse-coded function approximators (CMACs) whereas they used mostly global function approximators, and that we learned online whereas they learned o*ine. Boyan and Moore and others have suggested that the problems they encountered could be solved by using actual outcomes ("rollouts"), as in classical Monte Carlo methods, and as in the TD() algorithm when = 1. However, in our experiments this always resulted in substantially poorer performance. We conclude that reinforcement learning can work robustly in conjunction with function approximators, and that there is little justification at present for avoiding the case of general . Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #6220. 	[Title] Issues in Using Function Approximation for Reinforcement Learning [Abstract] Reinforcement learning techniques address the problem of learning to select actions in unknown, dynamic environments. It is widely acknowledged that to be of use in complex domains, reinforcement learning techniques must be combined with generalizing function approximation methods such as artificial neural networks. Little, however, is understood about the theoretical properties of such combinations, and many researchers have encountered failures in practice. In this paper we identify a prime source of such failuresnamely, a systematic overestimation of utility values. Using Watkins' Q-Learning [18] as an example, we give a theoretical account of the phenomenon, deriving conditions under which one may expected it to cause learning to fail. Employing some of the most popular function approximators, we present experimental results which support the theoretical findings. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #628815. 	[Title] Optimal Attitude Control of Satellites by Artificial Neural Networks: a Pilot Study [Abstract] A pilot study is described on the practical application of artificial neural networks. The limit cycle of the attitude control of a satellite is selected as the test case. One of the sources of the limit cycle is a position dependent error in the observed attitude. A Reinforcement Learning method is selected, which is able to adapt a controller such that a cost function is optimised. An estimate of the cost function is learned by a neural `critic'. In our approach, the estimated cost function is directly represented as a function of the parameters of a linear controller. The critic is implemented as a CMAC network. Results from simulations show that the method is able to find optimal parameters without unstable behaviour. In particular in the case of large discontinuities in the attitude measurements, the method shows a clear improvement compared to the conventional approach: the RMS attitude error decreases approximately 30%. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #6224. 	[Title] Tight Performance Bounds on Greedy Policies Based on Imperfect Value Functions [Abstract] Consider a given value function on states of a Markov decision problem, as might result from applying a reinforcement learning algorithm. Unless this value function equals the corresponding optimal value function, at some states there will be a discrepancy, which is natural to call the Bellman residual, between what the value function specifies at that state and what is obtained by a one-step lookahead along the seemingly best action at that state using the given value function to evaluate all succeeding states. This paper derives a tight bound on how far from optimal the discounted return for a greedy policy based on the given value function will be as a function of the maximum norm magnitude of this Bellman residual. A corresponding result is also obtained for value functions defined on state-action pairs, as are used in Q-learning. One significant application of these results is to problems where a function approximator is used to learn a value function, with training of the approxi-mator based on trying to minimize the Bellman residual across states or state-action pairs. When control is based on the use of the resulting value function, this result provides a link between how well the objectives of function approximator training are met and the quality of the resulting control. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #28249. 	[Title] Truncating Temporal Differences: On the Efficient Implementation of TD() for Reinforcement Learning [Abstract] Temporal difference (TD) methods constitute a class of methods for learning predictions in multi-step prediction problems, parameterized by a recency factor . Currently the most important application of these methods is to temporal credit assignment in reinforcement learning. Well known reinforcement learning algorithms, such as AHC or Q-learning, may be viewed as instances of TD learning. This paper examines the issues of the efficient and general implementation of TD() for arbitrary , for use with reinforcement learning algorithms optimizing the discounted sum of rewards. The traditional approach, based on eligibility traces, is argued to suffer from both inefficiency and lack of generality. The TTD (Truncated Temporal Differences) procedure is proposed as an alternative, that indeed only approximates TD(), but requires very little computation per action and can be used with arbitrary function representation methods. The idea from which it is derived is fairly simple and not new, but probably unexplored so far. Encouraging experimental results are presented, suggesting that using &gt; 0 with the TTD procedure allows one to obtain a significant learning speedup at essentially the same cost as usual TD(0) learning. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #13917. 	[Title] Designing Neural Networks for Adaptive Control [Abstract] This paper discusses the design of neural networks to solve specific problems of adaptive control. In particular, it investigates the influence of typical problems arising in real-world control tasks as well as techniques for their solution that exist in the framework of neurocontrol. Based on this investigation, a systematic design method is developed. The method is exemplified for the development of an adaptive force controller for a robot manipulator. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #27230. 	[Title] K unstliche Intelligenz Grdt: Enhancing Model-Based Learning for Its Application in Robot Navigation [Abstract]  Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #52835. 	[Title] On Biases in Estimating Multi-Valued Attributes [Abstract] We analyse the biases of eleven measures for estimating the quality of the multi-valued attributes. The values of information gain, J-measure, gini-index, and relevance tend to linearly increase with the number of values of an attribute. The values of gain-ratio, distance measure, Relief , and the weight of evidence decrease for informative attributes and increase for irrelevant attributes. The bias of the statistic tests based on the chi-square distribution is similar but these functions are not able to discriminate among the attributes of different quality. We also introduce a new function based on the MDL principle whose value slightly decreases with the increasing number of attribute's values. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #97892. 	[Title] Correcting Imperfect Domain Theories: A Knowledge-Level Analysis [Abstract] Explanation-Based Learning [Mitchell et al., 1986; DeJong and Mooney, 1986] has shown promise as a powerful analytical learning technique. However, EBL is severely hampered by the requirement of a complete and correct domain theory for successful learning to occur. Clearly, in non-trivial domains, developing such a domain theory is a nearly impossible task. Therefore, much research has been devoted to understanding how an imperfect domain theory can be corrected and extended during system performance. In this paper, we present a characterization of this problem, and use it to analyze past research in the area. Past characterizations of the problem (e.g, [Mitchell et al., 1986; Rajamoney and DeJong, 1987]) have viewed the types of performance errors caused by a faulty domain theory as primary. In contrast, we focus primarily on the types of knowledge deficiencies present in the theory, and from these derive the types of performance errors that can result. Correcting the theory can be viewed as a search through the space of possible domain theories, with a variety of knowledge sources that can be used to guide the search. We examine the types of knowledge used by a variety of past systems for this purpose. The hope is that this analysis will indicate the need for a "universal weak method" of domain theory correction, in which different sources of knowledge for theory correction can be freely and flexibly combined. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #1130600. 	[Title] Adaptive state space quantisation: adding and removing neurons [Abstract] This paper describes a self-learning control system for a mobile robot. Based on local sensor data, a robot is taught to avoid collisions with obstacles. The only feedback to the control system is a binary-valued external reinforcement signal, which indicates whether or not a collision has occured. A reinforcement learning scheme is used to find a correct mapping from input (sensor) space to output (steering signal) space. An adaptive quantisation scheme is introduced, through which the discrete division of input space is built up from scratch by the system itself. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #157805. 	[Title] Learning from Queries and Examples with Tree-structured Bias [Abstract] Incorporating declarative bias or prior knowledge into learning is an active research topic in machine learning. Tree-structured bias specifies the prior knowledge as a tree of "relevance" relationships between attributes. This paper presents a learning algorithm that implements tree-structured bias, i.e., learns any target function probably approximately correctly from random examples and membership queries if it obeys a given tree-structured bias. The theoretical predictions of the paper are em pirically validated. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #20593. 	[Title] Generating Neural Networks Through the Induction of Threshold Logic Unit Trees (Extended Abstract) [Abstract] We investigate the generation of neural networks through the induction of binary trees of threshold logic units (TLUs). Initially, we describe the framework for our tree construction algorithm and how such trees can be transformed into an isomorphic neural network topology. Several methods for learning the linear discriminant functions at each node of the tree structure are examined and shown to produce accuracy results that are comparable to classical information theoretic methods for constructing decision trees (which use single feature tests at each node). Our TLU trees, however, are smaller and thus easier to understand. Moreover, we show that it is possible to simultaneously learn both the topology and weight settings of a neural network simply using the training data set that we are given. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #114. 	[Title] Learning to Act using Real-Time Dynamic Programming [Abstract] fl The authors thank Rich Yee, Vijay Gullapalli, Brian Pinette, and Jonathan Bachrach for helping to clarify the relationships between heuristic search and control. We thank Rich Sutton, Chris Watkins, Paul Werbos, and Ron Williams for sharing their fundamental insights into this subject through numerous discussions, and we further thank Rich Sutton for first making us aware of Korf's research and for his very thoughtful comments on the manuscript. We are very grateful to Dimitri Bertsekas and Steven Sullivan for independently pointing out an error in an earlier version of this article. Finally, we thank Harry Klopf, whose insight and persistence encouraged our interest in this class of learning problems. This research was supported by grants to A.G. Barto from the National Science Foundation (ECS-8912623 and ECS-9214866) and the Air Force Office of Scientific Research, Bolling AFB (AFOSR-89-0526). Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #65653. 	[Title] Pessimistic decision tree pruning based on tree size [Abstract] In this work we develop a new criteria to perform pessimistic decision tree pruning. Our method is theoretically sound and is based on theoretical concepts such as uniform convergence and the Vapnik-Chervonenkis dimension. We show that our criteria is very well motivated, from the theory side, and performs very well in practice. The accuracy of the new criteria is comparable to that of the current method used in C4.5. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #28278. 	[Title] Chapter 1 Reinforcement Learning for Planning and Control [Abstract]  Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #48766. 	[Title] Discovering Complex Othello Strategies Through Evolutionary Neural Networks [Abstract] An approach to develop new game playing strategies based on artificial evolution of neural networks is presented. Evolution was directed to discover strategies in Othello against a random-moving opponent and later against an ff-fi search program. The networks discovered first a standard positional strategy, and subsequently a mobility strategy, an advanced strategy rarely seen outside of tournaments. The latter discovery demonstrates how evolutionary neural networks can develop novel solutions by turning an initial disadvantage into an advantage in a changed environment. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #28287. 	[Title] Robot Shaping: Developing Situated Agents through Learning [Abstract] Learning plays a vital role in the development of situated agents. In this paper, we explore the use of reinforcement learning to "shape" a robot to perform a predefined target behavior. We connect both simulated and real robots to A LECSYS, a parallel implementation of a learning classifier system with an extended genetic algorithm. After classifying different kinds of Animat-like behaviors, we explore the effects on learning of different types of agent's architecture (monolithic, flat and hierarchical) and of training strategies. In particular, hierarchical architecture requires the agent to learn how to coordinate basic learned responses. We show that the best results are achieved when both the agent's architecture and the training strategy match the structure of the behavior pattern to be learned. We report the results of a number of experiments carried out both in simulated and in real environments, and show that the results of simulations carry smoothly to real robots. While most of our experiments deal with simple reactive behavior, in one of them we demonstrate the use of a simple and general memory mechanism. As a whole, our experimental activity demonstrates that classifier systems with genetic algorithms can be practically employed to develop autonomous agents. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #128. 	[Title] Reinforcement Learning Algorithms for Average-Payoff Markovian Decision Processes [Abstract] Reinforcement learning (RL) has become a central paradigm for solving learning-control problems in robotics and artificial intelligence. RL researchers have focussed almost exclusively on problems where the controller has to maximize the discounted sum of payoffs. However, as emphasized by Schwartz (1993), in many problems, e.g., those for which the optimal behavior is a limit cycle, it is more natural and com-putationally advantageous to formulate tasks so that the controller's objective is to maximize the average payoff received per time step. In this paper I derive new average-payoff RL algorithms as stochastic approximation methods for solving the system of equations associated with the policy evaluation and optimal control questions in average-payoff RL tasks. These algorithms are analogous to the popular TD and Q-learning algorithms already developed for the discounted-payoff case. One of the algorithms derived here is a significant variation of Schwartz's R-learning algorithm. Preliminary empirical results are presented to validate these new algorithms. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #1113739. 	[Title] Decision Tree Function Approximation in Reinforcement Learning [Abstract] We present a decision tree based approach to function approximation in reinforcement learning. We compare our approach with table lookup and a neural network function approximator on three problems: the well known mountain car and pole balance problems as well as a simulated automobile race car. We find that the decision tree can provide better learning performance than the neural network function approximation and can solve large problems that are infeasible using table lookup. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #782486. 	[Title] Living in a partially structured environment: How to bypass the limitations of classical reinforcement techniques [Abstract] In this paper, we propose an unsupervised neural network allowing a robot to learn sensori-motor associations with a delayed reward. The robot task is to learn the "meaning" of pictograms in order to "survive" in a maze. First, we introduce a new neural conditioning rule (PCR: Probabilistic Conditioning Rule) allowing to test hypotheses (associations between visual categories and movements) during a given time span. Second, we describe a real maze experiment with our mobile robot. We propose a neural architecture to solve this problem and we discuss the difficulty to build visual categories dynamically while associating them to movements. Third, we propose to use our algorithm on a simulation in order to test it exhaustively. We give the results for different kind of mazes and we compare our system to an adapted version of the Q-learning algorithm. Finally, we conclude by showing the limitations of approaches that do not take into account the intrinsic complexity of a reasonning based on image recognition. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #3229. 	[Title] Book Review  Introduction to the Theory of Neural Computation Reviewed by: 2 [Abstract] Neural computation, also called connectionism, parallel distributed processing, neural network modeling or brain-style computation, has grown rapidly in the last decade. Despite this explosion, and ultimately because of impressive applications, there has been a dire need for a concise introduction from a theoretical perspective, analyzing the strengths and weaknesses of connectionist approaches and establishing links to other disciplines, such as statistics or control theory. The Introduction to the Theory of Neural Computation by Hertz, Krogh and Palmer (subsequently referred to as HKP) is written from the perspective of physics, the home discipline of the authors. The book fulfills its mission as an introduction for neural network novices, provided that they have some background in calculus, linear algebra, and statistics. It covers a number of models that are often viewed as disjoint. Critical analyses and fruitful comparisons between these models Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #124064. 	[Title] An Upper Bound on the Loss from Approximate Optimal-Value Functions [Abstract] Many reinforcement learning (RL) approaches can be formulated from the theory of Markov decision processes and the associated method of dynamic programming (DP). The value of this theoretical understanding, however, is tempered by many practical concerns. One important question is whether DP-based approaches that use function approximation rather than lookup tables, can avoid catastrophic effects on performance. This note presents a result in Bertsekas (1987) which guarantees that small errors in the approximation of a task's optimal value function cannot produce arbitrarily bad performance when actions are selected greedily. We derive an upper bound on performance loss which is slightly tighter than that in Bertsekas (1987), and we show the extension of the bound to Q-learning (Watkins, 1989). These results provide a theoretical justification for a practice that is common in reinforcement learning. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #591016. 	[Title] First Order Regression: Applications in Real-World Domains [Abstract] A first order regression algorithm capable of handling real-valued (continuous) variables is introduced and some of its applications are presented. Regressional learning assumes real-valued class and discrete or real-valued variables. The algorithm combines regressional learning with standard ILP concepts, such as first order concept description and background knowledge. A clause is generated by successively refining the initial clause by adding literals of the form A = v for the discrete attributes, A v and A v for the real-valued attributes, and background knowledge literals to the clause body. The algorithm employs a covering approach (beam search), a heuristic impurity function, and stopping criteria based on local improvement, minimum number of examples, maximum clause length, minimum local improvement, minimum description length, allowed error, and variable depth. An outline of the algorithm and the results of the system's application in some artificial and real-world domains are presented. The real-world domains comprise: modelling of the water behavior in a surge tank, modelling of the workpiece roughness in a steel grinding process and modelling of the operator's behavior during the process of electrical discharge machining. Special emphasis is given to the evaluation of obtained models by domain experts and their comments on the aspects of practical use of the induced knowledge. The results obtained during the knowledge acquisition process show several important guidelines for knowledge acquisition, concerning mainly the process of interaction with domain experts, exposing primarily the importance of comprehensibility of the induced knowledge. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #3243. 	[Title] A System for Induction of Oblique Decision Trees [Abstract] This article describes a new system for induction of oblique decision trees. This system, OC1, combines deterministic hill-climbing with two forms of randomization to find a good oblique split (in the form of a hyperplane) at each node of a decision tree. Oblique decision tree methods are tuned especially for domains in which the attributes are numeric, although they can be adapted to symbolic or mixed symbolic/numeric attributes. We present extensive empirical studies, using both real and artificial data, that analyze OC1's ability to construct oblique trees that are smaller and more accurate than their axis-parallel counterparts. We also examine the benefits of randomization for the construction of oblique decision trees. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #30895. 	[Title] Induction of Oblique Decision Trees [Abstract] This paper introduces a randomized technique for partitioning examples using oblique hyperplanes. Standard decision tree techniques, such as ID3 and its descendants, partition a set of points with axis-parallel hyper-planes. Our method, by contrast, attempts to find hyperplanes at any orientation. The purpose of this more general technique is to find smaller but equally accurate decision trees than those created by other methods. We have tested our algorithm on both real and simulated data, and found that in some cases it produces surprisingly small trees without losing predictive accuracy. Small trees allow us, in turn, to obtain simple qualitative descriptions of each problem domain. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #30901. 	[Title] Lookahead and Pathology in Decision Tree Induction [Abstract] The standard approach to decision tree induction is a top-down, greedy algorithm that makes locally optimal, irrevocable decisions at each node of a tree. In this paper, we empirically study an alternative approach, in which the algorithms use one-level lookahead to decide what test to use at a node. We systematically compare, using a very large number of real and artificial data sets, the quality of decision trees induced by the greedy approach to that of trees induced using lookahead. The main observations from our experiments are: (i) the greedy approach consistently produced trees that were just as accurate as trees produced with the much more expensive lookahead step; and (ii) we observed many instances of pathology, i.e., lookahead produced trees that were both larger and less accurate than trees produced without it. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #18615. 	[Title] Learning Trees and Rules with Set-valued Features [Abstract] In most learning systems examples are represented as fixed-length "feature vectors", the components of which are either real numbers or nominal values. We propose an extension of the feature-vector representation that allows the value of a feature to be a set of strings; for instance, to represent a small white and black dog with the nominal features size and species and the set-valued feature color, one might use a feature vector with size=small, species=canis-familiaris and color=fwhite,blackg. Since we make no assumptions about the number of possible set elements, this extension of the traditional feature-vector representation is closely connected to Blum's "infinite attribute" representation. We argue that many decision tree and rule learning algorithms can be easily extended to set-valued features. We also show by example that many real-world learning problems can be efficiently and naturally represented with set-valued features; in particular, text categorization problems and problems that arise in propositionalizing first-order representations lend themselves to set-valued features. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #85688. 	[Title] Finding new rules for incomplete theories: Explicit biases for induction with contextual information. In Proceedings [Abstract] addressed in KBANN (which translates a theory into a neural-net, refines it using backpropagation, and then retranslates the result back into rules) by adding extra hidden units and connections to the initial network; however, this would require predetermining the num In this paper, we have presented constructive induction techniques recently added to the EITHER theory refinement system. Intermediate concept utilization employs existing rules in the theory to derive higher-level features for use in induction. Intermediate concept creation employs inverse resolution to introduce new intermediate concepts in order to fill gaps in a theory than span multiple levels. These revisions allow EITHER to make use of imperfect domain theories in the ways typical of previous work in both constructive induction and theory refinement. As a result, EITHER is able to handle a wider range of theory imperfections than any other existing theory refinement system. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #1116347. 	[Title] Fast Online Q() [Abstract] Q()-learning uses TD()-methods to accelerate Q-learning. The update complexity of previous online Q() implementations based on lookup-tables is bounded by the size of the state/action space. Our faster algorithm's update complexity is bounded by the number of actions. The method is based on the observation that Q-value updates may be postponed until they are needed. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #1111230. 	[Title] Reinforcement Learning, Neural Networks and PI Control Applied to a Heating Coil [Abstract] An accurate simulation of a heating coil is used to compare the performance of a PI controller, a neural network trained to predict the steady-state output of the PI controller, a neural network trained to minimize the n-step ahead error between the coil output and the set point, and a reinforcement learning agent trained to minimize the sum of the squared error over time. Although the PI controller works very well for this task, the neural networks do result in improved performance. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #1120962. 	[Title] Experiments on the Transfer of Knowledge between Neural Networks Reprinted from: Computational Learning Theory and [Abstract] This chapter describes three studies which address the question of how neural network learning can be improved via the incorporation of information extracted from other networks. This general problem, which we call network transfer, encompasses many types of relationships between source and target networks. Our focus is on the utilization of weights from source networks which solve a subproblem of the target network task, with the goal of speeding up learning on the target task. We demonstrate how the approach described here can improve learning speed by up to ten times over learning starting with random weights. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #636098. 	[Title] NP-Completeness of Searches for Smallest Possible Feature Sets a subset of the set of all [Abstract] In many learning problems, the learning system is presented with values for features that are actually irrelevant to the concept it is trying to learn. The FOCUS algorithm, due to Almuallim and Dietterich, performs an explicit search for the smallest possible input feature set S that permits a consistent mapping from the features in S to the output feature. The FOCUS algorithm can also be seen as an algorithm for learning determinations or functional dependencies, as suggested in [6]. Another algorithm for learning determinations appears in [7]. The FOCUS algorithm has superpolynomial runtime, but Almuallim and Di-etterich leave open the question of tractability of the underlying problem. In this paper, the problem is shown to be NP-complete. We also describe briefly some experiments that demonstrate the benefits of determination learning, and show that finding lowest-cardinality determinations is easier in practice than finding minimal determi Define the MIN-FEATURES problem as follows: given a set X of examples (which are each composed of a a binary value specifying the value of the target feature and a vector of binary values specifying the values of the other features) and a number n, determine whether or not there exists some feature set S such that: We show that MIN-FEATURES is NP-complete by reducing VERTEX-COVER to MIN-FEATURES. 1 The VERTEX-COVER problem may be stated as the question: given a graph G with vertices V and edges E, is there a subset V 0 of V , of size m, such that each edge in E is connected to at least one vertex in V 0 ? We may reduce an instance of VERTEX-COVER to an instance of MIN-FEATURES by mapping each edge in E to an example in X, with one input feature for every vertex in V . 1 In [8], a "proof" is reported for this result by reduction to set covering. The proof therefore fails to show NP-completeness. nations. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #132806. 	[Title] USING MARKER-BASED GENETIC ENCODING OF NEURAL NETWORKS TO EVOLVE FINITE-STATE BEHAVIOUR [Abstract] A new mechanism for genetic encoding of neural networks is proposed, which is loosely based on the marker structure of biological DNA. The mechanism allows all aspects of the network structure, including the number of nodes and their connectivity, to be evolved through genetic algorithms. The effectiveness of the encoding scheme is demonstrated in an object recognition task that requires artificial creatures (whose behaviour is driven by a neural network) to develop high-level finite-state exploration and discrimination strategies. The task requires solving the sensory-motor grounding problem, i.e. developing a functional understanding of the effects that a creature's movement has on its sensory input. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #1108167. 	[Title] On Learning Read-k-Satisfy-j DNF [Abstract] We study the learnability of Read-k-Satisfy-j (RkSj) DNF formulas. These are boolean formulas in disjunctive normal form (DNF), in which the maximum number of occurrences of a variable is bounded by k, and the number of terms satisfied by any assignment is at most j. After motivating the investigation of this class of DNF formulas, we present an algorithm that with high probability finds a DNF formula that is logically equivalent to any unknown RkSj DNF formula to be learned. The algorithm uses the well-studied protocol of equivalence and membership queries, and runs in polynomial time for k j = O( log n log log n ), where n is the number of input variables. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #193742. 	[Title] Q-Learning with Hidden-Unit Restarting [Abstract] Platt's resource-allocation network (RAN) (Platt, 1991a, 1991b) is modified for a reinforcement-learning paradigm and to "restart" existing hidden units rather than adding new units. After restarting, units continue to learn via back-propagation. The resulting restart algorithm is tested in a Q-learning network that learns to solve an inverted pendulum problem. Solutions are found faster on average with the restart algorithm than without it. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #1102550. 	[Title] Culling Teaching -1 Culling and Teaching in Neuro-evolution [Abstract] The evolving population of neural nets contains information not only in terms of genes, but also in the collection of behaviors of the population members. Such information can be thought of as a kind of culture of the population. Two ways of exploiting that culture are explored in this paper: (1) Culling overlarge litters: Generate a large number of offspring with different crossovers, quickly evaluate them by comparing their performance to the population, and throw away those that appear poor. (2) Teaching: Use backpropagation to train offspring toward the performance of the population. Both techniques result in faster, more effective neuro-evolution, and they can be effectively combined, as is demonstrated on the inverted pendulum problem. Additional methods of cultural exploitation are possible and will be studied in future work. These results suggest that cultural exploitation is a powerful idea that allows leveraging several aspects of the genetic algorithm. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #23258. 	[Title] Exploration and Model Building in Mobile Robot Domains [Abstract] I present first results on COLUMBUS, an autonomous mobile robot. COLUMBUS operates in initially unknown, structured environments. Its task is to explore and model the environment efficiently while avoiding collisions with obstacles. COLUMBUS uses an instance-based learning technique for modeling its environment. Real-world experiences are generalized via two artificial neural networks that encode the characteristics of the robot's sensors, as well as the characteristics of typical environments the robot is assumed to face. Once trained, these networks allow for knowledge transfer across different environments the robot will face over its lifetime. COLUMBUS' models represent both the expected reward and the confidence in these expectations. Exploration is achieved by navigating to low confidence regions. An efficient dynamic programming method is employed in background to find minimal-cost paths that, executed by the robot, maximize exploration. COLUMBUS operates in real-time. It has been operating successfully in an office building environment for periods up to hours. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #192734. 	[Title] Preventing "Overfitting" of Cross-Validation Data [Abstract] Suppose that, for a learning task, we have to select one hypothesis out of a set of hypotheses (that may, for example, have been generated by multiple applications of a randomized learning algorithm). A common approach is to evaluate each hypothesis in the set on some previously unseen cross-validation data, and then to select the hypothesis that had the lowest cross-validation error. But when the cross-validation data is partially corrupted such as by noise, and if the set of hypotheses we are selecting from is large, then "folklore" also warns about "overfitting" the cross-validation data [Klockars and Sax, 1986, Tukey, 1949, Tukey, 1953]. In this paper, we explain how this "overfitting" really occurs, and show the surprising result that it can be overcome by selecting a hypothesis with a higher cross-validation error, over others with lower cross-validation errors. We give reasons for not selecting the hypothesis with the lowest cross-validation error, and propose a new algorithm, LOOCVCV, that uses a computa-tionally efficient form of leave-one-out cross-validation to select such a hypothesis. Finally, we present experimental results for one domain, that show LOOCVCV consistently beating picking the hypothesis with the lowest cross-validation error, even when using reasonably large cross-validation sets. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #93923. 	[Title] Inductive Constraint Logic and the Mutagenesis Problem [Abstract] A novel approach to learning first order logic formulae from positive and negative examples is incorporated in a system named ICL (Inductive Constraint Logic). In ICL, examples are viewed as interpretations which are true or false for the target theory, whereas in present inductive logic programming systems, examples are true and false ground facts (or clauses). Furthermore, ICL uses a clausal representation, which corresponds to a conjunctive normal form where each conjunct forms a constraint on positive examples, whereas classical learning techniques have concentrated on concept representations in disjunctive normal form. We present some experiments with this new system on the mutagenesis problem. These experiments illustrate some of the differences with other systems, and indicate that our approach should work at least as well as the more classical approaches. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #4330. 	[Title] Hierarchical Mixtures of Experts and the EM Algorithm [Abstract] We present a tree-structured architecture for supervised learning. The statistical model underlying the architecture is a hierarchical mixture model in which both the mixture coefficients and the mixture components are generalized linear models (GLIM's). Learning is treated as a maximum likelihood problem; in particular, we present an Expectation-Maximization (EM) algorithm for adjusting the parameters of the architecture. We also develop an on-line learning algorithm in which the parameters are updated incrementally. Comparative simulation results are presented in the robot dynamics domain. This report describes research done at the Dept. of Brain and Cognitive Sciences, the Center for Biological and Computational Learning, and the Artificial Intelligence Laboratory of the Massachusetts Institute of Technology. Support for CBCL is provided in part by a grant from the NSF (ASC-9217041). Support for the laboratory's artificial intelligence research is provided in part by the Advanced Research Projects Agency of the Dept. of Defense. The authors were supported by a grant from the McDonnell-Pew Foundation, by a grant from ATR Human Information Processing Research Laboratories, by a grant from Siemens Corporation, by by grant IRI-9013991 from the National Science Foundation, by grant N00014-90-J-1942 from the Office of Naval Research, and by NSF grant ECS-9216531 to support an Initiative in Intelligent Control at MIT. Michael I. Jordan is a NSF Presidential Young Investigator. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #1272. 	[Title] Learning a set of primitive actions with an Induction of decision trees. Machine Learning, 1(1):81-106, [Abstract]  Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #1129208. 	[Title] Figure 3: Average model size accepted from a ran-dom prefix-closed samples of various size, and [Abstract] that is based on Angluin's L fl algorithm. The algorithm maintains a model consistent with its past examples. When a new counterexample arrives it tries to extend the model in a minimal fashion. We conducted a set of experiments where random automata that represent different strategies were generated, and the algorithm tried to learn them based on prefix-closed samples of their behavior. The algorithm managed to learn very compact models that agree with the samples. The size of the sample had a small effect on the size of the model. The experimental results suggest that for random prefix-closed samples the algorithm behaves well. However, following Angluin's result on the difficulty of learning almost uniform complete samples [ An-gluin, 1978 ] , it is obvious that our algorithm does not solve the complexity issue of inferring a DFA from a general prefix-closed sample. We are currently looking for classes of prefix-closed samples in which US-L* behaves well. [ Carmel and Markovitch, 1994 ] D. Carmel and S. Markovitch. The M* algorithm: Incorporating opponent models into adversary search. Technical Report CIS report 9402, Technion, March 1994. [ Carmel and Markovitch, 1995 ] D. Carmel and S. Markovitch. Unsupervised learning of finite automata: A practical approach. Technical Report CIS report 9504, Technion, March 1995. [ Shoham and Tennenholtz, 1994 ] Y. Shoham and M. Tennenholtz. Co-Learning and the evolution of social activity. Technical Report STAN-CS-TR-94-1511, Stanford Univrsity, Department of Computer Science, 1994. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #112378. 	[Title] Language-Independent Data-Oriented Grapheme-to-Phoneme Conversion [Abstract] We describe an approach to grapheme-to-phoneme conversion which is both language-independent and data-oriented. Given a set of examples (spelling words with their associated phonetic representation) in a language, a grapheme-to-phoneme conversion system is automatically produced for that language which takes as its input the spelling of words, and produces as its output the phonetic transcription according to the rules implicit in the training data. We describe the design of the system, and compare its performance to knowledge-based and alternative data-oriented approaches. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #1135358. 	[Title] 0 Inductive learning of compact rule sets by using efficient hypotheses reduction [Abstract] A method is described which reduces the hypotheses space with an efficient and easily interpretable reduction criteria called a - reduction. A learning algorithm is described based on a - reduction and analyzed by using probability approximate correct learning results. The results are obtained by reducing a rule set to an equivalent set of kDNF formulas. The goal of the learning algorithm is to induce a compact rule set describing the basic dependencies within a set of data. The reduction is based on criterion which is very exible and gives a semantic interpretation of the rules which fulfill the criteria. Comparison with syntactical hypotheses reduction show that the a reduction improves search and has a smaller probability of missclassification. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #284414. 	[Title] BRACE: A Paradigm For the Discretization of Continuously Valued Data [Abstract] Discretization of continuously valued data is a useful and necessary tool because many learning paradigms assume nominal data. A list of objectives for efficient and effective discretization is presented. A paradigm called BRACE (Boundary Ranking And Classification Evaluation) that attempts to meet the objectives is presented along with an algorithm that follows the paradigm. The paradigm meets many of the objectives, with potential for extension to meet the remainder. Empirical results have been promising. For these reasons BRACE has potential as an effective and efficient method for discretization of continuously valued data. A further advantage of BRACE is that it is general enough to be extended to other types of clustering/unsupervised learning. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #6917. 	[Title] Covering vs. Divide-and-Conquer for Top-Down Induction of Logic Programs [Abstract] covering has been formalized and used extensively. In this work, the divide-and-conquer technique is formalized as well and compared to the covering technique in a logic programming framework. Covering works by repeatedly specializing an overly general hypothesis, on each iteration focusing on finding a clause with a high coverage of positive examples. Divide-and-conquer works by specializing an overly general hypothesis once, focusing on discriminating positive from negative examples. Experimental results are presented demonstrating that there are cases when more accurate hypotheses can be found by divide-and-conquer than by covering. Moreover, since covering considers the same alternatives repeatedly it tends to be less efficient than divide-and-conquer, which never considers the same alternative twice. On the other hand, covering searches a larger hypothesis space, which may result in that more compact hypotheses are found by this technique than by divide-and-conquer. Furthermore, divide-and-conquer is, in contrast to covering, not applicable to learn ing recursive definitions. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #90888. 	[Title] GAL: Networks that grow when they learn and shrink when they forget [Abstract] Learning when limited to modification of some parameters has a limited scope; the capability to modify the system structure is also needed to get a wider range of the learnable. In the case of artificial neural networks, learning by iterative adjustment of synaptic weights can only succeed if the network designer predefines an appropriate network structure, i.e., number of hidden layers, units, and the size and shape of their receptive and projective fields. This paper advocates the view that the network structure should not, as usually done, be determined by trial-and-error but should be computed by the learning algorithm. Incremental learning algorithms can modify the network structure by addition and/or removal of units and/or links. A survey of current connectionist literature is given on this line of thought. "Grow and Learn" (GAL) is a new algorithm that learns an association at one-shot due to being incremental and using a local representation. During the so-called "sleep" phase, units that were previously stored but which are no longer necessary due to recent modifications are removed to minimize network complexity. The incrementally constructed network can later be finetuned off-line to improve performance. Another method proposed that greatly increases recognition accuracy is to train a number of networks and vote over their responses. The algorithm and its variants are tested on recognition of handwritten numerals and seem promising especially in terms of learning speed. This makes the algorithm attractive for on-line learning tasks, e.g., in robotics. The biological plausibility of incremental learning is also discussed briefly. Earlier part of this work was realized at the Laboratoire de Microinformatique of Ecole Polytechnique Federale de Lausanne and was supported by the Fonds National Suisse de la Recherche Scientifique. Later part was realized at and supported by the International Computer Science Institute. A number of people helped by guiding, stimulating discussions or questions: Subutai Ahmad, Peter Clarke, Jerry Feldman, Christian Jutten, Pierre Marchal, Jean Daniel Nicoud, Steve Omohondro and Leon Personnaz. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #6923. 	[Title] Inductive Constraint Logic [Abstract] A novel approach to learning first order logic formulae from positive and negative examples is presented. Whereas present inductive logic programming systems employ examples as true and false ground facts (or clauses), we view examples as interpretations which are true or false for the target theory. This viewpoint allows to reconcile the inductive logic programming paradigm with classical attribute value learning in the sense that the latter is a special case of the former. Because of this property, we are able to adapt AQ and CN2 type algorithms in order to enable learning of full first order formulae. However, whereas classical learning techniques have concentrated on concept representations in disjunctive normal form, we will use a clausal representation, which corresponds to a conjuctive normal form where each conjunct forms a constraint on positive examples. This representation duality reverses also the role of positive and negative examples, both in the heuristics and in the algorithm. The resulting theory is incorporated in a system named ICL (Inductive Constraint Logic). Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #1120020. 	[Title] Learning from undiscounted delayed rewards [Abstract] The general framework of reinforcement learning has been proposed by several researchers for both the solution of optimization problems and the realization of adaptive control schemes. To allow for an efficient application of reinforcement learning in either of these areas, it is necessary to solve both the structural and the temporal credit assignment problem. In this paper, we concentrate on the latter which is usually tackled through the use of learning algorithms that employ discounted rewards. We argue that for realistic problems this kind of solution is not satisfactory, since it does not address the effect of noise originating from different experiences and does not allow for an easy explanation of the parameters involved in the learning process. As a possible solution, we propose to keep the delayed reward undiscounted, but to discount the actual adaptation rate. Empirical results show that dependent on the kind of discount used amore stable convergence and even an increase in performance can be obtained. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #628500. 	[Title] LEARNING TO AVOID COLLISIONS: A REINFORCEMENT LEARNING PARADIGM FOR MOBILE ROBOT NAVIGATION [Abstract] The paper describes a self-learning control system for a mobile robot. Based on sensor information the control system has to provide a steering signal in such a way that collisions are avoided. Since in our case no `examples' are available, the system learns on the basis of an external reinforcement signal which is negative in case of a collision and zero otherwise. We describe the adaptive algorithm which is used for a discrete coding of the state space, and the adaptive algorithm for learning the correct mapping from the input (state) vector to the output (steering) signal. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #520471. 	[Title] The Arguments of Newly Invented Predicates in ILP [Abstract] In this paper we investigate the problem of choosing arguments for a new predicate. We identify the relevant terms to be considered as arguments, and propose methods to choose among them based on propositional minimisation. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #1031453. 	[Title] A Neural Network Pole Balancer that Learns and Operates on a Real Robot in Real Time [Abstract] A neural network approach to the classic inverted pendulum task is presented. This task is the task of keeping a rigid pole, hinged to a cart and free to fall in a plane, in a roughly vertical orientation by moving the cart horizontally in the plane while keeping the cart within some maximum distance of its starting position. This task constitutes a difficult control problem if the parameters of the cart-pole system are not known precisely or are variable. It also forms the basis of an even more complex control-learning problem if the controller must learn the proper actions for successfully balancing the pole given only the current state of the system and a failure signal when the pole angle from the vertical becomes too great or the cart exceeds one of the boundaries placed on its position. The approach presented is demonstrated to be effective for the real-time control of a small, self-contained mini-robot, specially outfitted for the task. Origins and details of the learning scheme, specifics of the mini-robot hardware, and results of actual learning trials are presented. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #1102625. 	[Title] A systematic description of greedy optimisation algorithms for cost sensitive generalisation [Abstract] This paper defines a class of problems involving combinations of induction and (cost) optimisation. A framework is presented that systematically describes problems that involve construction of decision trees or rules, optimising accuracy as well as measurement- and misclassification costs. It does not present any new algorithms but shows how this framework can be used to configure greedy algorithms for constructing such trees or rules. The framework covers a number of existing algorithms. Moreover, the framework can also be used to define algorithm configurations with new functionalities, as expressed in their evaluation functions. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #560936. 	[Title] Monte Carlo Comparison of Non-hierarchical Unsupervised Classifiers [Abstract]  Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #1128256. 	[Title] Where Do SE-trees Perform? (Part I) [Abstract] As a classifier, a Set Enumeration (SE) tree can be viewed as a generalization of decision trees. We empirically characterize domains in which SE-trees are particularly advantageous relative to decision trees. Specifically, we show that: Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #31043. 	[Title] FONN: Combining First Order Logic with Connectionist Learning [Abstract] This paper presents a neural network architecture that can manage structured data and refine knowledge bases expressed in a first order logic language. The presented framework is well suited to classification problems in which concept de scriptions depend upon numerical features of the data. In fact, the main goal of the neural architecture is that of refining the numerical part of the knowledge base, without changing its structure. In particular, we discuss a method to translate a set of classification rules into neural computation units. Here, we focus our attention on the translation method and on algorithms to refine network weights on struc tured data. The classification theory to be refined can be manually handcrafted or automatically acquired by a symbolic relational learning system able to deal with numerical features. As a matter of fact, the primary goal is to bring into a neural network architecture the capability of dealing with structured data of unrestricted size, by allowing to dynamically bind the classification rules to different items occur ring in the input data. An extensive experimentation on a challenging artificial case study shows that the network converges quite fastly and generalizes much better than propositional learners on an equivalent task definition. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #28485. 	[Title] The Efficient Learning of Multiple Task Sequences [Abstract] I present a modular network architecture and a learning algorithm based on incremental dynamic programming that allows a single learning agent to learn to solve multiple Markovian decision tasks (MDTs) with significant transfer of learning across the tasks. I consider a class of MDTs, called composite tasks, formed by temporally concatenating a number of simpler, elemental MDTs. The architecture is trained on a set of composite and elemental MDTs. The temporal structure of a composite task is assumed to be unknown and the architecture learns to produce a temporal decomposition. It is shown that under certain conditions the solution of a composite MDT can be constructed by computationally inexpensive modifications of the solutions of its constituent elemental MDTs. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #28487. 	[Title] LEARNING TO SOLVE MARKOVIAN DECISION PROCESSES [Abstract]  Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #91975. 	[Title] Learning Curve Bounds for Markov Decision Processes with Undiscounted Rewards [Abstract] Markov decision processes (MDPs) with undis-counted rewards represent an important class of problems in decision and control. The goal of learning in these MDPs is to find a policy that yields the maximum expected return per unit time. In large state spaces, computing these averages directly is not feasible; instead, the agent must estimate them by stochastic exploration of the state space. In this case, longer exploration times enable more accurate estimates and more informed decision-making. The learning curve for an MDP measures how the agent's performance depends on the allowed exploration time, T . In this paper we analyze these learning curves for a simple control problem with undiscounted rewards. In particular, methods from statistical mechanics are used to calculate lower bounds on the agent's performance in the thermodynamic limit T ! 1, N ! 1, ff = T =N (finite), where T is the number of time steps allotted per policy evaluation and N is the size of the state space. In this limit, we provide a lower bound on the return of policies that appear optimal based on imperfect statistics. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #36167. 	[Title] Category: Control, Navigation and Planning Preference: Oral presentation Exploiting Model Uncertainty Estimates for Safe Dynamic [Abstract] Model learning combined with dynamic programming has been shown to be effective for learning control of continuous state dynamic systems. The simplest method assumes the learned model is correct and applies dynamic programming to it, but many approximators provide uncertainty estimates on the fit. How can they be exploited? This paper addresses the case where the system must be prevented from having catastrophic failures during learning. We propose a new algorithm adapted from the dual control literature and use Bayesian locally weighted regression models with stochastic dynamic programming. A common reinforcement learning assumption is that aggressive exploration should be encouraged. This paper addresses the converse case in which the system has to reign in exploration. The algorithm is illustrated on a 4 dimensional simulated control problem. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #444240. 	[Title] PLEASE: A prototype learning system using genetic algorithms [Abstract] Prototypes have been proposed as representation of concepts that are used effectively by humans. Developing computational schemes for generating prototypes from examples, however, has proved to be a difficult problem. We present a novel genetic algorithm based prototype learning system, PLEASE, for constructing appropriate prototypes from classified training instances. After constructing a set of prototypes for each of the possible classes, the class of a new input instance is determined by the nearest prototype to this instance. Attributes are assumed to be ordinal in nature and prototypes are represented as sets of feature-value pairs. A genetic algorithm is used to evolve the number of prototypes per class and their positions on the input space. We present experimental results on a series of artificial problems of varying complexity. PLEASE performs competitively with several nearest neighbor classification algorithms on the problem set. An analysis of the strengths and weaknesses of the initial version of our system motivates the need for additional operators. The inclusion of these operators substantially improves the performance of the system on particularly difficult problems. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #1365. 	[Title] Cholinergic suppression of transmission may allow combined associative memory function and self-organization in the neocortex. [Abstract] Selective suppression of transmission at feedback synapses during learning is proposed as a mechanism for combining associative feedback with self-organization of feedforward synapses. Experimental data demonstrates cholinergic suppression of synaptic transmission in layer I (feedback synapses), and a lack of suppression in layer IV (feed-forward synapses). A network with this feature uses local rules to learn mappings which are not linearly separable. During learning, sensory stimuli and desired response are simultaneously presented as input. Feedforward connections form self-organized representations of input, while suppressed feedback connections learn the transpose of feedfor-ward connectivity. During recall, suppression is removed, sensory input activates the self-organized representation, and activity generates the learned response. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #42847. 	[Title] GROWING RADIAL BASIS FUNCTION NETWORKS [Abstract] This paper presents and evaluates two algorithms for incrementally constructing Radial Basis Function Networks, a class of neural networks which looks more suitable for adtaptive control applications than the more popular backpropagation networks. The first algorithm has been derived by a previous method developed by Fritzke, while the second one has been inspired by the CART algorithm developed by Breiman for generation regression trees. Both algorithms proved to work well on a number of tests and exhibit comparable performances. An evaluation on the standard case study of the Mackey-Glass temporal series is reported. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #593260. 	[Title] Evolving Graphs and Networks with Edge Encoding: Preliminary Report [Abstract] We present an alternative to the cellular encoding technique [Gruau 1992] for evolving graph and network structures via genetic programming. The new technique, called edge encoding, uses edge operators rather than the node operators of cellular encoding. While both cellular encoding and edge encoding can produce all possible graphs, the two encodings bias the genetic search process in different ways; each may therefore be most useful for a different set of problems. The problems for which these techniques may be used, and for which we think edge encoding may be particularly useful, include the evolution of recurrent neural networks, finite automata, and graph-based queries to symbolic knowledge bases. In this preliminary report we present a technical description of edge encoding and an initial comparison to cellular encoding. Experimental investigation of the relative merits of these encoding schemes is currently in progress. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #251756. 	[Title] Cellular Encoding Applied to Neurocontrol [Abstract] Neural networks are trained for balancing 1 and 2 poles attached to a cart on a fixed track. For one variant of the single pole system, only pole angle and cart position variables are supplied as inputs; the network must learn to compute velocities. All of the problems are solved using a fixed architecture and using a new version of cellular encoding that evolves an application specific architecture with real-valued weights. The learning times and generalization capabilities are compared for neural networks developed using both methods. After a post processing simplification, topologies produced by cellular encoding were very simple and could be analyzed. Architectures with no hidden units were produced for the single pole and the two pole problem when velocity information is supplied as an input. Moreover, these linear solutions display good generalization. For all the control problems, cellular encoding can automatically generate architectures whose complexity and structure reflect the features of the problem to solve. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #13686. 	[Title] Framework for Combining Symbolic and Neural Learning rule extraction from neural networks the KBANN algorithm [Abstract] Technical Report 1123, Computer Sciences Department, University of Wisconsin - Madison, Nov. 1992 ABSTRACT This article describes an approach to combining symbolic and connectionist approaches to machine learning. A three-stage framework is presented and the research of several groups is reviewed with respect to this framework. The first stage involves the insertion of symbolic knowledge into neural networks, the second addresses the refinement of this prior knowledge in its neural representation, while the third concerns the extraction of the refined symbolic knowledge. Experimental results and open research issues are discussed. A shorter version of this paper will appear in Machine Learning. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #28542. 	[Title] Analysis of Some Incremental Variants of Policy Iteration: First Steps Toward Understanding Actor-Critic Learning Systems [Abstract] Northeastern University College of Computer Science Technical Report NU-CCS-93-11 fl We gratefully acknowledge the substantial contributions to this effort provided by Andy Barto, who sparked our original interest in these questions and whose continued encouragement and insightful comments and criticisms have helped us greatly. Recent discussions with Satinder Singh and Vijay Gullapalli have also had a helpful impact on this work. Special thanks also to Rich Sutton, who has influenced our thinking on this subject in numerous ways. This work was supported by Grant IRI-8921275 from the National Science Foundation and by the U. S. Air Force. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #1128839. 	[Title] Requirements and use of neural networks for industrial appli- [Abstract] Modern industry of today needs flexible, adaptive and fault-tolerant methods for information processing. Several applications have shown that neural networks fulfill these requirements. In this paper application areas, in which neural networks have been successfully used, are presented. Then a kind of check list is described, which mentioned the different steps, when applying neural networks. The paper finished with a discussion of some neural networks projects done in the research group Interactive Planning at the Research Center for Computer Science (FZI). Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #293285. 	[Title] Why Experimentation can be better than "Perfect Guidance" [Abstract] Many problems correspond to the classical control task of determining the appropriate control action to take, given some (sequence of) observations. One standard approach to learning these control rules, called behavior cloning, involves watching a perfect operator operate a plant, and then trying to emulate its behavior. In the experimental learning approach, by contrast, the learner first guesses an initial operation-to-action policy and tries it out. If this policy performs sub-optimally, the learner can modify it to produce a new policy, and recur. This paper discusses the relative effectiveness of these two approaches, especially in the presence of perceptual aliasing, showing in particular that the experimental learner can often learn more effectively than the cloning one. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #94641. 	[Title] Complexity Compression and Evolution [Abstract] Compression of information is an important concept in the theory of learning. We argue for the hypothesis that there is an inherent compression pressure towards short, elegant and general solutions in a genetic programming system and other variable length evolutionary algorithms. This pressure becomes visible if the size or complexity of solutions are measured without non-effective code segments called introns. The built in parsimony pressure effects complex fitness functions, crossover probability, generality, maximum depth or length of solutions, explicit parsimony, granularity of fitness function, initialization depth or length, and modulariz-ation. Some of these effects are positive and some are negative. In this work we provide a basis for an analysis of these effects and suggestions to overcome the negative implications in order to obtain the balance needed for successful evolution. An empirical investigation that supports our hypothesis is also presented. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #254923. 	[Title] A genetic prototype learner [Abstract] Supervised classification problems have received considerable attention from the machine learning community. We propose a novel genetic algorithm based prototype learning system, PLEASE, for this class of problems. Given a set of prototypes for each of the possible classes, the class of an input instance is determined by the prototype nearest to this instance. We assume ordinal attributes and prototypes are represented as sets of feature-value pairs. A genetic algorithm is used to evolve the number of prototypes per class and their positions on the input space as determined by corresponding feature-value pairs. Comparisons with C4.5 on a set of artificial problems of controlled complexity demonstrate the effectiveness of the pro posed system. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #85452. 	[Title] Constructive Induction Using a Non-Greedy Strategy for Feature Selection [Abstract] We present a method for feature construction and selection that finds a minimal set of conjunctive features that are appropriate to perform the classification task. For problems where this bias is appropriate, the method outperforms other constructive induction algorithms and is able to achieve higher classification accuracy. The application of the method in the search for minimal multi-level boolean expressions is presented and analyzed with the help of some examples. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #976334. 	[Title] Applying Online Search Techniques to Continuous-State Reinforcement Learning key to the success of the local [Abstract] In this paper, we describe methods for efficiently computing better solutions to control problems in continuous state spaces. We provide algorithms that exploit online search to boost the power of very approximate value functions discovered by traditional reinforcement learning techniques. We examine local searches, where the agent performs a finite-depth lookahead search, and global searches, where the agent performs a search for a trajectory all the way from the current state to a goal state. The key to the success of the global methods lies in using aggressive state-space search techniques such as uniform-cost search and A fl , tamed into a tractable form by exploiting neighborhood relations and trajectory constraints that arise from continuous-space dynamic control. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #168410. 	[Title] A Modular Q-Learning Architecture for Manipulator Task Decomposition `Data storage in the cerebellar model ar [Abstract] Compositional Q-Learning (CQ-L) (Singh 1992) is a modular approach to learning to perform composite tasks made up of several elemental tasks by reinforcement learning. Skills acquired while performing elemental tasks are also applied to solve composite tasks. Individual skills compete for the right to act and only winning skills are included in the decomposition of the composite task. We extend the original CQ-L concept in two ways: (1) a more general reward function, and (2) the agent can have more than one actuator. We use the CQ-L architecture to acquire skills for performing composite tasks with a simulated two-linked manipulator having large state and action spaces. The manipulator is a non-linear dynamical system and we require its end-effector to be at specific positions in the workspace. Fast function approximation in each of the Q-modules is achieved through the use of an array of Cerebellar Model Articulation Controller (CMAC) (Albus Our research interests involve the scaling up of machine learning methods, especially reinforcement learning, for autonomous robot control. We are interested in function approximators suitable for reinforcement learning in problems with large state spaces, such as the Cerebellar Model Articulation Controller (CMAC) (Albus 1975) which permit fast, online learning and good local generalization. In addition, we are interested in task decomposition by reinforcement learning and the use of hierarchical and modular function approximator architectures. We are examining the effectiveness of a modified Hierarchical Mixtures of Experts (HME) (Jordan & Jacobs 1993) approach for reinforcement learning since the original HME was developed mainly for supervised learning and batch learning tasks. The incorporation of domain knowledge into reinforcement learning agents is an important way of extending their capabilities. Default policies can be specified, and domain knowledge can also be used to restrict the size of the state-action space, leading to faster learning. We are investigating the use of Q-Learning (Watkins 1989) in planning tasks, using a classifier system (Holland 1986) to encode the necessary condition-action rules. Jordan, M. & Jacobs, R. (1993), Hierarchical mixtures of experts and the EM algorithm, Technical Report 9301, MIT Computational Cognitive Science. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #39403. 	[Title] Auto-exploratory Average Reward Reinforcement Learning [Abstract] We introduce a model-based average reward Reinforcement Learning method called H-learning and compare it with its discounted counterpart, Adaptive Real-Time Dynamic Programming, in a simulated robot scheduling task. We also introduce an extension to H-learning, which automatically explores the unexplored parts of the state space, while always choosing greedy actions with respect to the current value function. We show that this "Auto-exploratory H-learning" performs better than the original H-learning under previously studied exploration methods such as random, recency-based, or counter-based exploration. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #84459. 	[Title] Learning Concepts from Sensor Data of a Mobile Robot [Abstract] Machine learning can be a most valuable tool for improving the flexibility and efficiency of robot applications. Many approaches to applying machine learning to robotics are known. Some approaches enhance the robot's high-level processing, the planning capabilities. Other approaches enhance the low-level processing, the control of basic actions. In contrast, the approach presented in this paper uses machine learning for enhancing the link between the low-level representations of sensing and action and the high-level representation of planning. The aim is to facilitate the communication between the robot and the human user. A hierarchy of concepts is learned from route records of a mobile robot. Perception and action are combined at every level, i.e., the concepts are perceptually anchored. The relational learning algorithm grdt has been developed which completely searches in a hypothesis space, that is restricted by rule schemata, which the user defines in terms of grammars. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #1104379. 	[Title] GA-RBF: A Self-Optimising RBF Network [Abstract] The effects of a neural network's topology on its performance are well known, yet the question of finding optimal configurations automatically remains largely open. This paper proposes a solution to this problem for RBF networks. A self- optimising approach, driven by an evolutionary strategy, is taken. The algorithm uses output information and a computationally efficient approximation of RBF networks to optimise the K-means clustering process by co-evolving the two determinant parameters of the network's layout: the number of centroids and the centroids' positions. Empirical results demonstrate promise. Please summarise all the given information.

Paper #4584 has the title `References elements that can solve difficult learning control problems. on Simulation of Adaptive Behavior, pages
` and the abstract `Miller, G. (1956). The magical number seven, plus or minus two: Some limits on our capacity for processing information. The Psychological Review, 63(2):81-97. Schmidhuber, J. (1990b). Towards compositional learning with dynamic neural networks. Technical Report FKI-129-90, Technische Universitat Munchen, Institut fu Informatik. Servan-Schreiber, D., Cleermans, A., and McClelland, J. (1988). Encoding sequential structure in simple recurrent networks. Technical Report CMU-CS-88-183, Carnegie Mellon University, Computer Science Department. 
`.
Question: Which category should Paper #4584 be classified as ?
You can select one from ['Neural_Networks', 'Case_Based', 'Theory', 'Reinforcement_Learning', 'Probabilistic_Methods', 'Rule_Learning', 'Genetic_Algorithms'].


Paper #4584 has the title `References elements that can solve difficult learning control problems. on Simulation of Adaptive Behavior, pages
` and the abstract `Miller, G. (1956). The magical number seven, plus or minus two: Some limits on our capacity for processing information. The Psychological Review, 63(2):81-97. Schmidhuber, J. (1990b). Towards compositional learning with dynamic neural networks. Technical Report FKI-129-90, Technische Universitat Munchen, Institut fu Informatik. Servan-Schreiber, D., Cleermans, A., and McClelland, J. (1988). Encoding sequential structure in simple recurrent networks. Technical Report CMU-CS-88-183, Carnegie Mellon University, Computer Science Department. 
`.
Question: Which category should Paper #4584 be classified as ?
You can select one from ['Neural_Networks', 'Case_Based', 'Theory', 'Reinforcement_Learning', 'Probabilistic_Methods', 'Rule_Learning', 'Genetic_Algorithms'].

This paper should be classified into: Reinforcement_Learning