Above are papers related to the following paper: [Paper_ID] Paper #35852. 	[Title] Generalization and scaling in reinforcement learning [Abstract] In associative reinforcement learning, an environment generates input vectors, a learning system generates possible output vectors, and a reinforcement function computes feedback signals from the input-output pairs. The task is to discover and remember input-output pairs that generate rewards. Especially difficult cases occur when rewards are rare, since the expected time for any algorithm can grow exponentially with the size of the problem. Nonetheless, if a reinforcement function possesses regularities, and a learning algorithm exploits them, learning time can be reduced below that of non-generalizing algorithms. This paper describes a neural network algorithm called complementary reinforcement back-propagation (CRBP), and reports simulation results on problems designed to offer differing opportunities for generalization. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #35. 	[Title] 4 Implementing Application Specific Routines  Genetic algorithms in search, optimization, and machine learning. Reading, MA: Addison-Wesley. [Abstract] To implement a specific application, you should only have to change the file app.c. Section 2 describes the routines in app.c in detail. If you use additional variables for your specific problem, the easiest method of making them available to other program units is to declare them in sga.h and external.h. However, take care that you do not redeclare existing variables. Two example applications files are included in the SGA-C distribution. The file app1.c performs the simple example problem included with the Pascal version; finding the maximum of x 10 , where x is an integer interpretation of a chromosome. A slightly more complex application is include in app2.c. This application illustrates two features that have been added to SGA-C. The first of these is the ithruj2int function, which converts bits i through j in a chromosome to an integer. The second new feature is the utility pointer that is associated with each population member. The example application interprets each chromosome as a set of concatenated integers in binary form. The lengths of these integer fields is determined by the user-specified value of field size, which is read in by the function app data(). The field size must be less than the smallest of the chromosome length and the length of an unsigned integer. An integer array for storing the interpreted form of each chromosome is dynamically allocated and assigned to the chromosome's utility pointer in app malloc(). The ithruj2int routine (see utility.c) is used to translate each chromosome into its associated vector. The fitness for each chromosome is simply the sum of the squares of these integers. This example application will function for any chromosome length. SGA-C is intended to be a simple program for first-time GA experimentation. It is not intended to be definitive in terms of its efficiency or the grace of its implementation. The authors are interested in the comments, criticisms, and bug reports from SGA-C users, so that the code can be refined for easier use in subsequent versions. Please email your comments to rob@galab2.mh.ua.edu, or write to TCGA: The authors gratefully acknowledge support provided by NASA under Grant NGT-50224 and support provided by the National Science Foundation under Grant CTS-8451610. We also thank Hillol Kargupta for donating his tournament selection implementation. Booker, L. B. (1982). Intelligent behavior as an adaptation to the task environment (Doctoral dissertation, Technical Report No. 243. Ann Arbor: University of Michigan, Logic of Computers Group). Dissertations Abstracts International, 43(2), 469B. (University Microfilms No. 8214966) Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #41732. 	[Title] Emergent Control and Planning in an Autonomous Vehicle [Abstract] We use a connectionist network trained with reinforcement to control both an autonomous robot vehicle and a simulated robot. We show that given appropriate sensory data and architectural structure, a network can learn to control the robot for a simple navigation problem. We then investigate a more complex goal-based problem and examine the plan-like behavior that emerges. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #45605. 	[Title] Learning and evolution in neural networks [Abstract]  Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #1034. 	[Title] Landscapes, Learning Costs and Genetic Assimilation. [Abstract] The evolution of a population can be guided by phenotypic traits acquired by members of that population during their lifetime. This phenomenon, known as the Baldwin Effect, can speed the evolutionary process as traits that are initially acquired become genetically specified in later generations. This paper presents conditions under which this genetic assimilation can take place. As well as the benefits that lifetime adaptation can give a population, there may be a cost to be paid for that adaptive ability. It is the evolutionary trade-off between these costs and benefits that provides the selection pressure for acquired traits to become genetically specified. It is also noted that genotypic space, in which evolution operates, and phenotypic space, on which adaptive processes (such as learning) operate, are, in general, of a different nature. To guarantee an acquired characteristic can become genetically specified, then these spaces must have the property of neighbourhood correlation which means that a small distance between two individuals in phenotypic space implies that there is a small distance between the same two individuals in genotypic space. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #1035. 	[Title] EVOLVING SENSORS IN ENVIRONMENTS OF CONTROLLED COMPLEXITY [Abstract] 1 . Sensors represent a crucial link between the evolutionary forces shaping a species' relationship with its environment, and the individual's cognitive abilities to behave and learn. We report on experiments using a new class of "latent energy environments" (LEE) models to define environments of carefully controlled complexity which allow us to state bounds for random and optimal behaviors that are independent of strategies for achieving the behaviors. Using LEE's analytic basis for defining environments, we then use neural networks (NNets) to model individuals and a steady - state genetic algorithm to model an evolutionary process shaping the NNets, in particular their sensors. Our experiments consider two types of "contact" and "ambient" sensors, and variants where the NNets are not allowed to learn, learn via error correction from internal prediction, and via reinforcement learning. We find that predictive learning, even when using a larger repertoire of the more sophisticated ambient sensors, provides no advantage over NNets unable to learn. However, reinforcement learning using a small number of crude contact sensors does provide a significant advantage. Our analysis of these results points to a tradeoff between the genetic "robustness" of sensors and their informativeness to a learning system. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #141324. 	[Title] An Analysis of the MAX Problem in Genetic Programming hold only in some cases, in [Abstract] We present a detailed analysis of the evolution of genetic programming (GP) populations using the problem of finding a program which returns the maximum possible value for a given terminal and function set and a depth limit on the program tree (known as the MAX problem). We confirm the basic message of [ Gathercole and Ross, 1996 ] that crossover together with program size restrictions can be responsible for premature convergence to a suboptimal solution. We show that this can happen even when the population retains a high level of variety and show that in many cases evolution from the sub-optimal solution to the solution is possible if sufficient time is allowed. In both cases theoretical models are presented and compared with actual runs. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #1128975. 	[Title] learning easier tasks. More work is necessary in order to determine more precisely the relationship [Abstract] We have attempted to obtain a stronger correlation between the relationship between G 0 and G 1 and performance. This has included studying the variance in the fitnesses of the members of the population, as well as observing the rate of convergence of the GP with respect to G 1 when a population was evolved for G 0 . 13 Unfortunately, we have not yet been able to obtain a significant correlation. In future work, we plan to to track the genetic diversity (we have only considered phenotypic variance so far) of populations in order to shed some light on the underlying mechanism for priming. One factor that has made this analysis difficult so far is our use of genetic programming, for which the space of genotypes is very large, (i.e., there are many redundant solutions), and for which the neighborhood structure is less easily intuited than that of a standard genetic algorithm. Since there is every reason to believe that the underlying mechanism of incremental evolution is largely independent of the peculiarities of genetic programming, we are currently investigating the incremental evolution mechanism using genetic algorithms with fixed-length genotypes. This should enable a better understanding of the mechanism. Ultimately, we will scale up this research effort to analyze incremental evolution with more than one transition between test cases. This will involve many open issues regarding the optimization of the transition schedule between test cases. 13 We performed the following experiment: Let F it(I; G) be the fitness value of a genetic program I according to the evaluation function G, and Best Of(P op; t; G) be the member I fl of population P op at time t with highest fitness according to G | in other words, I fl = Best Of (P op; t; G) maximizes F it(I; G) over all I 2 P op. A population P op 0 was evolved in the usual manner using evaluation function G 0 for t = 25 generations. However, at each generation 1 i 25 we also evaluated the current population using evaluation function G 1 , and recorded the value of F it(Best Of (P op; i; G 1 ); G 1 ). In other words, we evolved the population using G 0 as the evaluation function, but at every generation we also computed the fitness of the best individual in the population according to G 1 and saved this value. Using the same random seed and control parameters, we then evolved a population P op 1 for t = 30 generations using G 1 as the evaluation function (note that at generation 0, P op 1 is identical to P op 0 ). For all values of t, we compared F it(Best Of (P op 0 ; t; G 1 ); G 1 ) with F it(Best Of (P op 1 ; t; G 1 ); G 1 ). in order to better formalize and exploit this notion of domain difficulty. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #1110515. 	[Title] Environmental Effects on Minimal Behaviors in the Minimat World [Abstract] The structure of an environment affects the behaviors of the organisms that have evolved in it. How is that structure to be described, and how can its behavioral consequences be explained and predicted? We aim to establish initial answers to these questions by simulating the evolution of very simple organisms in simple environments with different structures. Our artificial creatures, called "minimats," have neither sensors nor memory and behave solely by picking amongst the actions of moving, eating, reproducing, and sitting, according to an inherited probability distribution. Our simulated environments contain only food (and multiple minimats) and are structured in terms of their spatial and temporal food density and the patchiness with which the food appears. Changes in these environmental parameters affect the evolved behaviors of minimats in different ways, and all three parameters are of importance in describing the minimat world. One of the most useful behavioral strategies that evolves is "looping" movement, which allows minimats-despite their lack of internal state-to match their behavior to the temporal (and spatial) structure of their environment. Ultimately we find that minimats construct their own environments through their individual behaviors, making the study of the impact of global environment structure on individual behavior much more complex. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #35061. 	[Title] Evolving Networks: Using the Genetic Algorithm with Connectionist Learning [Abstract]  Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #1107062. 	[Title] Guiding or Hiding: Explorations into the Effects of Learning on the Rate of Evolution. [Abstract] Individual lifetime learning can `guide' an evolving population to areas of high fitness in genotype space through an evolutionary phenomenon known as the Baldwin effect (Baldwin, 1896; Hin-ton & Nowlan, 1987). It is the accepted wisdom that this guiding speeds up the rate of evolution. By highlighting another interaction between learning and evolution, that will be termed the Hiding effect, it will be argued here that this depends on the measure of evolutionary speed one adopts. The Hiding effect shows that learning can reduce the selection pressure between individuals by `hiding' their genetic differences. There is thus a trade-off between the Baldwin effect and the Hiding effect to determine learning's influence on evolution and two factors that contribute to this trade-off, the cost of learning and landscape epis tasis, are investigated experimentally. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #240791. 	[Title] Auto-teaching: networks that develop their own teaching input [Abstract] Backpropagation learning (Rumelhart, Hinton and Williams, 1986) is a useful research tool but it has a number of undesiderable features such as having the experimenter decide from outside what should be learned. We describe a number of simulations of neural networks that internally generate their own teaching input. The networks generate the teaching input by trasforming the network input through connection weights that are evolved using a form of genetic algorithm. What results is an innate (evolved) capacity not to behave efficiently in an environment but to learn to behave efficiently. The analysis of what these networks evolve to learn shows some interesting results. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #486840. 	[Title] Adaptation in constant utility non-stationary environments [Abstract] Environments that vary over time present a fundamental problem to adaptive systems. Although in the worst case there is no hope of effective adaptation, some forms environmental variability do provide adaptive opportunities. We consider a broad class of non-stationary environments, those which combine a variable result function with an invariant utility function, and demonstrate via simulation that an adaptive strategy employing both evolution and learning can tolerate a much higher rate of environmental variation than an evolution-only strategy. We suggest that in many cases where stability has previously been assumed, the constant utility non-stationary environment may in fact be a more powerful viewpoint. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #595063. 	[Title] Improving the Performance of Evolutionary Optimization by Dynamically Scaling the Evaluation Function [Abstract] Traditional evolutionary optimization algorithms assume a static evaluation function, according to which solutions are evolved. Incremental evolution is an approach through which a dynamic evaluation function is scaled over time in order to improve the performance of evolutionary optimization. In this paper, we present empirical results that demonstrate the effectiveness of this approach for genetic programming. Using two domains, a two-agent pursuit-evasion game and the Tracker [6] trail-following task, we demonstrate that incremental evolution is most successful when applied near the beginning of an evolutionary run. We also show that incremental evolution can be successful when the intermediate evaluation functions are more difficult than the target evaluation function, as well as when they are easier than the target function. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #1113438. 	[Title] Modeling the Evolution of Motivation [Abstract] In order for learning to improve the adaptiveness of an animal's behavior and thus direct evolution in the way Baldwin suggested, the learning mechanism must incorporate an innate evaluation of how the animal's actions influence its reproductive fitness. For example, many circumstances that damage an animal, or otherwise reduce its fitness are painful and tend to be avoided. We refer to the mechanism by which an animal evaluates the fitness consequences of its actions as a "motivation system," and argue that such a system must evolve along with the behaviors it evaluates. We describe simulations of the evolution of populations of agents instantiating a number of different architectures for generating action and learning, in worlds of differing complexity. We find that in some cases, members of the populations evolve motivation systems that are accurate enough to direct learning so as to increase the fitness of the actions the agents perform. Furthermore, the motivation systems tend to incorporate systematic distortions in their representations of the worlds they inhabit; these distortions can increase the adaptiveness of the behavior generated. Please summarise all the given information.

Paper #35852 has the title `Generalization and scaling in reinforcement learning  
` and the abstract `In associative reinforcement learning, an environment generates input vectors, a learning system generates possible output vectors, and a reinforcement function computes feedback signals from the input-output pairs. The task is to discover and remember input-output pairs that generate rewards. Especially difficult cases occur when rewards are rare, since the expected time for any algorithm can grow exponentially with the size of the problem. Nonetheless, if a reinforcement function possesses regularities, and a learning algorithm exploits them, learning time can be reduced below that of non-generalizing algorithms. This paper describes a neural network algorithm called complementary reinforcement back-propagation (CRBP), and reports simulation results on problems designed to offer differing opportunities for generalization.
`.
Question: Which category should Paper #35852 be classified as ?
You can select one from ['Neural_Networks', 'Case_Based', 'Theory', 'Reinforcement_Learning', 'Probabilistic_Methods', 'Rule_Learning', 'Genetic_Algorithms'].


Paper #35852 has the title `Generalization and scaling in reinforcement learning  
` and the abstract `In associative reinforcement learning, an environment generates input vectors, a learning system generates possible output vectors, and a reinforcement function computes feedback signals from the input-output pairs. The task is to discover and remember input-output pairs that generate rewards. Especially difficult cases occur when rewards are rare, since the expected time for any algorithm can grow exponentially with the size of the problem. Nonetheless, if a reinforcement function possesses regularities, and a learning algorithm exploits them, learning time can be reduced below that of non-generalizing algorithms. This paper describes a neural network algorithm called complementary reinforcement back-propagation (CRBP), and reports simulation results on problems designed to offer differing opportunities for generalization.
`.
Question: Which category should Paper #35852 be classified as ?
You can select one from ['Neural_Networks', 'Case_Based', 'Theory', 'Reinforcement_Learning', 'Probabilistic_Methods', 'Rule_Learning', 'Genetic_Algorithms'].

This paper should be classified into: Reinforcement_Learning