Above are papers related to the following paper: [Paper_ID] Paper #214472. 	[Title] A Computer Scientist's View of Life, the Universe, and Everything [Abstract] Is the universe computable? If so, it may be much cheaper in terms of information requirements to compute all computable universes instead of just ours. I apply basic concepts of Kolmogorov complexity theory to the set of possible universes, and chat about perceived and true randomness, life, generalization, and learning in a given universe. Assumptions. A long time ago, the Great Programmer wrote a program that runs all possible universes on His Big Computer. "Possible" means "computable": (1) Each universe evolves on a discrete time scale. (2) Any universe's state at a given time is describable by a finite number of bits. One of the many universes is ours, despite some who evolved in it and claim it is incomputable. Computable universes. Let T M denote an arbitrary universal Turing machine with unidirectional output tape. T M 's input and output symbols are "0", "1", and "," (comma). T M 's possible input programs can be ordered alphabetically: "" (empty program), "0", "1", ",", "00", "01", "0,", "10", "11", "1,", ",0", ",1", ",,", "000", etc. Let A k denote T M 's k-th program in this list. Its output will be a finite or infinite string over the alphabet f "0","1",","g. This sequence of bitstrings separated by commas will be interpreted as the evolution E k of universe U k . If E k includes at least one comma, then let U l k represents U k 's state at the l-th time step of E k (k; l 2 f1; 2; : : : ; g). E k is represented by the sequence U 1 k corresponds to U k 's big bang. Different algorithms may compute the same universe. Some universes are finite (those whose programs cease producing outputs at some point), others are not. I don't know about ours. TM not important. The choice of the Turing machine is not important. This is due to the compiler theorem: for each universal Turing machine C there exists a constant prefix C 2 f "0","1",","g fl such that for all possible programs p, C's output in response to program C p is identical to T M 's output in response to p. The prefix C is the compiler that compiles programs for T M into equivalent programs for C. k denote the l-th (possibly empty) bitstring before the l-th comma. U l Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #170338. 	[Title] Learning Team Strategies With Multiple Policy-Sharing Agents: A Soccer Case Study [Abstract] We use simulated soccer to study multiagent learning. Each team's players (agents) share action set and policy but may behave differently due to position-dependent inputs. All agents making up a team are rewarded or punished collectively in case of goals. We conduct simulations with varying team sizes, and compare two learning algorithms: TD-Q learning with linear neural networks (TD-Q) and Probabilistic Incremental Program Evolution (PIPE). TD-Q is based on evaluation functions (EFs) mapping input/action pairs to expected reward, while PIPE searches policy space directly. PIPE uses adaptive "probabilistic prototype trees" to synthesize programs that calculate action probabilities from current inputs. Our results show that TD-Q encounters several difficulties in learning appropriate shared EFs. PIPE, however, does not depend on EFs and can find good policies faster and more reliably. This suggests that in multiagent learning scenarios direct search through policy space can offer advantages over EF-based approaches. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #358884. 	[Title] THE DISCOVERY OF ALGORITHMIC PROBABILITY [Abstract]  Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #75693. 	[Title] REINFORCEMENT LEARNING WITH SELF-MODIFYING POLICIES [Abstract] A learner's modifiable components are called its policy. An algorithm that modifies the policy is a learning algorithm. If the learning algorithm has modifiable components represented as part of the policy, then we speak of a self-modifying policy (SMP). SMPs can modify the way they modify themselves etc. They are of interest in situations where the initial learning algorithm itself can be improved by experience | this is what we call "learning to learn". How can we force some (stochastic) SMP to trigger better and better self-modifications? The success-story algorithm (SSA) addresses this question in a lifelong reinforcement learning context. During the learner's life-time, SSA is occasionally called at times computed according to SMP itself. SSA uses backtracking to undo those SMP-generated SMP-modifications that have not been empirically observed to trigger lifelong reward accelerations (measured up until the current SSA call | this evaluates the long-term effects of SMP-modifications setting the stage for later SMP-modifications). SMP-modifications that survive SSA represent a lifelong success history. Until the next SSA call, they build the basis for additional SMP-modifications. Solely by self-modifications our SMP/SSA-based learners solve a complex task in a partially observable environment (POE) whose state space is far bigger than most reported in the POE literature. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #75694. 	[Title] Machine Learning,  Shifting Inductive Bias with Success-Story Algorithm, Adaptive Levin Search, and Incremental Self-Improvement [Abstract] We study task sequences that allow for speeding up the learner's average reward intake through appropriate shifts of inductive bias (changes of the learner's policy). To evaluate long-term effects of bias shifts setting the stage for later bias shifts we use the "success-story algorithm" (SSA). SSA is occasionally called at times that may depend on the policy itself. It uses backtracking to undo those bias shifts that have not been empirically observed to trigger long-term reward accelerations (measured up until the current SSA call). Bias shifts that survive SSA represent a lifelong success history. Until the next SSA call, they are considered useful and build the basis for additional bias shifts. SSA allows for plugging in a wide variety of learning algorithms. We plug in (1) a novel, adaptive extension of Levin search and (2) a method for embedding the learner's policy modification strategy within the policy itself (incremental self-improvement). Our inductive transfer case studies involve complex, partially observable environments where traditional reinforcement learning fails. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #1105148. 	[Title] ENVIRONMENT-INDEPENDENT REINFORCEMENT ACCELERATION  difference between time and space is that you can't reuse time. [Abstract] A reinforcement learning system with limited computational resources interacts with an unrestricted, unknown environment. Its goal is to maximize cumulative reward, to be obtained throughout its limited, unknown lifetime. System policy is an arbitrary modifiable algorithm mapping environmental inputs and internal states to outputs and new internal states. The problem is: in realistic, unknown environments, each policy modification process (PMP) occurring during system life may have unpredictable influence on environmental states, rewards and PMPs at any later time. Existing reinforcement learning algorithms cannot properly deal with this. Neither can naive exhaustive search among all policy candidates | not even in case of very small search spaces. In fact, a reasonable way of measuring performance improvements in such general (but typical) situations is missing. I define such a measure based on the novel "reinforcement acceleration criterion" (RAC). At a given time, RAC is satisfied if the beginning of each completed PMP that computed a currently valid policy modification has been followed by long-term acceleration of average reinforcement intake (the computation time for later PMPs is taken into account). I present a method called "environment-independent reinforcement acceleration" (EIRA) which is guaranteed to achieve RAC. EIRA does neither care whether the system's policy allows for changing itself, nor whether there are multiple, interacting learning systems. Consequences are: (1) a sound theoretical framework for "meta-learning" (because the success of a PMP recursively depends on the success of all later PMPs, for which it is setting the stage). (2) A sound theoretical framework for multi-agent learning. The principles have been implemented (1) in a single system using an assembler-like programming language to modify its own policy, and (2) a system consisting of multiple agents, where each agent is in fact just a connection in a fully recurrent reinforcement learning neural net. A by-product of this research is a general reinforcement learning algorithm for such nets. Preliminary experiments illustrate the theory. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #131318. 	[Title] FLAT MINIMA Neural Computation 9(1):1-42 (1997) [Abstract] We present a new algorithm for finding low complexity neural networks with high generalization capability. The algorithm searches for a "flat" minimum of the error function. A flat minimum is a large connected region in weight-space where the error remains approximately constant. An MDL-based, Bayesian argument suggests that flat minima correspond to "simple" networks and low expected overfitting. The argument is based on a Gibbs algorithm variant and a novel way of splitting generalization error into underfitting and overfitting error. Unlike many previous approaches, ours does not require Gaussian assumptions and does not depend on a "good" weight prior instead we have a prior over input/output functions, thus taking into account net architecture and training set. Although our algorithm requires the computation of second order derivatives, it has backprop's order of complexity. Automatically, it effectively prunes units, weights, and input lines. Various experiments with feedforward and recurrent nets are described. In an application to stock market prediction, flat minimum search outperforms (1) conventional backprop, (2) weight decay, (3) "optimal brain surgeon" / "optimal brain damage". We also provide pseudo code of the algorithm (omitted from the NC-version). Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #13208. 	[Title] ON LEARNING HOW TO LEARN LEARNING STRATEGIES [Abstract] This paper introduces the "incremental self-improvement paradigm". Unlike previous methods, incremental self-improvement encourages a reinforcement learning system to improve the way it learns, and to improve the way it improves the way it learns ..., without significant theoretical limitations | the system is able to "shift its inductive bias" in a universal way. Its major features are: (1) There is no explicit difference between "learning", "meta-learning", and other kinds of information processing. Using a Turing machine equivalent programming language, the system itself occasionally executes self-delimiting, initially highly random "self-modification programs" which modify the context-dependent probabilities of future action sequences (including future self-modification programs). (2) The system keeps only those probability modifications computed by "useful" self-modification programs: those which bring about more payoff (reward, reinforcement) per time than all previous self-modification programs. (3) The computation of payoff per time takes into account all the computation time required for learning | the entire system life is considered: boundaries between learning trials are ignored (if there are any). A particular implementation based on the novel paradigm is presented. It is designed to exploit what conventional digital machines are good at: fast storage addressing, arithmetic operations etc. Experiments illustrate the system's mode of operation. Keywords: Self-improvement, self-reference, introspection, machine-learning, reinforcement learning. Note: This is the revised and extended version of an earlier report from November 24, 1994. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #13212. 	[Title] DISCOVERING NEURAL NETS WITH LOW KOLMOGOROV COMPLEXITY AND HIGH GENERALIZATION CAPABILITY Neural Networks 10(5):857-873, 1997 [Abstract] Many neural net learning algorithms aim at finding "simple" nets to explain training data. The expectation is: the "simpler" the networks, the better the generalization on test data (! Occam's razor). Previous implementations, however, use measures for "simplicity" that lack the power, universality and elegance of those based on Kolmogorov complexity and Solomonoff's algorithmic probability. Likewise, most previous approaches (especially those of the "Bayesian" kind) suffer from the problem of choosing appropriate priors. This paper addresses both issues. It first reviews some basic concepts of algorithmic complexity theory relevant to machine learning, and how the Solomonoff-Levin distribution (or universal prior) deals with the prior problem. The universal prior leads to a probabilistic method for finding "algorithmically simple" problem solutions with high generalization capability. The method is based on Levin complexity (a time-bounded generalization of Kolmogorov complexity) and inspired by Levin's optimal universal search algorithm. For a given problem, solution candidates are computed by efficient "self-sizing" programs that influence their own runtime and storage size. The probabilistic search algorithm finds the "good" programs (the ones quickly computing algorithmically probable solutions fitting the training data). Simulations focus on the task of discovering "algorithmically simple" neural networks with low Kolmogorov complexity and high generalization capability. It is demonstrated that the method, at least with certain toy problems where it is computationally feasible, can lead to generalization results unmatchable by previous neural net algorithms. Much remains do be done, however, to make large scale applications and "incremental learning" feasible. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #411005. 	[Title] GUESSING CAN OUTPERFORM MANY LONG TIME LAG ALGORITHMS [Abstract] Numerous recent papers focus on standard recurrent nets' problems with long time lags between relevant signals. Some propose rather sophisticated, alternative methods. We show: many problems used to test previous methods can be solved more quickly by random weight guessing. Please summarise all the given information.

Paper #214472 has the title `A Computer Scientist's View of Life, the Universe, and Everything  
` and the abstract `Is the universe computable? If so, it may be much cheaper in terms of information requirements to compute all computable universes instead of just ours. I apply basic concepts of Kolmogorov complexity theory to the set of possible universes, and chat about perceived and true randomness, life, generalization, and learning in a given universe. Assumptions. A long time ago, the Great Programmer wrote a program that runs all possible universes on His Big Computer. "Possible" means "computable": (1) Each universe evolves on a discrete time scale. (2) Any universe's state at a given time is describable by a finite number of bits. One of the many universes is ours, despite some who evolved in it and claim it is incomputable. Computable universes. Let T M denote an arbitrary universal Turing machine with unidirectional output tape. T M 's input and output symbols are "0", "1", and "," (comma). T M 's possible input programs can be ordered alphabetically: "" (empty program), "0", "1", ",", "00", "01", "0,", "10", "11", "1,", ",0", ",1", ",,", "000", etc. Let A k denote T M 's k-th program in this list. Its output will be a finite or infinite string over the alphabet f "0","1",","g. This sequence of bitstrings separated by commas will be interpreted as the evolution E k of universe U k . If E k includes at least one comma, then let U l k represents U k 's state at the l-th time step of E k (k; l 2 f1; 2; : : : ; g). E k is represented by the sequence U 1 k corresponds to U k 's big bang. Different algorithms may compute the same universe. Some universes are finite (those whose programs cease producing outputs at some point), others are not. I don't know about ours. TM not important. The choice of the Turing machine is not important. This is due to the compiler theorem: for each universal Turing machine C there exists a constant prefix C 2 f "0","1",","g fl such that for all possible programs p, C's output in response to program C p is identical to T M 's output in response to p. The prefix C is the compiler that compiles programs for T M into equivalent programs for C. k denote the l-th (possibly empty) bitstring before the l-th comma. U l
`.
Question: Which category should Paper #214472 be classified as ?
You can select one from ['Neural_Networks', 'Case_Based', 'Theory', 'Reinforcement_Learning', 'Probabilistic_Methods', 'Rule_Learning', 'Genetic_Algorithms'].


Paper #214472 has the title `A Computer Scientist's View of Life, the Universe, and Everything  
` and the abstract `Is the universe computable? If so, it may be much cheaper in terms of information requirements to compute all computable universes instead of just ours. I apply basic concepts of Kolmogorov complexity theory to the set of possible universes, and chat about perceived and true randomness, life, generalization, and learning in a given universe. Assumptions. A long time ago, the Great Programmer wrote a program that runs all possible universes on His Big Computer. "Possible" means "computable": (1) Each universe evolves on a discrete time scale. (2) Any universe's state at a given time is describable by a finite number of bits. One of the many universes is ours, despite some who evolved in it and claim it is incomputable. Computable universes. Let T M denote an arbitrary universal Turing machine with unidirectional output tape. T M 's input and output symbols are "0", "1", and "," (comma). T M 's possible input programs can be ordered alphabetically: "" (empty program), "0", "1", ",", "00", "01", "0,", "10", "11", "1,", ",0", ",1", ",,", "000", etc. Let A k denote T M 's k-th program in this list. Its output will be a finite or infinite string over the alphabet f "0","1",","g. This sequence of bitstrings separated by commas will be interpreted as the evolution E k of universe U k . If E k includes at least one comma, then let U l k represents U k 's state at the l-th time step of E k (k; l 2 f1; 2; : : : ; g). E k is represented by the sequence U 1 k corresponds to U k 's big bang. Different algorithms may compute the same universe. Some universes are finite (those whose programs cease producing outputs at some point), others are not. I don't know about ours. TM not important. The choice of the Turing machine is not important. This is due to the compiler theorem: for each universal Turing machine C there exists a constant prefix C 2 f "0","1",","g fl such that for all possible programs p, C's output in response to program C p is identical to T M 's output in response to p. The prefix C is the compiler that compiles programs for T M into equivalent programs for C. k denote the l-th (possibly empty) bitstring before the l-th comma. U l
`.
Question: Which category should Paper #214472 be classified as ?
You can select one from ['Neural_Networks', 'Case_Based', 'Theory', 'Reinforcement_Learning', 'Probabilistic_Methods', 'Rule_Learning', 'Genetic_Algorithms'].

This paper should be classified into: Reinforcement_Learning