Above are papers related to the following paper: [Paper_ID] Paper #144408. 	[Title] Neural net architectures for temporal sequence processing [Abstract] I present a general taxonomy of neural net architectures for processing time-varying patterns. This taxonomy subsumes many existing architectures in the literature, and points to several promising architectures that have yet to be examined. Any architecture that processes time-varying patterns requires two conceptually distinct components: a short-term memory that holds on to relevant past events and an associator that uses the short-term memory to classify or predict. My taxonomy is based on a characterization of short-term memory models along the dimensions of form, content, and adaptability. Experiments on predicting future values of a financial time series (US dollar-Swiss franc exchange rates) are presented using several alternative memory models. The results of these experiments serve as a baseline against which more sophisticated architectures can be compared. Neural networks have proven to be a promising alternative to traditional techniques for nonlinear temporal prediction tasks (e.g., Curtiss, Brandemuehl, & Kreider, 1992; Lapedes & Farber, 1987; Weigend, Huberman, & Rumelhart, 1992). However, temporal prediction is a particularly challenging problem because conventional neural net architectures and algorithms are not well suited for patterns that vary over time. The prototypical use of neural nets is in structural pattern recognition. In such a task, a collection of features|visual, semantic, or otherwise|is presented to a network and the network must categorize the input feature pattern as belonging to one or more classes. For example, a network might be trained to classify animal species based on a set of attributes describing living creatures such as "has tail", "lives in water", or "is carnivorous"; or a network could be trained to recognize visual patterns over a two-dimensional pixel array as a letter in fA; B; . . . ; Zg. In such tasks, the network is presented with all relevant information simultaneously. In contrast, temporal pattern recognition involves processing of patterns that evolve over time. The appropriate response at a particular point in time depends not only on the current input, but potentially all previous inputs. This is illustrated in Figure 1, which shows the basic framework for a temporal prediction problem. I assume that time is quantized into discrete steps, a sensible assumption because many time series of interest are intrinsically discrete, and continuous series can be sampled at a fixed interval. The input at time t is denoted x(t). For univariate series, this input Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #1125895. 	[Title] Packet Routing and Reinforcement Learning: Estimating Shortest Paths in Dynamic Graphs [Abstract]  Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #29708. 	[Title] Diffusion of Context and Credit Information in Markovian Models [Abstract] This paper studies the problem of ergodicity of transition probability matrices in Marko-vian models, such as hidden Markov models (HMMs), and how it makes very difficult the task of learning to represent long-term context for sequential data. This phenomenon hurts the forward propagation of long-term context information, as well as learning a hidden state representation to represent long-term context, which depends on propagating credit information backwards in time. Using results from Markov chain theory, we show that this problem of diffusion of context and credit is reduced when the transition probabilities approach 0 or 1, i.e., the transition probability matrices are sparse and the model essentially deterministic. The results found in this paper apply to learning approaches based on continuous optimization, such as gradient descent and the Baum-Welch algorithm. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #643597. 	[Title] A Nonparametric Bayesian Approach to Modelling Nonlinear Time Series [Abstract] The Bayesian multivariate adaptive regression spline (BMARS) methodology of Denison et al. (1997) is extended to cope with nonlinear time series and financial datasets. The nonlinear time series model is closely related to the adaptive spline threshold autoregressive (ASTAR) method of Lewis and Stevens (1991) while the financial models can be thought of as Bayesian versions of both the generalised and simple autoregressive conditional het-eroscadastic (GARCH and ARCH) models. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #35343. 	[Title] Backpropagation Convergence Via Deterministic Nonmonotone Perturbed Minimization [Abstract] The fundamental backpropagation (BP) algorithm for training artificial neural networks is cast as a deterministic nonmonotone perturbed gradient method . Under certain natural assumptions, such as the series of learning rates diverging while the series of their squares converging, it is established that every accumulation point of the online BP iterates is a stationary point of the BP error function. The results presented cover serial and parallel online BP, modified BP with a momentum term, and BP with weight decay. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #35863. 	[Title] Associative Reinforcement Learning: Functions in k-DNF [Abstract] An agent that must learn to act in the world by trial and error faces the reinforcement learning problem, which is quite different from standard concept learning. Although good algorithms exist for this problem in the general case, they are often quite inefficient and do not exhibit generalization. One strategy is to find restricted classes of action policies that can be learned more efficiently. This paper pursues that strategy by developing algorithms that can efficiently learn action maps that are expressible in k-DNF. The algorithms are compared with existing methods in empirical trials and are shown to have very good performance. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #31769. 	[Title] Representing and Learning Visual Schemas in Neural Networks for Scene Analysis [Abstract] Using scene analysis as the task, this research focuses on three fundamental problems in neural network systems: (1) limited processing resources, (2) representing schemas, and (3) learning schemas. The first problem arises because no practical neural network can process all the visual input simultaneously and efficiently. The solution is to process a small amount of the input in parallel, and successively focus on the other parts of the input. This strategy requires that the system maintains structured knowledge for describing and interpreting the gathered information. The system should also learn to represent structured knowledge from examples of objects and scenes. VISOR, the system described in this paper, consists of three main components. The Low-Level Visual Module (simulated using procedural programs) extracts featural and positional information from the visual input. The Schema Module encodes structured knowledge about possible objects, and provides top-down information for the Low-Level Visual Module to focus attention at different parts of the scene. The Response Module learns to associate the schema activation patterns with external responses. It enables the external environment to provide reinforcement feedback for the learning of schematic structures. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #27174. 	[Title] Mathematical Programming in Neural Networks [Abstract] This paper highlights the role of mathematical programming, particularly linear programming, in training neural networks. A neural network description is given in terms of separating planes in the input space that suggests the use of linear programming for determining these planes. A more standard description in terms of a mean square error in the output space is also given, which leads to the use of unconstrained minimization techniques for training a neural network. The linear programming approach is demonstrated by a brief description of a system for breast cancer diagnosis that has been in use for the last four years at a major medical facility. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #6184. 	[Title] Forward models: Supervised learning with a distal teacher [Abstract] Internal models of the environment have an important role to play in adaptive systems in general and are of particular importance for the supervised learning paradigm. In this paper we demonstrate that certain classical problems associated with the notion of the "teacher" in supervised learning can be solved by judicious use of learned internal models as components of the adaptive system. In particular, we show how supervised learning algorithms can be utilized in cases in which an unknown dynamical system intervenes between actions and desired outcomes. Our approach applies to any supervised learning algorithm that is capable of learning in multi-layer networks. *This paper is a revised version of MIT Center for Cognitive Science Occasional Paper #40. We wish to thank Michael Mozer, Andrew Barto, Robert Jacobs, Eric Loeb, and James McClelland for helpful comments on the manuscript. This project was supported in part by BRSG 2 S07 RR07047-23 awarded by the Biomedical Research Support Grant Program, Division of Research Resources, National Institutes of Health, by a grant from ATR Auditory and Visual Perception Research Laboratories, by a grant from Siemens Corporation, by a grant from the Human Frontier Science Program, and by grant N00014-90-J-1942 awarded by the Office of Naval Research. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #131117. 	[Title] Learning Sequential Tasks by Incrementally Adding Higher Orders [Abstract] An incremental, higher-order, non-recurrent network combines two properties found to be useful for learning sequential tasks: higher-order connections and incremental introduction of new units. The network adds higher orders when needed by adding new units that dynamically modify connection weights. Since the new units modify the weights at the next time-step with information from the previous step, temporal tasks can be learned without the use of feedback, thereby greatly simplifying training. Furthermore, a theoretically unlimited number of units can be added to reach into the arbitrarily distant past. Experiments with the Reber grammar have demonstrated speedups of two orders of magnitude over recurrent networks. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #1050679. 	[Title] Diplomarbeit A Genetic Algorithm for the Topological Optimization of Neural Networks [Abstract]  Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #1140289. 	[Title] LU TP 93-24 Predicting System Loads with Artificial Neural [Abstract] Networks Methods and Results from Abstract: We devise a feed-forward Artificial Neural Network (ANN) procedure for predicting utility loads and present the resulting predictions for two test problems given by "The Great Energy Predictor Shootout The First Building Data Analysis and Prediction Competition" [1]. Key ingredients in our approach are a method (ffi -test) for determining relevant inputs and the Multilayer Perceptron. These methods are briefly reviewed together with comments on alternative schemes like fitting to polynomials and the use of recurrent networks. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #948299. 	[Title] Visualizing High-Dimensional Structure with the Incremental Grid Growing Neural Network [Abstract]  Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #1154124. 	[Title]  [Abstract] Most connectionist modeling assumes noise-free inputs. This assumption is often violated. This paper introduces the idea of clearning, of simultaneously cleaning the data and learning the underlying structure. The cleaning step can be viewed as top-down processing (where the model modifies the data), and the learning step can be viewed as bottom-up processing (where the data modifies the model). Clearning is used in conjunction with standard pruning. This paper discusses the statistical foundation of clearning, gives an interpretation in terms of a mechanical model, describes how to obtain both point predictions and conditional densities for the output, and shows how the resulting model can be used to discover properties of the data otherwise not accessible (such as the signal-to-noise ratio of the inputs). This paper uses clearning to predict foreign exchange rates, a noisy time series problem with well-known benchmark performances. On the out-of-sample 1993-1994 test period, clearning obtains an annualized return on investment above 30%, significantly better than an otherwise identical network. The final ultra-sparse network with 36 remaining non-zero input-to-hidden weights (of the 1035 initial weights between 69 inputs and 15 hidden units) is very robust against overfitting. This small network also lends itself to interpretation. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #249421. 	[Title] PERCEPTION OF TIME AS PHASE: TOWARD AN ADAPTIVE-OSCILLATOR MODEL OF RHYTHMIC PATTERN PROCESSING 1 [Abstract]  Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #69198. 	[Title] LEARNING COMPLEX, EXTENDED SEQUENCES USING THE PRINCIPLE OF HISTORY COMPRESSION (Neural Computation, 4(2):234-242, 1992) [Abstract] Previous neural network learning algorithms for sequence processing are computationally expensive and perform poorly when it comes to long time lags. This paper first introduces a simple principle for reducing the descriptions of event sequences without loss of information. A consequence of this principle is that only unexpected inputs can be relevant. This insight leads to the construction of neural architectures that learn to `divide and conquer' by recursively decomposing sequences. I describe two architectures. The first functions as a self-organizing multi-level hierarchy of recurrent networks. The second, involving only two recurrent networks, tries to collapse a multi-level predictor hierarchy into a single recurrent net. Experiments show that the system can require less computation per time step and many fewer training sequences than conventional training algorithms for recurrent nets. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #96335. 	[Title] Geometry in Learning [Abstract] One of the fundamental problems in learning is identifying members of two different classes. For example, to diagnose cancer, one must learn to discriminate between benign and malignant tumors. Through examination of tumors with previously determined diagnosis, one learns some function for distinguishing the benign and malignant tumors. Then the acquired knowledge is used to diagnose new tumors. The perceptron is a simple biologically inspired model for this two-class learning problem. The perceptron is trained or constructed using examples from the two classes. Then the perceptron is used to classify new examples. We describe geometrically what a perceptron is capable of learning. Using duality, we develop a framework for investigating different methods of training a perceptron. Depending on how we define the "best" perceptron, different minimization problems are developed for training the perceptron. The effectiveness of these methods is evaluated empirically on four practical applications: breast cancer diagnosis, detection of heart disease, political voting habits, and sonar recognition. This paper does not assume prior knowledge of machine learning or pattern recognition. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #1154123. 	[Title]  [Abstract] Most connectionist modeling assumes noise-free inputs. This assumption is often violated. This paper introduces the idea of clearning, of simultaneously cleaning the data and learning the underlying structure. The cleaning step can be viewed as top-down processing (where the model modifies the data), and the learning step can be viewed as bottom-up processing (where the data modifies the model). Clearning is used in conjunction with standard pruning. This paper discusses the statistical foundation of clearning, gives an interpretation in terms of a mechanical model, describes how to obtain both point predictions and conditional densities for the output, and shows how the resulting model can be used to discover properties of the data otherwise not accessible (such as the signal-to-noise ratio of the inputs). This paper uses clearning to predict foreign exchange rates, a noisy time series problem with well-known benchmark performances. On the out-of-sample 1993-1994 test period, clearning obtains an annualized return on investment above 30%, significantly better than an otherwise identical network. The final ultra-sparse network with 36 remaining non-zero input-to-hidden weights (of the 1035 initial weights between 69 inputs and 15 hidden units) is very robust against overfitting. This small network also lends itself to interpretation. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #16461. 	[Title] On the Applicability of Neural Network and Machine Learning Methodologies to Natural Language Processing [Abstract] We examine the inductive inference of a complex grammar specifically, we consider the task of training a model to classify natural language sentences as grammatical or ungrammatical, thereby exhibiting the same kind of discriminatory power provided by the Principles and Parameters linguistic framework, or Government-and-Binding theory. We investigate the following models: feed-forward neural networks, Fransconi-Gori-Soda and Back-Tsoi locally recurrent networks, Elman, Narendra & Parthasarathy, and Williams & Zipser recurrent networks, Euclidean and edit-distance nearest-neighbors, simulated annealing, and decision trees. The feed-forward neural networks and non-neural network machine learning models are included primarily for comparison. We address the question: How can a neural network, with its distributed nature and gradient descent based iterative calculations, possess linguistic capability which is traditionally handled with symbolic computation and recursive processes? Initial simulations with all models were only partially successful by using a large temporal window as input. Models trained in this fashion did not learn the grammar to a significant degree. Attempts at training recurrent networks with small temporal input windows failed until we implemented several techniques aimed at improving the convergence of the gradient descent training algorithms. We discuss the theory and present an empirical study of a variety of models and learning algorithms which highlights behaviour not present when attempting to learn a simpler grammar. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #35922. 	[Title] Mean Field Theory for Sigmoid Belief Networks [Abstract] We develop a mean field theory for sigmoid belief networks based on ideas from statistical mechanics. Our mean field theory provides a tractable approximation to the true probability distribution in these networks; it also yields a lower bound on the likelihood of evidence. We demonstrate the utility of this framework on a benchmark problem in statistical pattern recognition|the classification of handwritten digits. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #1103960. 	[Title] Surgery [Abstract] Object localization has applications in many areas of engineering and science. The goal is to spatially locate an arbitrarily-shaped object. In many applications, it is desirable to minimize the number of measurements collected for this purpose, while ensuring sufficient localization accuracy. In surgery, for example, collecting a large number of localization measurements may either extend the time required to perform a surgical procedure, or increase the radiation dosage to which a patient is exposed. Localization accuracy is a function of the spatial distribution of discrete measurements over an object when measurement noise is present. In [Simon et al., 1995a], metrics were presented to evaluate the information available from a set of discrete object measurements. In this study, new approaches to the discrete point data selection problem are described. These include hillclimbing, genetic algorithms (GAs), and Population-Based Incremental Learning (PBIL). Extensions of the standard GA and PBIL methods, which employ multiple parallel populations, are explored. The results of extensive empirical testing are provided. The results suggest that a combination of PBIL and hillclimbing result in the best overall performance. A computer-assisted surgical system which incorporates some of the methods presented in this paper is currently being evaluated in cadaver trials. Evolution-Based Methods for Selecting Point Data Shumeet Baluja was supported by a National Science Foundation Graduate Student Fellowship and a Graduate Student Fellowship from the National Aeronautics and Space Administration, administered by the Lyndon B. Johnson Space Center, Houston, TX. David Simon was partially supported by a National Science Foundation National Challenge grant (award IRI-9422734). for Object Localization: Applications to Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #400473. 	[Title] Sequence Learning with Incremental Higher-Order Neural Networks [Abstract] An incremental, higher-order, non-recurrent neural-network combines two properties found to be useful for sequence learning in neural-networks: higher-order connections and the incremental introduction of new units. The incremental, higher-order neural-network adds higher orders when needed by adding new units that dynamically modify connection weights. The new units modify the weights at the next time-step with information from the previous step. Since a theoretically unlimited number of units can be added to the network, information from the arbitrarily distant past can be brought to bear on each prediction. Temporal tasks can thereby be learned without the use of feedback, in contrast to recurrent neural-networks. Because there are no recurrent connections, training is simple and fast. Experiments have demonstrated speedups of two orders of magnitude over recurrent networks. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #25181. 	[Title] Bilinear Separation of Two Sets in n-Space [Abstract] The NP-complete problem of determining whether two disjoint point sets in the n-dimensional real space R n can be separated by two planes is cast as a bilinear program, that is minimizing the scalar product of two linear functions on a polyhedral set. The bilinear program, which has a vertex solution, is processed by an iterative linear programming algorithm that terminates in a finite number of steps at a point satisfying a necessary optimality condition or at a global minimum. Encouraging computational experience on a number of test problems is reported. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #25184. 	[Title] Feature Selection via Mathematical Programming [Abstract] The problem of discriminating between two finite point sets in n-dimensional feature space by a separating plane that utilizes as few of the features as possible, is formulated as a mathematical program with a parametric objective function and linear constraints. The step function that appears in the objective function can be approximated by a sigmoid or by a concave exponential on the nonnegative real line, or it can be treated exactly by considering the equivalent linear program with equilibrium constraints (LPEC). Computational tests of these three approaches on publicly available real-world databases have been carried out and compared with an adaptation of the optimal brain damage (OBD) method for reducing neural network complexity. One feature selection algorithm via concave minimization (FSV) reduced cross-validation error on a cancer prognosis database by 35.4% while reducing problem features from 32 to 4. Feature selection is an important problem in machine learning [18, 15, 16, 17, 33]. In its basic form the problem consists of eliminating as many of the features in a given problem as possible, while still carrying out a preassigned task with acceptable accuracy. Having a minimal number of features often leads to better generalization and simpler models that can be more easily interpreted. In the present work, our task is to discriminate between two given sets in an n-dimensional feature space by using as few of the given features as possible. We shall formulate this problem as a mathematical program with a parametric objective function that will attempt to achieve this task by generating a separating plane in a feature space of as small a dimension as possible while minimizing the average distance of misclassified points to the plane. One of the computational experiments that we carried out on our feature selection procedure showed its effectiveness, not only in minimizing the number of features selected, but also in quickly recognizing and removing spurious random features that were introduced. Thus, on the Wisconsin Prognosis Breast Cancer WPBC database [36] with a feature space of 32 dimensions and 6 random features added, one of our algorithms FSV (11) immediately removed the 6 random features as well as 28 of the original features resulting in a separating plane in a 4-dimensional reduced feature space. By using tenfold cross-validation [35], separation error in the 4-dimensional space was reduced 35.4% from the corresponding error in the original problem space. (See Section 3 for details.) We note that mathematical programming approaches to the feature selection problem have been recently proposed in [4, 22]. Even though the approach of [4] is based on an LPEC formulation, both the LPEC and its method of solution are different from the ones used here. The polyhedral concave minimization approach of [22] is principally involved with theoretical considerations of one specific algorithm and no cross-validatory results are given. Other effective computational applications of mathematical programming to neural networks are given in [30, 26]. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #30817. 	[Title] Analysis of Linsker's application of Hebbian rules to Linear Networks [Abstract] Linsker has reported the development of structured receptive fields in simulations using a Hebb-type synaptic plasticity rule in a feed-forward linear network. The synapses develop under dynamics determined by a matrix that is closely related to the covariance matrix of input cell activities. We analyse the dynamics of the learning rule in terms of the eigenvectors of this matrix. These eigenvectors represent independently evolving weight structures. Some general theorems are presented regarding the properties of these eigenvectors and their eigenvalues. For a general covariance matrix four principal parameter regimes are predicted. We concentrate on the gaussian covariances at layer B ! C of Linsker's network. Analytic and numerical solutions for the eigenvectors at this layer are presented. Three eigenvectors dominate the dynamics: a DC eigenvector, in which all synapses have the same sign; a bi-lobed, oriented eigenvector; and a circularly symmetric, centre-surround eigenvector. Analysis of the circumstances in which each of these vectors dominates yields an explanation of the emergence of centre-surround structures and symmetry-breaking bi-lobed structures. Criteria are developed estimating the boundary of the parameter regime in which centre-surround structures emerge. The application of our analysis to Linsker's higher layers, at which the covariance functions were oscillatory, is briefly discussed. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #25702. 	[Title] Nonlinear Prediction of Chaotic Time Series Using Support Vector Machines [Abstract] A novel method for regression has been recently proposed by V. Vapnik et al. [8, 9]. The technique, called Support Vector Machine (SVM), is very well founded from the mathematical point of view and seems to provide a new insight in function approximation. We implemented the SVM and tested it on the same data base of chaotic time series that was used in [1] to compare the performances of different approximation techniques, including polynomial and rational approximation, local polynomial techniques, Radial Basis Functions, and Neural Networks. The SVM performs better than the approaches presented in [1]. We also study, for a particular time series, the variability in performance with respect to the few free parameters of SVM. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #1154169. 	[Title]  [Abstract] 11] M.H. Overmars. A random approach to motion planning. Technical Report RUU-CS-92-32, Department of Computer Science, Utrecht University, October 1992. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #396412. 	[Title] Boltzmann Machine learning using mean field theory and linear response correction [Abstract] We present a new approximate learning algorithm for Boltzmann Machines, using a systematic expansion of the Gibbs free energy to second order in the weights. The linear response correction to the correlations is given by the Hessian of the Gibbs free energy. The computational complexity of the algorithm is cubic in the number of neurons. We compare the performance of the exact BM learning algorithm with first order (Weiss) mean field theory and second order (TAP) mean field theory. The learning task consists of a fully connected Ising spin glass model on 10 neurons. We conclude that 1) the method works well for paramagnetic problems 2) the TAP correction gives a significant improvement over the Weiss mean field theory, both for paramagnetic and spin glass problems and 3) that the inclusion of diagonal weights improves the Weiss approximation for paramagnetic problems, but not for spin glass problems. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #1131647. 	[Title] Connectionist Modeling of the Fast Mapping Phenomenon [Abstract]  Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #33412. 	[Title] Learning to Predict Reading Frames in E. coli DNA Sequences [Abstract] Two fundamental problems in analyzing DNA sequences are (1) locating the regions of a DNA sequence that encode proteins, and (2) determining the reading frame for each region. We investigate using artificial neural networks (ANNs) to find coding regions, determine reading frames, and detect frameshift errors in E. coli DNA sequences. We describe our adaptation of the approach used by Uberbacher and Mural to identify coding regions in human DNA, and we compare the performance of ANNs to several conventional methods for predicting reading frames. Our experiments demonstrate that ANNs can outperform these conventional approaches. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #1106568. 	[Title] Local Feedforward Networks [Abstract] Although feedforward neural networks are well suited to function approximation, in some applications networks experience problems when learning a desired function. One problem is interference which occurs when learning in one area of the input space causes unlearning in another area. Networks that are less susceptible to interference are referred to as spatially local networks. To understand these properties, a theoretical framework, consisting of a measure of interference and a measure of network localization, is developed that incorporates not only the network weights and architecture but also the learning algorithm. Using this framework to analyze sigmoidal multi-layer perceptron (MLP) networks that employ the back-prop learning algorithm, we address a familiar misconception that sigmoidal networks are inherently non-local by demonstrating that given a sufficiently large number of adjustable parameters, sigmoidal MLPs can be made arbitrarily local while retaining the ability to represent any continuous function on a compact domain. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #189577. 	[Title] Chain graphs for learning [Abstract]  Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #1126037. 	[Title] Parallel Environments for Implementing Neural Networks [Abstract] As artificial neural networks (ANNs) gain popularity in a variety of application domains, it is critical that these models run fast and generate results in real time. Although a number of implementations of neural networks are available on sequential machines, most of these implementations require an inordinate amount of time to train or run ANNs, especially when the ANN models are large. One approach for speeding up the implementation of ANNs is to implement them on parallel machines. This paper surveys the area of parallel environments for the implementations of ANNs, and prescribes desired characteristics to look for in such implementations. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #152219. 	[Title] A Connectionist Symbol Manipulator That Discovers the Structure of Context-Free Languages [Abstract] We present a neural net architecture that can discover hierarchical and recursive structure in symbol strings. To detect structure at multiple levels, the architecture has the capability of reducing symbols substrings to single symbols, and makes use of an external stack memory. In terms of formal languages, the architecture can learn to parse strings in an LR(0) context-free grammar. Given training sets of positive and negative exemplars, the architecture has been trained to recognize many different grammars. The architecture has only one layer of modifiable weights, allowing for a Many cognitive domains involve complex sequences that contain hierarchical or recursive structure, e.g., music, natural language parsing, event perception. To illustrate, "the spider that ate the hairy fly" is a noun phrase containing the embedded noun phrase "the hairy fly." Understanding such multilevel structures requires forming reduced descriptions (Hinton, 1988) in which a string of symbols or states ("the hairy fly") is reduced to a single symbolic entity (a noun phrase). We present a neural net architecture that learns to encode the structure of symbol strings via such reduction transformations. The difficult problem of extracting multilevel structure from complex, extended sequences has been studied by Mozer (1992), Ring (1993), Rohwer (1990), and Schmidhuber (1992), among others. While these previous efforts have made some straightforward interpretation of its behavior. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #3229. 	[Title] Book Review  Introduction to the Theory of Neural Computation Reviewed by: 2 [Abstract] Neural computation, also called connectionism, parallel distributed processing, neural network modeling or brain-style computation, has grown rapidly in the last decade. Despite this explosion, and ultimately because of impressive applications, there has been a dire need for a concise introduction from a theoretical perspective, analyzing the strengths and weaknesses of connectionist approaches and establishing links to other disciplines, such as statistics or control theory. The Introduction to the Theory of Neural Computation by Hertz, Krogh and Palmer (subsequently referred to as HKP) is written from the perspective of physics, the home discipline of the authors. The book fulfills its mission as an introduction for neural network novices, provided that they have some background in calculus, linear algebra, and statistics. It covers a number of models that are often viewed as disjoint. Critical analyses and fruitful comparisons between these models Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #1152673. 	[Title] Misclassification Minimization [Abstract] The problem of minimizing the number of misclassified points by a plane, attempting to separate two point sets with intersecting convex hulls in n-dimensional real space, is formulated as a linear program with equilibrium constraints (LPEC). This general LPEC can be converted to an exact penalty problem with a quadratic objective and linear constraints. A Frank-Wolfe-type algorithm is proposed for the penalty problem that terminates at a stationary point or a global solution. Novel aspects of the approach include: (i) A linear complementarity formulation of the step function that "counts" misclassifications, (ii) Exact penalty formulation without boundedness, nondegeneracy or constraint qualification assumptions, (iii) An exact solution extraction from the sequence of minimizers of the penalty function for a finite value of the penalty parameter for the general LPEC and an explicitly exact solution for the LPEC with uncoupled constraints, and (iv) A parametric quadratic programming formulation of the LPEC associated with the misclassification minimization problem. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #1117348. 	[Title] MIXED MEMORY MARKOV MODELS FOR TIME SERIES ANALYSIS [Abstract] This paper presents a method for analyzing coupled time series using Markov models in a domain where the state space is immense. To make the parameter estimation tractable, the large state space is represented as the Cartesian product of smaller state spaces, a paradigm known as factorial Markov models. The transition matrix for this model is represented as a mixture of the transition matrices of the underlying dynamical processes. This formulation is know as mixed memory Markov models. Using this framework, we analyze the daily exchange rates for five currencies - British pound, Canadian dollar, Deutsch mark, Japanese yen, and Swiss franc as measured against the U.S. dollar. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #82087. 	[Title] Fast Probabilistic Modeling for Combinatorial Optimization [Abstract] Probabilistic models have recently been utilized for the optimization of large combinatorial search problems. However, complex probabilistic models that attempt to capture inter-parameter dependencies can have prohibitive computational costs. The algorithm presented in this paper, termed COMIT, provides a method for using probabilistic models in conjunction with fast search techniques. We show how COMIT can be used with two very different fast search algorithms: hillclimbing and Population-based incremental learning (PBIL). The resulting algorithms maintain many of the benefits of probabilistic modeling, with far less computational expense. Extensive empirical results are provided; COMIT has been successfully applied to jobshop scheduling, traveling salesman, and knapsack problems. This paper also presents a review of probabilistic modeling for combi natorial optimization. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #1153195. 	[Title] A `SELF-REFERENTIAL' WEIGHT MATRIX [Abstract] Weight modifications in traditional neural nets are computed by hard-wired algorithms. Without exception, all previous weight change algorithms have many specific limitations. Is it (in principle) possible to overcome limitations of hard-wired algorithms by allowing neural nets to run and improve their own weight change algorithms? This paper constructively demonstrates that the answer (in principle) is `yes'. I derive an initial gradient-based sequence learning algorithm for a `self-referential' recurrent network that can `speak' about its own weight matrix in terms of activations. It uses some of its input and output units for observing its own errors and for explicitly analyzing and modifying its own weight matrix, including those parts of the weight matrix responsible for analyzing and modifying the weight matrix. The result is the first `introspective' neural net with explicit potential control over all of its own adaptive parameters. A disadvantage of the algorithm is its high computational complexity per time step which is independent of the sequence length and equals O(n conn logn conn ), where n conn is the number of connections. Another disadvantage is the high number of local minima of the unusually complex error surface. The purpose of this paper, however, is not to come up with the most efficient `introspective' or `self-referential' weight change algorithm, but to show that such algorithms are possible at all. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #82098. 	[Title] Hill Climbing with Learning (An Abstraction of Genetic Algorithm) [Abstract] Simple modification of standard hill climbing optimization algorithm by taking into account learning features is discussed. Basic concept of this approach is the socalled probability vector, its single entries determine probabilities of appearance of '1' entries in n-bit vectors. This vector is used for the random generation of n-bit vectors that form a neighborhood (specified by the given probability vector). Within the neighborhood a few best solutions (with smallest functional values of a minimized function) are recorded. The feature of learning is introduced here so that the probability vector is updated by a formal analogue of Hebbian learning rule, well-known in the theory of artificial neural networks. The process is repeated until the probability vector entries are close either to zero or to one. The resulting probability vector unambiguously determines an n-bit vector which may be interpreted as an optimal solution of the given optimization task. Resemblance with genetic algorithms is discussed. Effectiveness of the proposed method is illustrated by an example of looking for global minima of a highly multimodal function. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #1112767. 	[Title] Alternative Discrete-Time Operators and Their Application to Nonlinear Models [Abstract]  Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #40131. 	[Title] A comparison of neural net and conventional techniques for lighting control [Abstract] We compare two techniques for lighting control in an actual room equipped with seven banks of lights and photoresistors to detect the lighting level at four sensing points. Each bank of lights can be independently set to one of sixteen intensity levels. The task is to determine the device intensity levels that achieve a particular configuration of sensor readings. One technique we explored uses a neural network to approximate the mapping between sensor readings and device intensity levels. The other technique we examined uses a conventional feedback control loop. The neural network approach appears superior both in that it does not require experimentation on the fly (and hence fluctuating light intensity levels during settling, and lengthy settling times) and in that it can deal with complex interactions that conventional control techniques do not handle well. This comparison was performed as part of the "Adaptive House" project, which is described briefly. Further directions for control in the Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #1154251. 	[Title] Misclassification Minimization [Abstract] The problem of minimizing the number of misclassified points by a plane, attempting to separate two point sets with intersecting convex hulls in n-dimensional real space, is formulated as a linear program with equilibrium constraints (LPEC). This general LPEC can be converted to an exact penalty problem with a quadratic objective and linear constraints. A Frank-Wolfe-type algorithm is proposed for the penalty problem that terminates at a stationary point or a global solution. Novel aspects of the approach include: (i) A linear complementarity formulation of the step function that "counts" misclassifications, (ii) Exact penalty formulation without boundedness, nondegeneracy or constraint qualification assumptions, (iii) An exact solution extraction from the sequence of minimizers of the penalty function for a finite value of the penalty parameter for the general LPEC and an explicitly exact solution for the LPEC with uncoupled constraints, and (iv) A parametric quadratic programming formulation of the LPEC associated with the misclassification minimization problem. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #157401. 	[Title] Convergence-Zone Episodic Memory: Analysis and Simulations [Abstract] Human episodic memory provides a seemingly unlimited storage for everyday experiences, and a retrieval system that allows us to access the experiences with partial activation of their components. The system is believed to consist of a fast, temporary storage in the hippocampus, and a slow, long-term storage within the neocortex. This paper presents a neural network model of the hippocampal episodic memory inspired by Damasio's idea of Convergence Zones. The model consists of a layer of perceptual feature maps and a binding layer. A perceptual feature pattern is coarse coded in the binding layer, and stored on the weights between layers. A partial activation of the stored features activates the binding pattern, which in turn reactivates the entire stored pattern. For many configurations of the model, a theoretical lower bound for the memory capacity can be derived, and it can be an order of magnitude or higher than the number of all units in the model, and several orders of magnitude higher than the number of binding-layer units. Computational simulations further indicate that the average capacity is an order of magnitude larger than the theoretical lower bound, and making the connectivity between layers sparser causes an even further increase in capacity. Simulations also show that if more descriptive binding patterns are used, the errors tend to be more plausible (patterns are confused with other similar patterns), with a slight cost in capacity. The convergence-zone episodic memory therefore accounts for the immediate storage and associative retrieval capability and large capacity of the hippocampal memory, and shows why the memory encoding areas can be much smaller than the perceptual maps, consist of rather coarse computational units, and be only sparsely connected to the perceptual maps. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #1125082. 	[Title] Parallel Search for Neural Network  Under the guidance of [Abstract]  Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #1131748. 	[Title] Figure 1: The architecture of a Kohonen network. Each input neuron is fully connected with [Abstract]  Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #567018. 	[Title] Representing Rhythmic Patterns in a Network of Oscillators [Abstract] This paper describes an evolving computational model of the perception and production of simple rhythmic patterns. The model consists of a network of oscillators of different resting frequencies which couple with input patterns and with each other. Oscillators whose frequencies match periodicities in the input tend to become activated. Metrical structure is represented explicitly in the network in the form of clusters of oscillators whose frequencies and phase angles are constrained to maintain the harmonic relationships that characterize meter. Rests in rhythmic patterns are represented by explicit rest oscillators in the network, which become activated when an expected beat in the pattern fails to appear. The model makes predictions about the relative difficulty of The nested periodicity that defines musical, and probably also linguistic, meter appears to be fundamental to the way in which people perceive and produce patterns in time. Meter by itself, however, is not sufficient to describe patterns which are interesting or memorable because of how they deviate from the metrical hierarchy. The simplest deviations are rests or gaps where one or more levels in the hierarchy would normally have a beat. When beats are removed at regular intervals which match the period of some level of the metrical hierarchy, we have what we will call a simple rhythmic pattern. Figure 1 shows an example of a simple rhythmic pattern. Below it is a grid representation of the meter which is behind the pattern. patterns and the effect of deviations from periodicity in the input. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #1108209. 	[Title] Evolutionary Learning of the Crossover Operator [Abstract] 1 Abstract Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #949511. 	[Title] Application of Neural Networks for the Classification of Diffuse Liver Disease by Quantitative Echography [Abstract] Three different methods were investigated to determine their ability to detect and classify various categories of diffuse liver disease. A statistical method, i.e., discriminant analysis, a supervised neural network called backpropagation and a nonsupervised, self-organizing feature map were examined. The investigation was performed on the basis of a previously selected set of acoustic and image texture parameters. The limited number of patients was successfully extended by generating additional but independent data with identical statistical properties. The generated data were used for training and test sets. The final test was made with the original patient data as a validation set. It is concluded that neural networks are an attractive alternative to traditional statistical techniques when dealing with medical detection and classification tasks. Moreover, the use of generated data for training the networks and the discriminant classifier has been shown to be justified and profitable. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #90888. 	[Title] GAL: Networks that grow when they learn and shrink when they forget [Abstract] Learning when limited to modification of some parameters has a limited scope; the capability to modify the system structure is also needed to get a wider range of the learnable. In the case of artificial neural networks, learning by iterative adjustment of synaptic weights can only succeed if the network designer predefines an appropriate network structure, i.e., number of hidden layers, units, and the size and shape of their receptive and projective fields. This paper advocates the view that the network structure should not, as usually done, be determined by trial-and-error but should be computed by the learning algorithm. Incremental learning algorithms can modify the network structure by addition and/or removal of units and/or links. A survey of current connectionist literature is given on this line of thought. "Grow and Learn" (GAL) is a new algorithm that learns an association at one-shot due to being incremental and using a local representation. During the so-called "sleep" phase, units that were previously stored but which are no longer necessary due to recent modifications are removed to minimize network complexity. The incrementally constructed network can later be finetuned off-line to improve performance. Another method proposed that greatly increases recognition accuracy is to train a number of networks and vote over their responses. The algorithm and its variants are tested on recognition of handwritten numerals and seem promising especially in terms of learning speed. This makes the algorithm attractive for on-line learning tasks, e.g., in robotics. The biological plausibility of incremental learning is also discussed briefly. Earlier part of this work was realized at the Laboratoire de Microinformatique of Ecole Polytechnique Federale de Lausanne and was supported by the Fonds National Suisse de la Recherche Scientifique. Later part was realized at and supported by the International Computer Science Institute. A number of people helped by guiding, stimulating discussions or questions: Subutai Ahmad, Peter Clarke, Jerry Feldman, Christian Jutten, Pierre Marchal, Jean Daniel Nicoud, Steve Omohondro and Leon Personnaz. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #390922. 	[Title] A MEAN FIELD LEARNING ALGORITHM FOR UNSUPERVISED NEURAL NETWORKS [Abstract] We introduce a learning algorithm for unsupervised neural networks based on ideas from statistical mechanics. The algorithm is derived from a mean field approximation for large, layered sigmoid belief networks. We show how to (approximately) infer the statistics of these networks without resort to sampling. This is done by solving the mean field equations, which relate the statistics of each unit to those of its Markov blanket. Using these statistics as target values, the weights in the network are adapted by a local delta rule. We evaluate the strengths and weaknesses of these networks for problems in statistical pattern recognition. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #1122574. 	[Title] Tau Net: A Neural Network for Modeling Temporal Variability [Abstract] The ability to handle temporal variation is important when dealing with real-world dynamic signals. In many applications, inputs do not come in as fixed-rate sequences, but rather as signals with time scales that can vary from one instance to the next; thus, modeling dynamic signals requires not only the ability to recognize sequences but also the ability to handle temporal changes in the signal. This paper discusses "Tau Net," a neural network for modeling dynamic signals, and its application to speech. In Tau Net, sequence learning is accomplished using a combination of prediction, recurrence and time-delay connections. Temporal variability is modeled by having adaptable time constants in the network, which are adjusted with respect to the prediction error. Adapting the time constants changes the time scale of the network, and the adapted value of the network's time constant provides a measure of temporal variation in the signal. Tau Net has been applied to several simple signals: sets of sine waves differing in frequency and in phase [2], a multidimensional signal representing the walking gait of children [3], and the energy contour of a simple speech utterance [11]. Tau Net has also been shown to work on a voicing distinction task using synthetic speech data [12]. In this paper, Tau Net is applied to two speaker-independent tasks, vowel recognition (of f/ae/,/iy/,/ux/g) and consonant recognition (of f/p/,/t/,/k/g) using speech data taken from the TIMIT database. It is shown that Tau Nets, trained on medium-rate tokens, achieved about the same performance as networks without time constants trained on tokens at all rates, and performed better than networks without time constants trained on medium-rate tokens. Our results demonstrate Tau Net's ability to identify vowels and consonants at variable speech rates by extrapolating to rates not represented in the training set. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #1123087. 	[Title] Noisy Time Series Prediction using Symbolic Representation and Recurrent Neural Network Grammatical Inference [Abstract] Financial forecasting is an example of a signal processing problem which is challenging due to small sample sizes, high noise, non-stationarity, and non-linearity. Neural networks have been very successful in a number of signal processing applications. We discuss fundamental limitations and inherent difficulties when using neural networks for the processing of high noise, small sample size signals. We introduce a new intelligent signal processing method which addresses the difficulties. The method uses conversion into a symbolic representation with a self-organizing map, and grammatical inference with recurrent neural networks. We apply the method to the prediction of daily foreign exchange rates, addressing difficulties with non-stationarity, overfitting, and unequal a priori class probabilities, and we find significant predictability in comprehensive experiments covering 5 different foreign exchange rates. The method correctly predicts the direction of change for the next day with an error rate of 47.1%. The error rate reduces to around 40% when rejecting examples where the system has low confidence in its prediction. The symbolic representation aids the extraction of symbolic knowledge from the recurrent neural networks in the form of deterministic finite state automata. These automata explain the operation of the system and are often relatively simple. Rules related to well known behavior such as trend following and mean reversal are extracted. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #447250. 	[Title] Introduction to Radial Basis Function Networks [Abstract] This document is an introduction to radial basis function (RBF) networks, a type of artificial neural network for application to problems of supervised learning (e.g. regression, classification and time series prediction). It is available in either PostScript or hyper-text 2 . Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #1152277. 	[Title] Misclassification Minimization [Abstract] The problem of minimizing the number of misclassified points by a plane, attempting to separate two point sets with intersecting convex hulls in n-dimensional real space, is formulated as a linear program with equilibrium constraints (LPEC). This general LPEC can be converted to an exact penalty problem with a quadratic objective and linear constraints. A Frank-Wolfe-type algorithm is proposed for the penalty problem that terminates at a stationary point or a global solution. Novel aspects of the approach include: (i) A linear complementarity formulation of the step function that "counts" misclassifications, (ii) Exact penalty formulation without boundedness, nondegeneracy or constraint qualification assumptions, (iii) An exact solution extraction from the sequence of minimizers of the penalty function for a finite value of the penalty parameter for the general LPEC and an explicitly exact solution for the LPEC with uncoupled constraints, and (iv) A parametric quadratic programming formulation of the LPEC associated with the misclassification minimization problem. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #231198. 	[Title] A FIXED SIZE STORAGE O(n 3 TIME COMPLEXITY LEARNING ALGORITHM FOR FULLY RECURRENT CONTINUALLY RUNNING [Abstract] The RTRL algorithm for fully recurrent continually running networks (Robinson and Fallside, 1987)(Williams and Zipser, 1989) requires O(n 4 ) computations per time step, where n is the number of non-input units. I describe a method suited for on-line learning which computes exactly the same gradient and requires fixed-size storage of the same order but has an average time complexity 1 per time step of O(n 3 ). Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #444191. 	[Title] Automatic Smoothing Spline Projection Pursuit  Automatic Smoothing Spline Projection Pursuit. [Abstract] kj = 1. The standard PPR algorithm of Friedman and Stuet-zle (1981) estimates the smooth functions f j using the supersmoother nonparametric scatterplot smoother. Friedman's algorithm constructs a model with M max linear combinations, then prunes back to a simpler model of size M M max , where M and M max are specified by the user. This paper discusses an alternative algorithm in which the smooth functions are estimated using smoothing splines. The direction coefficients ff j , the amount of smoothing in each direction, and the number of terms M and M max are determined to optimize a single generalized cross-validation measure. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #118559. 	[Title] On-Line Adaptation of a Signal Predistorter through Dual Reinforcement Learning [Abstract] Several researchers have demonstrated how neural networks can be trained to compensate for nonlinear signal distortion in e.g. digital satellite communications systems. These networks, however, require that both the original signal and its distorted version are known. Therefore, they have to be trained off-line, and they cannot adapt to changing channel characteristics. In this paper, a novel dual reinforcement learning approach is proposed that can adapt on-line while the system is performing. Assuming that the channel characteristics are the same in both directions, two predistorters at each end of the communication channel co-adapt using the output of the other predistorter to determine their own reinforcement. Using the common Volterra Series model to simulate the channel, the system is shown to successfully learn to compensate for distortions up to 30%, which is significantly higher than what might be expected in an actual channel. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #52515. 	[Title] Unsupervised learning of distributions on binary vectors using two layer networks [Abstract] We present a distribution model for binary vectors, called the influence combination model and show how this model can be used as the basis for unsupervised learning algorithms for feature selection. The model is closely related to the Harmonium model defined by Smolensky [RM86][Ch.6]. In the first part of the paper we analyze properties of this distribution representation scheme. We show that arbitrary distributions of binary vectors can be approximated by the combination model. We show how the weight vectors in the model can be interpreted as high order correlation patterns among the input bits. We compare the combination model with the mixture model and with principle component analysis. In the second part of the paper we present two algorithms for learning the combination model from examples. The first algorithm is based on gradient ascent. Here we give a closed form for this gradient that is significantly easier to compute than the corresponding gradient for the general Boltzmann machine. The second learning algorithm is a greedy method that creates the hidden units and computes their weights one at a time. This method is a variant of projection pursuit density estimation. In the third part of the paper we give experimental results for these learning methods on synthetic data and on natural data of handwritten digit images. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #29492. 	[Title] Induction of Multiscale Temporal Structure [Abstract] Learning structure in temporally-extended sequences is a difficult computational problem because only a fraction of the relevant information is available at any instant. Although variants of back propagation can in principle be used to find structure in sequences, in practice they are not sufficiently powerful to discover arbitrary contingencies, especially those spanning long temporal intervals or involving high order statistics. For example, in designing a connectionist network for music composition, we have encountered the problem that the net is able to learn musical structure that occurs locally in time|e.g., relations among notes within a musical phrase|but not structure that occurs over longer time periods|e.g., relations among phrases. To address this problem, we require a means of constructing a reduced description of the sequence that makes global aspects more explicit or more readily detectable. I propose to achieve this using hidden units that operate with different time constants. Simulation experiments indicate that slower time-scale hidden units are able to pick up global structure, structure that simply can not be learned by standard Many patterns in the world are intrinsically temporal, e.g., speech, music, the unfolding of events. Recurrent neural net architectures have been devised to accommodate time-varying sequences. For example, the architecture shown in Figure 1 can map a sequence of inputs to a sequence of outputs. Learning structure in temporally-extended sequences is a difficult computational problem because the input pattern may not contain all the task-relevant information at any instant. Thus, back propagation. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #219446. 	[Title] RESONANCE AND THE PERCEPTION OF MUSICAL METER [Abstract] Many connectionist approaches to musical expectancy and music composition let the question of What next? overshadow the equally important question of When next?. One cannot escape the latter question, one of temporal structure, when considering the perception of musical meter. We view the perception of metrical structure as a dynamic process where the temporal organization of external musical events synchronizes, or entrains, a listeners internal processing mechanisms. This article introduces a novel connectionist unit, based upon a mathematical model of entrainment, capable of phase and frequency-locking to periodic components of incoming rhythmic patterns. Networks of these units can self-organize temporally structured responses to rhythmic patterns. The resulting network behavior embodies the perception of metrical structure. The article concludes with a discussion of the implications of our approach for theories of metrical structure and musical expectancy. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #1105718. 	[Title] Computational Models of Sensorimotor Integration  Computational Maps and Motor Control. [Abstract] The sensorimotor integration system can be viewed as an observer attempting to estimate its own state and the state of the environment by integrating multiple sources of information. We describe a computational framework capturing this notion, and some specific models of integration and adaptation that result from it. Psychophysical results from two sensorimotor systems, subserving the integration and adaptation of visuo-auditory maps, and estimation of the state of the hand during arm movements, are presented and analyzed within this framework. These results suggest that: (1) Spatial information from visual and auditory systems is integrated so as to reduce the variance in localization. (2) The effects of a remapping in the relation between visual and auditory space can be predicted from a simple learning rule. (3) The temporal propagation of errors in estimating the hand's state is captured by a linear dynamic observer, providing evidence for the existence of an internal model which simulates the dynamic behavior of the arm. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #672064. 	[Title] Direct Multi-Step Time Series Prediction Using TD() [Abstract] This paper explores the application of Temporal Difference (TD) learning (Sutton, 1988) to forecasting the behavior of dynamical systems with real-valued outputs (as opposed to game-like situations). The performance of TD learning in comparison to standard supervised learning depends on the amount of noise present in the data. In this paper, we use a deterministic chaotic time series from a low-noise laser. For the task of direct five-step ahead predictions, our experiments show that standard supervised learning is better than TD learning. The TD algorithm can be viewed as linking adjacent predictions. A similar effect can be obtained by sharing the internal representation in the network. We thus compare two architectures for both paradigms: the first architecture (separate hidden units) consists of individual networks for each of the five direct multi-step prediction tasks, the second (shared hidden units) has a single (larger) hidden layer that finds a representation from which all five predictions for the next five steps are generated. For this data set we do not find any significant difference between the two architectures. fl http://www.cs.colorado.edu/~andreas/Home.html. This paper is available as ftp://ftp.cs.colorado.edu/pub/Time-Series/MyPapers/kazlas.weigend nips7.ps.Z Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #49482. 	[Title] Topography And Ocular Dominance: A Model Exploring Positive Correlations [Abstract] The map from eye to brain in vertebrates is topographic, i.e. neighbouring points in the eye map to neighbouring points in the brain. In addition, when two eyes innervate the same target structure, the two sets of fibres segregate to form ocular dominance stripes. Experimental evidence from the frog and goldfish suggests that these two phenomena may be subserved by the same mechanisms. We present a computational model that addresses the formation of both topography and ocular dominance. The model is based on a form of competitive learning with subtractive enforcement of a weight normalization rule. Inputs to the model are distributed patterns of activity presented simultaneously in both eyes. An important aspect of this model is that ocular dominance segregation can occur when the two eyes are positively correlated, whereas previous models have tended to assume zero or negative correlations between the eyes. This allows investigation of the dependence of the pattern of stripes on the degree of correlation between the eyes: we find that increasing correlation leads to narrower stripes. Experiments are suggested to test this prediction. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #919885. 	[Title] Product Unit Learning constructive algorithm is then introduced which adds product units to a network [Abstract] Product units provide a method of automatically learning the higher-order input combinations required for the efficient synthesis of Boolean logic functions by neural networks. Product units also have a higher information capacity than sigmoidal networks. However, this activation function has not received much attention in the literature. A possible reason for this is that one encounters some problems when using standard backpropagation to train networks containing these units. This report examines these problems, and evaluates the performance of three training algorithms on networks of this type. Empirical results indicate that the error surface of networks containing product units have more local minima than corresponding networks with summation units. For this reason, a combination of local and global training algorithms were found to provide the most reliable convergence. We then investigate how `hints' can be added to the training algorithm. By extracting a common frequency from the input weights, and training this frequency separately, we show that convergence can be accelerated. In order to compare their performance with other transfer functions, product units were implemented as candidate units in the Cascade Correlation (CC) [13] system. Using these candidate units resulted in smaller networks which trained faster than when the any of the standard (three sigmoidal types and one Gaussian) transfer functions were used. This superiority was confirmed when a pool of candidate units of four different nonlinear activation functions were used, which have to compete for addition to the network. Extensive simulations showed that for the problem of implementing random Boolean logic functions, product units are always chosen above any of the other transfer functions. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #1108834. 	[Title] ce: [39] <author> Yoda, M. </author> <year> (1994). </year> <title> Predicting the Tokyo stock market. </title> <editor> In Deboeck, G.J. (Ed.) </editor> <year> (1994). </year> <title> Trading on the Edge. </title> <address> New York: </address> <publisher> Wiley., </publisher> <pages> 66-79. </pages> <institution> VITA Graduate School Southern Illinois University Daniel Nikolaev Nikovski Date of Birth: </institution> <address> April 13, 1969 606 West College Street, Apt.4, Rm. 6, Carbondale, Illinois 62901 150 Hristo Botev Boulevard, Apt. </address> <month> 54, </month> <title> 4004 Plovdiv, Bulgaria Technical University - Sofia, Bulgaria Engineer of Computer Systems and Control Thesis Title: Adaptive Computation Techniques for Time Series Analysis Major Professor: </title> <journal> Dr. Mehdi Zargham </journal> [Abstract]  Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #100197. 	[Title] Neural Networks and Statistical Models Proceedings of the Nineteenth Annual SAS Users Group International Conference, [Abstract]  Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #1112426. 	[Title] REDUCED MEMORY REPRESENTATIONS FOR MUSIC [Abstract] We address the problem of musical variation (identification of different musical sequences as variations) and its implications for mental representations of music. According to reductionist theories, listeners judge the structural importance of musical events while forming mental representations. These judgments may result from the production of reduced memory representations that retain only the musical gist. In a study of improvised music performance, pianists produced variations on melodies. Analyses of the musical events retained across variations provided support for the reductionist account of structural importance. A neural network trained to produce reduced memory representations for the same melodies represented structurally important events more efficiently than others. Agreement among the musicians' improvisations, the network model, and music-theoretic predictions suggest that perceived constancy across musical variation is a natural result of a reductionist mechanism for producing memory representations. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #7022. 	[Title] Transfer in a Connectionist Model of the Acquisition of Morphology [Abstract] The morphological systems of natural languages are replete with examples of the same devices used for multiple purposes: (1) the same type of morphological process (for example, suffixation for both noun case and verb tense) and (2) identical morphemes (for example, the same suffix for English noun plural and possessive). These sorts of similarity would be expected to convey advantages on language learners in the form of transfer from one morphological category to another. Connectionist models of morphology acquisition have been faulted for their supposed inability to represent phonological similarity across morphological categories and hence to facilitate transfer. This paper describes a connectionist model of the acquisition of morphology which is shown to exhibit transfer of this type. The model treats the morphology acquisition problem as one of learning to map forms onto meanings and vice versa. As the network learns these mappings, it makes phonological generalizations which are embedded in connection weights. Since these weights are shared by different morphological categories, transfer is enabled. In a set of experiments with artificial stimuli, networks were trained first on one morphological task (e.g., tense) and then on a second (e.g., number). It is shown that in the context of suffixation, prefixation, and template rules, the second task is facilitated when the second category either makes use of the same forms or the same general process type (e.g., prefixation) as the first. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #83826. 	[Title] Beyond the Cognitive Map: Contributions to a Computational Neuroscience Theory of Rodent Navigation for the [Abstract]  Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #1153922. 	[Title] Locally Connected Recurrent Networks [Abstract] Lai-Wan CHAN and Evan Fung-Yu YOUNG Computer Science Department, The Chinese University of Hong Kong New Territories, Hong Kong Email : lwchan@cs.cuhk.hk Technical Report : CS-TR-95-10 Abstract The fully connected recurrent network (FRN) using the on-line training method, Real Time Recurrent Learning (RTRL), is computationally expensive. It has a computational complexity of O(N 4 ) and storage complexity of O(N 3 ), where N is the number of non-input units. We have devised a locally connected recurrent model which has a much lower complexity in both computational time and storage space. The ring-structure recurrent network (RRN), the simplest kind of the locally connected has the corresponding complexity of O(mn+np) and O(np) respectively, where p, n and m are the number of input, hidden and output units respectively. We compare the performance between RRN and FRN in sequence recognition and time series prediction. We tested the networks' ability in temporal memorizing power and time warpping ability in the sequence recognition task. In the time series prediction task, we used both networks to train and predict three series; a periodic series with white noise, a deterministic chaotic series and the sunspots data. Both tasks show that RRN needs a much shorter training time and the performance of RRN is comparable to that of FRN. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #1132418. 	[Title] NONPARAMETRIC SELECTION OF INPUT VARIABLES FOR CONNECTIONIST LEARNING [Abstract]  Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #1127812. 	[Title] Meter as Mechanism: A Neural Network that Learns Metrical Patterns [Abstract] One kind of prosodic structure that apparently underlies both music and language is meter. Yet detailed measurements of both music and speech show that the nested periodicities that define metrical structure are noisy in some sense. What kind of system could produce or perceive such variable metrical timing? And what would it take to store particular metrical patterns in the long-term memory of the system? We have developed a network of coupled oscillators that both produces and perceives metrical patterns of pulses. In addition, beginning with an initial state with no biases, it learns to prefer 3-beat patterns (like waltzes) over 2-beat patterns. Models of this general class could learn to entrain to musical patterns. And given a way to process speech to extract appropriate pulses, the model should be applicable to metrical structure in speech as well. Is language metrical? Meter refers both to particular sorts of patterns in time and to an abstract description of such patterns, potentially a cognitive representation of them. In both cases there are two or more hierarchical levels at which equally spaced events occur, and the periods characterizing these levels are integral multiples of each other (usually 2 or 3). The hierarchy is implied in standard Western musical notation, where the different levels are indicated by kinds of notes (quarter notes, half notes, etc.) and by bars separating measures. For example, in a basic waltz-time meter, there are individual beats, all with the same spacing, grouped into sets of three with every third one receiving a stronger accent. In such a meter, there is a hierarchy consisting of both a faster periodic cycle (at the beat level) and a slower one (the measure level) that is 1/3 as fast with its onset (or zero phase angle) coinciding with the zero phase angle of every third beat. Metrical systems like this seem to underlie most forms of music around the world and are often said to underlie human speech as well (Jones, 1932; Martin, 1972). However, an awkward difficulty is that the definition employs the notion of an integer since data on both music and speech show clearly that the perfect temporal ratios predicted by such a definition are not observed in performance. In music performance, various kinds of systematic temporal deviations in the timing specified by musical notation are known to Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #24966. 	[Title] A Practical Bayesian Framework for Backprop Networks [Abstract] A quantitative and practical Bayesian framework is described for learning of mappings in feedforward networks. The framework makes possible: (1) objective comparisons between solutions using alternative network architectures; (2) objective stopping rules for network pruning or growing procedures; (3) objective choice of magnitude and type of weight decay terms or additive regularisers (for penalising large weights, etc.); (4) a measure of the effective number of well-determined parameters in a model; (5) quantified estimates of the error bars on network parameters and on network output; (6) objective comparisons with alternative learning and interpolation models such as splines and radial basis functions. The Bayesian `evidence' automatically embodies `Occam's razor,' penalising over-flexible and over-complex models. The Bayesian approach helps detect poor underlying assumptions in learning models. For learning models well matched to a problem, a good correlation between generalisation ability and the Bayesian evidence is obtained. This paper makes use of the Bayesian framework for regularisation and model comparison described in the companion paper `Bayesian interpolation' (MacKay, 1991a). This framework is due to Gull and Skilling (Gull, 1989a). Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #1128839. 	[Title] Requirements and use of neural networks for industrial appli- [Abstract] Modern industry of today needs flexible, adaptive and fault-tolerant methods for information processing. Several applications have shown that neural networks fulfill these requirements. In this paper application areas, in which neural networks have been successfully used, are presented. Then a kind of check list is described, which mentioned the different steps, when applying neural networks. The paper finished with a discussion of some neural networks projects done in the research group Interactive Planning at the Research Center for Computer Science (FZI). Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #62347. 	[Title] Embodiment and Manipulation Learning Process for a Humanoid Hand [Abstract]  Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #202639. 	[Title] PREDICTING SUNSPOTS AND EXCHANGE RATES WITH CONNECTIONIST NETWORKS [Abstract] We investigate the effectiveness of connectionist networks for predicting the future continuation of temporal sequences. The problem of overfitting, particularly serious for short records of noisy data, is addressed by the method of weight-elimination: a term penalizing network complexity is added to the usual cost function in back-propagation. The ultimate goal is prediction accuracy. We analyze two time series. On the benchmark sunspot series, the networks outperform traditional statistical approaches. We show that the network performance does not deteriorate when there are more input units than needed. Weight-elimination also manages to extract some part of the dynamics of the notoriously noisy currency exchange rates and makes the network solution interpretable. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #1109392. 	[Title] Disambiguation and Grammar as Emergent Soft Constraints [Abstract] When reading a sentence such as "The diplomat threw the ball in the ballpark for the princess" our interpretation changes from a dance event to baseball and back to dance. Such on-line disambiguation happens automatically and appears to be based on dynamically combining the strengths of association between the keywords and the two senses. Subsymbolic neural networks are very good at modeling such behavior. They learn word meanings as soft constraints on interpretation, and dynamically combine these constraints to form the most likely interpretation. On the other hand, it is very difficult to show how systematic language structures such as relative clauses could be processed in such a system. The network would only learn to associate them to specific contexts and would not be able to process new combinations of them. A closer look at understanding embedded clauses shows that humans are not very systematic in processing grammatical structures either. For example, "The girl who the boy who the girl who lived next door blamed hit cried" is very difficult to understand, whereas "The car that the man who the dog that had rabies bit drives is in the garage" is not. This difference emerges from the same semantic constraints that are at work in the disambiguation task. In this chapter we will show how the subsymbolic parser can be combined with high-level control that allows the system to process novel combinations of relative clauses systematically, while still being sensitive to the semantic constraints. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #92065. 	[Title] ADAPTIVE REGULARIZATION [Abstract] Regularization, e.g., in the form of weight decay, is important for training and optimization of neural network architectures. In this work we provide a tool based on asymptotic sampling theory, for iterative estimation of weight decay parameters. The basic idea is to do a gradient descent in the estimated generalization error with respect to the regularization parameters. The scheme is implemented in our Designer Net framework for network training and pruning, i.e., is based on the diagonal Hessian approximation. The scheme does not require essential computational overhead in addition to what is needed for training and pruning. The viability of the approach is demonstrated in an experiment concerning prediction of the chaotic Mackey-Glass series. We find that the optimized weight decays are relatively large for densely connected networks in the initial pruning phase, while they decrease as pruning proceeds. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #1130915. 	[Title] Combining Neural Network Forecasts on Wavelet-Transformed Time Series [Abstract]  Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #1128868. 	[Title] Identification of Protein Coding Regions In Genomic DNA Molecular, Cellular and Developmental Biology, Keywords: gene [Abstract]  Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #72101. 	[Title] Constructive Learning of Recurrent Neural Networks: Limitations of Recurrent Casade Correlation and a Simple Solution [Abstract] It is often difficult to predict the optimal neural network size for a particular application. Constructive or destructive methods that add or subtract neurons, layers, connections, etc. might offer a solution to this problem. We prove that one method, Recurrent Cascade Correlation, due to its topology, has fundamental limitations in representation and thus in its learning capabilities. It cannot represent with monotone (i.e. sigmoid) and hard-threshold activation functions certain finite state automata. We give a "preliminary" approach on how to get around these limitations by devising a simple constructive training method that adds neurons during training while still preserving the powerful fully-recurrent structure. We illustrate this approach by simulations which learn many examples of regular grammars that the Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #48550. 	[Title] Identification and Control of Nonlinear Systems Using Neural Network Models: Design and Stability Analysis [Abstract] Report 91-09-01 September 1991 (revised) May 1994 Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #133550. 	[Title] OBSERVABILITY IN RECURRENT NEURAL NETWORKS [Abstract] Report SYCON-92-07rev ABSTRACT We obtain a characterization of observability for a class of nonlinear systems which appear in neural networks research. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #1130927. 	[Title] Parametric regression 1.1 Learning problem model f bw b w in turn is an estimator [Abstract] Let us present briefly the learning problem we will address in this chapter and the following. The ultimate goal is the modelling of a mapping f : x 7! y from multidimensional input x to output y. The output can be multi-dimensional, but we will mostly address situations where it is a one dimensional real value. Furthermore, we should take into account the fact that we scarcely ever observe the actual true mapping y = f (x). This is due to perturbations such as e.g. observational noise. We will rather have a joint probability p (x; y). We expect this probability to be peaked for values of x and y corresponding to the mapping. We focus on automatic learning by example. A set D = of data sampled from the joint distribution p (x; y) = p (yjx) p (x) is collected. With the help of this set, we try to identify a model of the data, parameterised by a set of 1.2 Learning and optimisation The fit of the model to the system in a given point x is measured using a criterion representing the distance from the model prediction b y to the system, e (y; f w (x)). This is the local risk . The performance of the model is measured by the expected This quantity represents the ability to yield good performance for all the possible situations (i.e. (x; y) pairs) and is thus called generalisation error . The optimal set 1 parameters w: f w : x 7! b y. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #1130931. 	[Title] TH presentee par STATISTICAL LEARNING AND REGULARISATION FOR REGRESSION Application to system identification and time [Abstract]  Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #641976. 	[Title] A comparison of some error estimates for neural network models  Summary [Abstract] We discuss a number of methods for estimating the standard error of predicted values from a multi-layer perceptron. These methods include the delta method based on the Hessian, bootstrap estimators, and the "sandwich" estimator. The methods are described and compared in a number of examples. We find that the bootstrap methods perform best, partly because they capture variability due to the choice of starting weights. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #242637. 	[Title] A Delay Damage Model Selection Algorithm for NARX Neural Networks [Abstract] Recurrent neural networks have become popular models for system identification and time series prediction. NARX (Nonlinear AutoRegressive models with eXogenous inputs) neural network models are a popular subclass of recurrent networks and have been used in many applications. Though embedded memory can be found in all recurrent network models, it is particularly prominent in NARX models. We show that using intelligent memory order selection through pruning and good initial heuristics significantly improves the generalization and predictive performance of these nonlinear systems on problems as diverse as grammatical inference and time series prediction. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #23502. 	[Title] Dyslexic and Category-Specific Aphasic Impairments in a Self-Organizing Feature Map Model of the Lexicon [Abstract] DISLEX is an artificial neural network model of the mental lexicon. It was built to test com-putationally whether the lexicon could consist of separate feature maps for the different lexical modalities and the lexical semantics, connected with ordered pathways. In the model, the orthographic, phonological, and semantic feature maps and the associations between them are formed in an unsupervised process, based on cooccurrence of the lexical symbol and its meaning. After the model is organized, various damage to the lexical system can be simulated, resulting in dyslexic and category-specific aphasic impairments similar to those observed in human patients. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #27631. 	[Title] Control Systems Magazine, 14, 1, pp.57-71. Robot Juggling: An Implementation of Memory-based Learning [Abstract] This paper explores issues involved in implementing robot learning for a challenging dynamic task, using a case study from robot juggling. We use a memory-based local model - ing approach (locally weighted regression) to represent a learned model of the task to be performed. Statistical tests are given to examine the uncertainty of a model, to optimize its pre diction quality, and to deal with noisy and corrupted data. We develop an exploration algorithm that explicitly deals with prediction accuracy requirements dur ing explo - ration. Using all these ingredients in combination with methods from optimal control, our robot achieves fast real - time learning of the task within 40 to 100 trials. * Address of both authors: Massachusetts Institute of Technology, The Artificial Intelligence Laboratory & The Department of Brain and Cognitive Sciences, 545 Technology Square, Cambride, MA 02139, USA. Email: ss-chaal@ai.mit.edu, cga@ai.mit.edu. Support was provided by the Air Force Office of Sci entific Research and by Siemens Cor pora tion. Support for the first author was provided by the Ger man Scholar ship Foundation and the Alexander von Hum boldt Founda tion. Support for the second author was provided by a Na tional Sci ence Foundation Pre sidential Young Investigator Award. We thank Gideon Stein for im ple ment ing the first version of LWR on the i860 microprocessor, and Gerrie van Zyl for build ing the devil stick robot and implementing the first version of devil stick learning. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #654326. 	[Title] Evolutionary Training of CLP-Constrained Neural Networks [Abstract] The paper is concerned with the integration of constraint logic programming systems (CLP) with systems based on genetic algorithms (GA). The resulting framework is tailored for applications that require a first phase in which a number of constraints need to be generated, and a second phase in which an optimal solution satisfying these constraints is produced. The first phase is carried by the CLP and the second one by the GA. We present a specific framework where ECL i PS e (ECRC Common Logic Programming System) and GENOCOP (GEnetic algorithm for Numerical Optimization for COnstrained Problems) are integrated in a framework called CoCo (COmputational intelligence plus COnstraint logic programming). The CoCo system is applied to the training problem for neural networks. We consider constrained networks, e.g. neural networks with shared weights, constraints on the weights for example domain constraints for hardware implementation etc. Then ECL i PS e is used to generate the chromosome representation together with other constraints which ensure, in most cases, that each network is specified by exactly one chromosome. Thus the problem becomes a constrained optimization problem, where the optimization criterion is to optimize the error of the network, and GENOCOP is used to find an optimal solution. Note: The work of the second author was partially supported by SION, a department of the NWO, the National Foundation for Scientific Research. This work has been carried out while the third author was visiting CWI, Amsterdam, and the fourth author was visiting Leiden University. Please summarise all the given information.

Paper #144408 has the title `Neural net architectures for temporal sequence processing  
` and the abstract `I present a general taxonomy of neural net architectures for processing time-varying patterns. This taxonomy subsumes many existing architectures in the literature, and points to several promising architectures that have yet to be examined. Any architecture that processes time-varying patterns requires two conceptually distinct components: a short-term memory that holds on to relevant past events and an associator that uses the short-term memory to classify or predict. My taxonomy is based on a characterization of short-term memory models along the dimensions of form, content, and adaptability. Experiments on predicting future values of a financial time series (US dollar-Swiss franc exchange rates) are presented using several alternative memory models. The results of these experiments serve as a baseline against which more sophisticated architectures can be compared. Neural networks have proven to be a promising alternative to traditional techniques for nonlinear temporal prediction tasks (e.g., Curtiss, Brandemuehl, & Kreider, 1992; Lapedes & Farber, 1987; Weigend, Huberman, & Rumelhart, 1992). However, temporal prediction is a particularly challenging problem because conventional neural net architectures and algorithms are not well suited for patterns that vary over time. The prototypical use of neural nets is in structural pattern recognition. In such a task, a collection of features|visual, semantic, or otherwise|is presented to a network and the network must categorize the input feature pattern as belonging to one or more classes. For example, a network might be trained to classify animal species based on a set of attributes describing living creatures such as "has tail", "lives in water", or "is carnivorous"; or a network could be trained to recognize visual patterns over a two-dimensional pixel array as a letter in fA; B; . . . ; Zg. In such tasks, the network is presented with all relevant information simultaneously. In contrast, temporal pattern recognition involves processing of patterns that evolve over time. The appropriate response at a particular point in time depends not only on the current input, but potentially all previous inputs. This is illustrated in Figure 1, which shows the basic framework for a temporal prediction problem. I assume that time is quantized into discrete steps, a sensible assumption because many time series of interest are intrinsically discrete, and continuous series can be sampled at a fixed interval. The input at time t is denoted x(t). For univariate series, this input 
`.
Question: Which category should Paper #144408 be classified as ?
You can select one from ['Neural_Networks', 'Case_Based', 'Theory', 'Reinforcement_Learning', 'Probabilistic_Methods', 'Rule_Learning', 'Genetic_Algorithms'].


Paper #144408 has the title `Neural net architectures for temporal sequence processing  
` and the abstract `I present a general taxonomy of neural net architectures for processing time-varying patterns. This taxonomy subsumes many existing architectures in the literature, and points to several promising architectures that have yet to be examined. Any architecture that processes time-varying patterns requires two conceptually distinct components: a short-term memory that holds on to relevant past events and an associator that uses the short-term memory to classify or predict. My taxonomy is based on a characterization of short-term memory models along the dimensions of form, content, and adaptability. Experiments on predicting future values of a financial time series (US dollar-Swiss franc exchange rates) are presented using several alternative memory models. The results of these experiments serve as a baseline against which more sophisticated architectures can be compared. Neural networks have proven to be a promising alternative to traditional techniques for nonlinear temporal prediction tasks (e.g., Curtiss, Brandemuehl, & Kreider, 1992; Lapedes & Farber, 1987; Weigend, Huberman, & Rumelhart, 1992). However, temporal prediction is a particularly challenging problem because conventional neural net architectures and algorithms are not well suited for patterns that vary over time. The prototypical use of neural nets is in structural pattern recognition. In such a task, a collection of features|visual, semantic, or otherwise|is presented to a network and the network must categorize the input feature pattern as belonging to one or more classes. For example, a network might be trained to classify animal species based on a set of attributes describing living creatures such as "has tail", "lives in water", or "is carnivorous"; or a network could be trained to recognize visual patterns over a two-dimensional pixel array as a letter in fA; B; . . . ; Zg. In such tasks, the network is presented with all relevant information simultaneously. In contrast, temporal pattern recognition involves processing of patterns that evolve over time. The appropriate response at a particular point in time depends not only on the current input, but potentially all previous inputs. This is illustrated in Figure 1, which shows the basic framework for a temporal prediction problem. I assume that time is quantized into discrete steps, a sensible assumption because many time series of interest are intrinsically discrete, and continuous series can be sampled at a fixed interval. The input at time t is denoted x(t). For univariate series, this input 
`.
Question: Which category should Paper #144408 be classified as ?
You can select one from ['Neural_Networks', 'Case_Based', 'Theory', 'Reinforcement_Learning', 'Probabilistic_Methods', 'Rule_Learning', 'Genetic_Algorithms'].

This paper should be classified into: Neural_Networks