Above are papers related to the following paper: [Paper_ID] Paper #16437. 	[Title] Learning Context-free Grammars: Capabilities and Limitations of a Recurrent Neural Network with an External Stack Memory [Abstract] This work describes an approach for inferring Deterministic Context-free (DCF) Grammars in a Connectionist paradigm using a Recurrent Neural Network Pushdown Automaton (NNPDA). The NNPDA consists of a recurrent neural network connected to an external stack memory through a common error function. We show that the NNPDA is able to learn the dynamics of an underlying pushdown automaton from examples of grammatical and non-grammatical strings. Not only does the network learn the state transitions in the automaton, it also learns the actions required to control the stack. In order to use continuous optimization methods, we develop an analog stack which reverts to a discrete stack by quantization of all activations, after the network has learned the transition rules and stack actions. We further show an enhancement of the network's learning capabilities by providing hints. In addition, an initial comparative study of simulations with first, second and third order recurrent networks has shown that the increased degree of freedom in a higher order networks improve generalization but not necessarily learning speed. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #20193. 	[Title] The Structure-Mapping Engine: Algorithm and Examples [Abstract] This paper describes the Structure-Mapping Engine (SME), a program for studying analogical processing. SME has been built to explore Gentner's Structure-mapping theory of analogy, and provides a "tool kit" for constructing matching algorithms consistent with this theory. Its flexibility enhances cognitive simulation studies by simplifying experimentation. Furthermore, SME is very efficient, making it a useful component in machine learning systems as well. We review the Structure-mapping theory and describe the design of the engine. We analyze the complexity of the algorithm, and demonstrate that most of the steps are polynomial, typically bounded by O (N 2 ). Next we demonstrate some examples of its operation taken from our cognitive simulation studies and work in machine learning. Finally, we compare SME to other analogy programs and discuss several areas for future work. This paper appeared in Artificial Intelligence, 41, 1989, pp 1-63. For more information, please contact forbus@ils.nwu.edu Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #127940. 	[Title] TRAINREC: A System for Training Feedforward Simple Recurrent Networks Efficiently and Correctly [Abstract]  Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #430329. 	[Title] AN ADAPTIVE NEURAL NETWORK PARSER [Abstract] We inv estigate the applicability of an adaptive neural network to problems with time-dependent input by demonstrating that a deterministic parser for natural language inputs of significant syntactic complexity can be developed using recurrent connectionist architectures. The traditional stacking mechanism, known to be necessary for proper treatment of context-free languages in symbolic systems, is absent from the design, having been subsumed by recurrency in the network. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #1135115. 	[Title] Rapid learning of binding-match and binding-error detector circuits via long-term potentiation [Abstract] It is argued that the memorization of events and situations (episodic memory) requires the rapid formation of neural circuits responsive to binding errors and binding matches. While the formation of circuits responsive to binding matches can be modeled by associative learning mechanisms, the rapid formation of circuits responsive to binding errors is difficult to explain given their seemingly paradoxical behavior; such a circuit must be formed in response to the occurrence of a binding (i.e., a particular pattern in the input), but subsequent to its formation, it must not fire anymore in response to the occurrence of the very binding (i.e., pattern) that led to its formation. A plausible account of the formation of such circuits has not been offered. A computational model is described that demonstrates how a transient pattern of activity representing an event can lead to the rapid formation of circuits for detecting bindings and binding errors as a result of long-term potentiation within structures whose architecture and circuitry are similar to those of the hippocampal formation, a neural structure known to be critical to episodic memory. The model exhibits a high memory capacity and is robust against limited amounts of diffuse cell loss. The model also offers an alternate interpretation of the functional role of region CA3 in the formation of episodic memories, and predicts the nature of memory impairment that would result from damage to various regions of the hippocampal formation. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #131117. 	[Title] Learning Sequential Tasks by Incrementally Adding Higher Orders [Abstract] An incremental, higher-order, non-recurrent network combines two properties found to be useful for learning sequential tasks: higher-order connections and incremental introduction of new units. The network adds higher orders when needed by adding new units that dynamically modify connection weights. Since the new units modify the weights at the next time-step with information from the previous step, temporal tasks can be learned without the use of feedback, thereby greatly simplifying training. Furthermore, a theoretically unlimited number of units can be added to reach into the arbitrarily distant past. Experiments with the Reber grammar have demonstrated speedups of two orders of magnitude over recurrent networks. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #1153933. 	[Title] Scaling-up RAAMs [Abstract] Modifications to Recursive Auto-Associative Memory are presented, which allow it to store deeper and more complex data structures than previously reported. These modifications include adding extra layers to the compressor and reconstructor networks, employing integer rather than real-valued representations, pre-conditioning the weights and pre-setting the representations to be compatible with them. The resulting system is tested on a data set of syntactic trees extracted from the Penn Treebank. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #3085. 	[Title] MAC/FAC: A Model of Similarity-based Retrieval [Abstract] We present a model of similarity-based retrieval which attempts to capture three psychological phenomena: (1) people are extremely good at judging similarity and analogy when given items to compare. (2) Superficial remindings are much more frequent than structural remindings. (3) People sometimes experience and use purely structural analogical re-mindings. Our model, called MAC/FAC (for "many are called but few are chosen") consists of two stages. The first stage (MAC) uses a computationally cheap, non-structural matcher to filter candidates from a pool of memory items. That is, we redundantly encode structured representations as content vectors, whose dot product yields an estimate of how well the corresponding structural representations will match. The second stage (FAC) uses SME to compute a true structural match between the probe and output from the first stage. MAC/FAC has been fully implemented, and we show that it is capable of modeling patterns of access found in psychological data. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #395725. 	[Title] Distributed Representations and Nested Compositional Structure [Abstract]  Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #12337. 	[Title] A Unified Gradient-Descent/Clustering Architecture for Finite State Machine Induction [Abstract] Although recurrent neural nets have been moderately successful in learning to emulate finite-state machines (FSMs), the continuous internal state dynamics of a neural net are not well matched to the discrete behavior of an FSM. We describe an architecture, called DOLCE, that allows discrete states to evolve in a net as learning progresses. dolce consists of a standard recurrent neural net trained by gradient descent and an adaptive clustering technique that quantizes the state space. dolce is based on the assumption that a finite set of discrete internal states is required for the task, and that the actual network state belongs to this set but has been corrupted by noise due to inaccuracy in the weights. dolce learns to recover the discrete state with maximum a posteriori probability from the noisy state. Simulations show that dolce leads to a significant improvement in generalization performance over earlier neural net approaches to FSM induction. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #29492. 	[Title] Induction of Multiscale Temporal Structure [Abstract] Learning structure in temporally-extended sequences is a difficult computational problem because only a fraction of the relevant information is available at any instant. Although variants of back propagation can in principle be used to find structure in sequences, in practice they are not sufficiently powerful to discover arbitrary contingencies, especially those spanning long temporal intervals or involving high order statistics. For example, in designing a connectionist network for music composition, we have encountered the problem that the net is able to learn musical structure that occurs locally in time|e.g., relations among notes within a musical phrase|but not structure that occurs over longer time periods|e.g., relations among phrases. To address this problem, we require a means of constructing a reduced description of the sequence that makes global aspects more explicit or more readily detectable. I propose to achieve this using hidden units that operate with different time constants. Simulation experiments indicate that slower time-scale hidden units are able to pick up global structure, structure that simply can not be learned by standard Many patterns in the world are intrinsically temporal, e.g., speech, music, the unfolding of events. Recurrent neural net architectures have been devised to accommodate time-varying sequences. For example, the architecture shown in Figure 1 can map a sequence of inputs to a sequence of outputs. Learning structure in temporally-extended sequences is a difficult computational problem because the input pattern may not contain all the task-relevant information at any instant. Thus, back propagation. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #1129621. 	[Title] Analysis of Dynamical Recognizers [Abstract] Pollack (1991) demonstrated that second-order recurrent neural networks can act as dynamical recognizers for formal languages when trained on positive and negative examples, and observed both phase transitions in learning and IFS-like fractal state sets. Follow-on work focused mainly on the extraction and minimization of a finite state automaton (FSA) from the trained network. However, such networks are capable of inducing languages which are not regular, and therefore not equivalent to any FSA. Indeed, it may be simpler for a small network to fit its training data by inducing such a non-regular language. But when is the network's language not regular? In this paper, using a low dimensional network capable of learning all the Tomita data sets, we present an empirical method for testing whether the language induced by the network is regular or not. We also provide a detailed "-machine analysis of trained networks for both regular and non-regular languages. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #51831. 	[Title] Using Prior Knowledge in an NNPDA to Learn Context-Free Languages [Abstract] Although considerable interest has been shown in language inference and automata induction using recurrent neural networks, success of these models has mostly been limited to regular languages. We have previously demonstrated that Neural Network Pushdown Automaton (NNPDA) model is capable of learning deterministic context-free languages (e.g., a n b n and parenthesis languages) from examples. However, the learning task is computationally intensive. In this paper we discuss some ways in which a priori knowledge about the task and data could be used for efficient learning. We also observe that such knowledge is often an experimental prerequisite for learning nontrivial languages (eg. a n b n cb m a m ). Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #16471. 	[Title] Constructing Deterministic Finite-State Automata in Recurrent Neural Networks [Abstract] Recurrent neural networks that are trained to behave like deterministic finite-state automata (DFAs) can show deteriorating performance when tested on long strings. This deteriorating performance can be attributed to the instability of the internal representation of the learned DFA states. The use of a sigmoidal discriminant function together with the recurrent structure contribute to this instability. We prove that a simple algorithm can construct second-order recurrent neural networks with a sparse interconnection topology and sigmoidal discriminant function such that the internal DFA state representations are stable, i.e. the constructed network correctly classifies strings of arbitrary length. The algorithm is based on encoding strengths of weights directly into the neural network. We derive a relationship between the weight strength and the number of DFA states for robust string classification. For a DFA with n states and m input alphabet symbols, the constructive algorithm generates a "programmed" neural network with O(n) neurons and O(mn) weights. We compare our algorithm to other methods proposed in the literature. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #16470. 	[Title] Finite State Machines and Recurrent Neural Networks Automata and Dynamical Systems Approaches [Abstract]  Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #16474. 	[Title] Rule Revision with Recurrent Neural Networks [Abstract] Recurrent neural networks readily process, recognize and generate temporal sequences. By encoding grammatical strings as temporal sequences, recurrent neural networks can be trained to behave like deterministic sequential finite-state automata. Algorithms have been developed for extracting grammatical rules from trained networks. Using a simple method for inserting prior knowledge (or rules) into recurrent neural networks, we show that recurrent neural networks are able to perform rule revision. Rule revision is performed by comparing the inserted rules with the rules in the finite-state automata extracted from trained networks. The results from training a recurrent neural network to recognize a known non-trivial, randomly generated regular grammar show that not only do the networks preserve correct rules but that they are able to correct through training inserted rules which were initially incorrect. (By incorrect, we mean that the rules were not the ones in the randomly generated grammar.) Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #152219. 	[Title] A Connectionist Symbol Manipulator That Discovers the Structure of Context-Free Languages [Abstract] We present a neural net architecture that can discover hierarchical and recursive structure in symbol strings. To detect structure at multiple levels, the architecture has the capability of reducing symbols substrings to single symbols, and makes use of an external stack memory. In terms of formal languages, the architecture can learn to parse strings in an LR(0) context-free grammar. Given training sets of positive and negative exemplars, the architecture has been trained to recognize many different grammars. The architecture has only one layer of modifiable weights, allowing for a Many cognitive domains involve complex sequences that contain hierarchical or recursive structure, e.g., music, natural language parsing, event perception. To illustrate, "the spider that ate the hairy fly" is a noun phrase containing the embedded noun phrase "the hairy fly." Understanding such multilevel structures requires forming reduced descriptions (Hinton, 1988) in which a string of symbols or states ("the hairy fly") is reduced to a single symbolic entity (a noun phrase). We present a neural net architecture that learns to encode the structure of symbol strings via such reduction transformations. The difficult problem of extracting multilevel structure from complex, extended sequences has been studied by Mozer (1992), Ring (1993), Rohwer (1990), and Schmidhuber (1992), among others. While these previous efforts have made some straightforward interpretation of its behavior. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #16476. 	[Title] Extraction of Rules from Discrete-Time Recurrent Neural Networks [Abstract] The extraction of symbolic knowledge from trained neural networks and the direct encoding of (partial) knowledge into networks prior to training are important issues. They allow the exchange of information between symbolic and connectionist knowledge representations. The focus of this paper is on the quality of the rules that are extracted from recurrent neural networks. Discrete-time recurrent neural networks can be trained to correctly classify strings of a regular language. Rules defining the learned grammar can be extracted from networks in the form of deterministic finite-state automata (DFA's) by applying clustering algorithms in the output space of recurrent state neurons. Our algorithm can extract different finite-state automata that are consistent with a training set from the same network. We compare the generalization performances of these different models and the trained network and we introduce a heuristic that permits us to choose among the consistent DFA's the model which best approximates the learned regular grammar. Keywords: Recurrent Neural Networks, Grammatical Inference, Regular Languages, Deterministic Finite-State Automata, Rule Extraction, Generalization Performance, Model Selection, Occam's Razor. fl Technical Report CS-TR-3465 and UMIACS-TR-95-54, University of Maryland, College Park, MD 20742. Ac cepted for publication in Neural Networks. Please summarise all the given information.

Paper #16437 has the title `Learning Context-free Grammars: Capabilities and Limitations of a Recurrent Neural Network with an External Stack Memory  
` and the abstract `This work describes an approach for inferring Deterministic Context-free (DCF) Grammars in a Connectionist paradigm using a Recurrent Neural Network Pushdown Automaton (NNPDA). The NNPDA consists of a recurrent neural network connected to an external stack memory through a common error function. We show that the NNPDA is able to learn the dynamics of an underlying pushdown automaton from examples of grammatical and non-grammatical strings. Not only does the network learn the state transitions in the automaton, it also learns the actions required to control the stack. In order to use continuous optimization methods, we develop an analog stack which reverts to a discrete stack by quantization of all activations, after the network has learned the transition rules and stack actions. We further show an enhancement of the network's learning capabilities by providing hints. In addition, an initial comparative study of simulations with first, second and third order recurrent networks has shown that the increased degree of freedom in a higher order networks improve generalization but not necessarily learning speed. 
`.
Question: Which category should Paper #16437 be classified as ?
You can select one from ['Neural_Networks', 'Case_Based', 'Theory', 'Reinforcement_Learning', 'Probabilistic_Methods', 'Rule_Learning', 'Genetic_Algorithms'].


Paper #16437 has the title `Learning Context-free Grammars: Capabilities and Limitations of a Recurrent Neural Network with an External Stack Memory  
` and the abstract `This work describes an approach for inferring Deterministic Context-free (DCF) Grammars in a Connectionist paradigm using a Recurrent Neural Network Pushdown Automaton (NNPDA). The NNPDA consists of a recurrent neural network connected to an external stack memory through a common error function. We show that the NNPDA is able to learn the dynamics of an underlying pushdown automaton from examples of grammatical and non-grammatical strings. Not only does the network learn the state transitions in the automaton, it also learns the actions required to control the stack. In order to use continuous optimization methods, we develop an analog stack which reverts to a discrete stack by quantization of all activations, after the network has learned the transition rules and stack actions. We further show an enhancement of the network's learning capabilities by providing hints. In addition, an initial comparative study of simulations with first, second and third order recurrent networks has shown that the increased degree of freedom in a higher order networks improve generalization but not necessarily learning speed. 
`.
Question: Which category should Paper #16437 be classified as ?
You can select one from ['Neural_Networks', 'Case_Based', 'Theory', 'Reinforcement_Learning', 'Probabilistic_Methods', 'Rule_Learning', 'Genetic_Algorithms'].

This paper should be classified into: Neural_Networks