Above are papers related to the following paper: [Paper_ID] Paper #123556. 	[Title] BACKPROPAGATION SEPARATES WHERE PERCEPTRONS DO [Abstract] Feedforward nets with sigmoidal activation functions are often designed by minimizing a cost criterion. It has been pointed out before that this technique may be outperformed by the classical perceptron learning rule, at least on some problems. In this paper, we show that no such pathologies can arise if the error criterion is of a threshold LMS type, i.e., is zero for values "beyond" the desired target values. More precisely, we show that if the data are linearly separable, and one considers nets with no hidden neurons, then an error function as above cannot have any local minima that are not global. Simulations of networks with hidden units are consistent with these results, in that often data which can be classified when minimizing a threshold LMS criterion may fail to be classified when using instead a simple LMS cost. In addition, the proof gives the following stronger result, under the stated hypotheses: the continuous gradient adjustment procedure is such that from any initial weight configuration a separating set of weights is obtained in finite time. This is a precise analogue of the Perceptron Learning Theorem. The results are then compared with the more classical pattern recognition problem of threshold LMS with linear activations, where no spurious local minima exist even for nonseparable data: here it is shown that even if using the threshold criterion, such bad local minima may occur, if the data are not separable and sigmoids are used. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #289088. 	[Title] Critical Points for Least-Squares Problems Involving Certain Analytic Functions, with Applications to Sigmoidal Nets [Abstract] This paper deals with nonlinear least-squares problems involving the fitting to data of parameterized analytic functions. For generic regression data, a general result establishes the countability, and under stronger assumptions finiteness, of the set of functions giving rise to critical points of the quadratic loss function. In the special case of what are usually called "single-hidden layer neural networks," which are built upon the standard sigmoidal activation tanh(x) (or equivalently (1 + e x ) 1 ), a rough upper bound for this cardinality is provided as well. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #1109891. 	[Title] Interconnected Automata and Linear Systems: A Theoretical Framework in Discrete-Time In Hybrid Systems III: Verification [Abstract] This paper summarizes the definitions and several of the main results of an approach to hybrid systems, which combines finite automata and linear systems, developed by the author in the early 1980s. Some related more recent results are briefly mentioned as well. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #141160. 	[Title] LINEAR SYSTEMS WITH SIGN-OBSERVATIONS [Abstract] This paper deals with systems that are obtained from linear time-invariant continuous-or discrete-time devices followed by a function that just provides the sign of each output. Such systems appear naturally in the study of quantized observations as well as in signal processing and neural network theory. Results are given on observability, minimal realizations, and other system-theoretic concepts. Certain major differences exist with the linear case, and other results generalize in a surprisingly straightforward manner. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #1111240. 	[Title] On the Distribution of Performance from Multiple Neural Network Trials, On the Distribution of Performance [Abstract] Andrew D. Back was with the Department of Electrical and Computer Engineering, University of Queensland. St. Lucia, Australia. He is now with the Brain Information Processing Group, Frontier Research Program, RIKEN, The Institute of Physical and Chemical Research, 2-1 Hirosawa, Wako-shi, Saitama 351-01, Japan Abstract The performance of neural network simulations is often reported in terms of the mean and standard deviation of a number of simulations performed with different starting conditions. However, in many cases, the distribution of the individual results does not approximate a Gaussian distribution, may not be symmetric, and may be multimodal. We present the distribution of results for practical problems and show that assuming Gaussian distributions can significantly affect the interpretation of results, especially those of comparison studies. For a controlled task which we consider, we find that the distribution of performance is skewed towards better performance for smoother target functions and skewed towards worse performance Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #1131345. 	[Title] Lemma 2.3 The system is reachable and observable and realizes the same input/output behavior as [Abstract] Here we show a similar construction for multiple-output systems, with some modifications. Let = (A; B; C) s be a discrete-time sign-linear system with state space IR n and p outputs. Perform a change of ; where A 1 (n 1 fi n 1 ) is invertible and A 2 (n 2 fi n 2 ) is nilpotent. If (A; B) is a reachable pair and (A; C) is an observable pair, then is minimal in the sense that any other sign-linear system with the same input/output behavior has dimension at least n. But, if n 1 &lt; n, then det A = 0 and is not observable and hence not canonical. Let us find another system ~ (necessarily not sign-linear) which has the same input/output behavior as , but is canonical. Let i be the relative degree of the ith row of the Markov sequence A, and = minf i : i = 1; : : : ; pg. Let the initial state be x. There is a difference between the case when the smallest relative degree is greater or equal to n 2 and the case when &lt; n 2 . Roughly speaking, when n 2 the outputs of the sign-linear system give us information about sign (Cx), sign (CAx), : : : , sign (CA 1 x), which are the first outputs of the sys tem. After that, we can use the inputs and outputs to learn only about x 1 (the first n 1 components of x). When &lt; n 2 , we may be able to use some controls to learn more about x 2 (the last n 2 components of x) before time n 2 when the nilpotency of A 2 has finally Lemma 2.4 Two states x and z are indistinguishable for if and only if (x) = (z). Proof. In the case n 2 , we have only the equations x 1 = z 1 and the equality of the 's. The first ` output terms for are exactly the terms of . So these equalities are satisfied if and only if the first ` output terms coincide for x and z, for any input. Equality of everything but the first n 1 components is equivalent to the first n 2 output terms coinciding for x and z, since the jth row of the qth output, for initial state x, for example, is either sign (c j A q x) if j &gt; q, or sign (c j A q x + + A j j u q j +1 + ) if j q in which case we may use the control u q j +1 to identify c j A q x (using Remark 3.3 in [1]). Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #50354. 	[Title] BACKPROPAGATION CAN GIVE RISE TO SPURIOUS LOCAL MINIMA EVEN FOR NETWORKS WITHOUT HIDDEN LAYERS [Abstract] We give an example of a neural net without hidden layers and with a sigmoid transfer function, together with a training set of binary vectors, for which the sum of the squared errors, regarded as a function of the weights, has a local minimum which is not a global minimum. The example consists of a set of 125 training instances, with four weights and a threshold to be learnt. We do not know if substantially smaller binary examples exist. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #133553. 	[Title] Sample Complexity for Learning Recurrent Perceptron Mappings [Abstract] Recurrent perceptron classifiers generalize the classical perceptron model. They take into account those correlations and dependences among input coordinates which arise from linear digital filtering. This paper provides tight bounds on sample complexity associated to the fitting of such models to experimental data. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #917493. 	[Title] Observability of Linear Systems with Saturated Outputs [Abstract] In this paper, we present necessary and sufficient conditions for observability of the class of output-saturated systems. These are linear systems whose output passes through a saturation function before it can be measured. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #14807. 	[Title] Exponentially many local minima for single neurons [Abstract] We show that for a single neuron with the logistic function as the transfer function the number of local minima of the error function based on the square loss can grow exponentially in the dimension. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #264347. 	[Title] Relative Loss Bounds for Multidimensional Regression Problems [Abstract] We study on-line generalized linear regression with multidimensional outputs, i.e., neural networks with multiple output nodes but no hidden nodes. We allow at the final layer transfer functions such as the softmax function that need to consider the linear activations to all the output neurons. We also use a parameterization function which transforms parameter vectors maintained by the algorithm into the actual weights. The on-line algorithm we consider updates the parameters in an additive manner, analogous to the delta rule, but because the actual weights are obtained via the possibly nonlinear parameterization function they may behave in a very different manner. Our approach is based on applying the notion of a matching loss function in two different contexts. First, we measure the loss of the algorithm in terms of the loss that matches the transfer function used to produce the outputs. Second, the loss function that matches the parameterization function can be used both as a measure of distance between models in motivating the update rule of the algorithm and as a potential function in analyzing its relative performance compared to an arbitrary fixed model. As a result, we have a unified treatment that generalizes earlier results for the gradient descent and exponentiated gradient algorithms to multidimensional outputs, including multiclass logistic regression. Please summarise all the given information.

Paper #123556 has the title `BACKPROPAGATION SEPARATES WHERE PERCEPTRONS DO  
` and the abstract `Feedforward nets with sigmoidal activation functions are often designed by minimizing a cost criterion. It has been pointed out before that this technique may be outperformed by the classical perceptron learning rule, at least on some problems. In this paper, we show that no such pathologies can arise if the error criterion is of a threshold LMS type, i.e., is zero for values "beyond" the desired target values. More precisely, we show that if the data are linearly separable, and one considers nets with no hidden neurons, then an error function as above cannot have any local minima that are not global. Simulations of networks with hidden units are consistent with these results, in that often data which can be classified when minimizing a threshold LMS criterion may fail to be classified when using instead a simple LMS cost. In addition, the proof gives the following stronger result, under the stated hypotheses: the continuous gradient adjustment procedure is such that from any initial weight configuration a separating set of weights is obtained in finite time. This is a precise analogue of the Perceptron Learning Theorem. The results are then compared with the more classical pattern recognition problem of threshold LMS with linear activations, where no spurious local minima exist even for nonseparable data: here it is shown that even if using the threshold criterion, such bad local minima may occur, if the data are not separable and sigmoids are used. 
`.
Question: Which category should Paper #123556 be classified as ?
You can select one from ['Neural_Networks', 'Case_Based', 'Theory', 'Reinforcement_Learning', 'Probabilistic_Methods', 'Rule_Learning', 'Genetic_Algorithms'].


Paper #123556 has the title `BACKPROPAGATION SEPARATES WHERE PERCEPTRONS DO  
` and the abstract `Feedforward nets with sigmoidal activation functions are often designed by minimizing a cost criterion. It has been pointed out before that this technique may be outperformed by the classical perceptron learning rule, at least on some problems. In this paper, we show that no such pathologies can arise if the error criterion is of a threshold LMS type, i.e., is zero for values "beyond" the desired target values. More precisely, we show that if the data are linearly separable, and one considers nets with no hidden neurons, then an error function as above cannot have any local minima that are not global. Simulations of networks with hidden units are consistent with these results, in that often data which can be classified when minimizing a threshold LMS criterion may fail to be classified when using instead a simple LMS cost. In addition, the proof gives the following stronger result, under the stated hypotheses: the continuous gradient adjustment procedure is such that from any initial weight configuration a separating set of weights is obtained in finite time. This is a precise analogue of the Perceptron Learning Theorem. The results are then compared with the more classical pattern recognition problem of threshold LMS with linear activations, where no spurious local minima exist even for nonseparable data: here it is shown that even if using the threshold criterion, such bad local minima may occur, if the data are not separable and sigmoids are used. 
`.
Question: Which category should Paper #123556 be classified as ?
You can select one from ['Neural_Networks', 'Case_Based', 'Theory', 'Reinforcement_Learning', 'Probabilistic_Methods', 'Rule_Learning', 'Genetic_Algorithms'].

This paper should be classified into: Neural_Networks