exploring constructive cascade networks constructive algorithms have proved to be powerful methods for training feedforward neural networks an important property of these algorithms is generalization a series of empirical studies were performed to examine the effect of regularization on generalization in constructive cascade algorithms it was found that the combination of early stopping and regularization resulted in better generalization than the use of early stopping alone a cubic penalty term that greatly penalizes large weights was shown to be beneficial for generalization in cascade networks an adaptive method of setting the regularization magnitude in constructive algorithms was introduced and shown to produce generalization results similar to those obtained with a fixed user optimized regularization setting this adaptive method also resulted in the construction of smaller networks for more complex problems the acasper algorithm which incorporates the insights obtained from the empirical studies was shown to have good generalization and network construction properties this algorithm was compared to the cascade correlation algorithm on the proben 1 and additional regression data sets

coherent states for the unitary symplectic group we present an explicit construction of coherent states for an arbitrary irreducible representation of the unitary symplectic group usp 4 three different families of coherent states are obtained corresponding to the subgroups u 1 u 1 u 2 and su 2 su 2 the symplectic structure on the manifold of coherent states is obtained and canonical coordinates are used to express the classical limit of quantum observables one of the families is seen to provide a trivial classical limit

universal approximation using radial basis function networks there have been several recent studies concerning feedforward networks and the problem of approximating arbitrary functionals of a finite number of real variables some of these studies deal with cases in which the hidden layer nonlinearity is not a sigmoid this was motivated by successful applications of feedforward networks with nonsigmoidal hidden layer units this paper reports on a related study of radial basis function rbf networks and it is proved that rbf networks having one hidden layer are capable of universal approximation here the emphasis is on the case of typical rbf networks and the results show that a certain class of rbf networks with the same smoothing factor in each kernel node is broad enough for universal approximation

walks on graphs and their connections with tensor invariants and centralizer algebras abstract the number of walks of k steps from the node 0 to the node on the mckay quiver determined by a finite group g and a g module v is the multiplicity of the irreducible g module g in the tensor power v k and it is also the dimension of the irreducible module labeled by for the centralizer algebra z k g end g v k this paper explores ways to effectively calculate that number using the character theory of g we determine the corresponding poincare series the special case 0 gives the poincare series for the tensor invariants t v g k 0 v k g and a tensor analog of molien s formula for polynomial invariants when g is abelian we show that the exponential generating function for the number of walks is a product of generalized hyperbolic functions many graphs such as circulant graphs can be viewed as mckay quivers and the methods presented here provide efficient ways to compute the number of walks on them

matrix elements for the symplectic sp 4 lie algebra the closed gelfand tsetlin type formulae are constructed in this paper for the matrix elements of the symplectic sp 4 lie algebra in an arbitrary irreducible representation using the reduction chain sp 4 sp 2 sp 2 all resulting formulae for the matrix elements were obtained by directly solving the cartan weyl commutation relations of the sp 4 algebra

hermite neural network correlation and application a method for obtaining the correlation of two hermite neural networks is developed the method is based on the fact that a hermite function is unchanged by the fourier transform which allows an expression for the correlation to be obtained directly from the network weights without the need for the fourier transform comparative results with other neural network correlation methods are presented on simulated radar signals

regression modeling in back propagation and projection pursuit learning we study and compare two types of connectionist learning methods for model free regression problems 1 the backpropagation learning bpl and 2 the projection pursuit learning ppl emerged in recent years in the statistical estimation literature both the bpl and the ppl are based on projections of the data in directions determined from interconnection weights however unlike the use of fixed nonlinear activations usually sigmoidal for the hidden neurons in bpl the ppl systematically approximates the unknown nonlinear activations moreover the bpl estimates all the weights simultaneously at each iteration while the ppl estimates the weights cyclically neuron by neuron and layer by layer at each iteration although the bpl and the ppl have comparable training speed when based on a gauss newton optimization algorithm the ppl proves more parsimonious in that the ppl requires a fewer hidden neurons to approximate the true function to further improve the statistical performance of the ppl an orthogonal polynomial approximation is used in place of the supersmoother method originally proposed for nonlinear activation approximation in the ppl

minimax linear observers and regulators for stochastic systems with uncertain second order statistics the problem of minimax design of linear observers and regulators for linear time varying multivariable stochastic systems with uncertain models of their second order statistics is treated in this paper general classes of allowable covariance matrices and means of the process and observation noises and of the random initial condition are considered a game formulation of the problem is adopted and it is shown that the optimal filter for the least favorable set of covariances is minimax robust for each of the filtering situations analyzed conditions satisfied by the saddle point solutions are given and their utility for finding the worst case covariances is illustrated by way of several examples of uncertainty classes of practical interest

radial basis function neural network for approximation and estimation of nonlinear stochastic dynamic systems this paper presents a means to approximate the dynamic and static equations of stochastic nonlinear systems and to estimate state variables based on radial basis function neural network rbfnn after a nonparametric approximate model of the system is constructed from a priori experiments or simulations a suboptimal filter is designed based on the upper bound error in approximating the original unknown plant with nonlinear state and output equations the procedures for both training and state estimation are described along with discussions on approximation error nonlinear systems with linear output equations are considered as a special case of the general formulation finally applications of the proposed rbfnn to the state estimation of highly nonlinear systems are presented to demonstrate the performance and effectiveness of the method

a variable span smoother abstract a variable span smoother based on linear fits is described local cross validation is used to estimate the optimal span as a function of abscissa value computationally efficient algorithms making use of updating formulas are presented

constructive neural networks as estimators of bayesian discriminant functions abstract of crucial importance to the successful use of artificial neural networks for pattern classification problems is how the appropriate network size can be automatically determined this issue is addressed by formulating the process as an automatic search in the space of functions that corresponds to a subclass of multilayer feedforward networks learning is thus a dynamic network construction process which involves adjusting both the network weights and the topology adding new hidden units corresponds to extracting higher level features from the original input features for reducing the residual classification errors it can be shown that the resultant network approximates a bayesian classifier that implements the bayesian decision rule for classification the empirical results of several pattern classification experiments are also reported

a simple lemma on greedy approximation in hilbert space and convergence rates for projection pursuit regression and neural network training a general convergence criterion for certain iterative sequences in hilbert space is presented for an important subclass of these sequences estimates of the rate of convergence are given under very mild assumptions these results establish an 0 1 f4n nonsampling convergence rate for projection pursuit regression and neural network training where n represents the number of ridge functions neurons or coefficients in a greedy basis expansion

ultrametrics in the genetic code and the genome abstract ultrametric approach to the genetic code and the genome is considered and developed p adic degeneracy of the genetic code is pointed out ultrametric tree of the codon space is presented it is shown that codons and amino acids can be treated as p adic ultrametric networks ultrametric modification of the hamming distance is defined and noted how it can be useful ultrametric approach with p adic distance is an attractive and promising trend towards investigation of bioinformation

use of bias term in projection pursuit learning improves approximation and convergence properties in a regression problem one is given a multidimensional random vector x the components of which are called predictor variables and a random variable y called response a regression surface describes a general relationship between x and y a nonparametric regression technique that has been successfully applied to high dimensional data is projection pursuit regression ppr the regression surface is approximated by a sum of empirically determined univariate functions of linear combinations of the predictors projection pursuit learning ppl formulates ppr using a 2 layer feedforward neural network the smoothers in ppr are nonparametric whereas those in ppl are based on hermite functions of some predefined highest order r we demonstrate that ppl networks in the original form do not have the universal approximation property for any finite r and thus cannot converge to the desired function even with an arbitrarily large number of hidden units but by including a bias term in each linear projection of the predictor variables ppl networks can regain these capabilities independent of the exact choice of r experimentally it is shown in this paper that this modification increases the rate of convergence with respect to the number of hidden units improves the generalization performance and makes it less sensitive to the setting of r finally we apply ppl to chaotic time series prediction and obtain superior results compared with the cascade correlation architecture

static spherically symmetric solutions of the so 5 einstein yang mills equations globally regular ie asymptotically flat and regular interior spher ically symmetric and localised particle like solutions of the coupled ein stein yang mills eym equations with gauge group su 2 have been known for more than 20 years yet their properties are still not well understood spher ically symmetric yang mills fields are classified by a choice of isotropy genera tor and so 5 is distinguished as the simplest model with a non abelian resid ual little group su 2 u 1 and which admits globally regular particle like solutions we exhibit an algebraic gauge condition which normalises the resid ual gauge freedom to a finite number of discrete symmetries this generalises the well known reduction to the real magnetic potential w r t in the original su 2 ym model reformulating using gauge invariant polynomials dramat ically simplifies the system and makes numerical search techniques feasible we find three families of embedded su 2 eym equations within the so 5 system one of which was first detected only within the gauge invariant polyno mial reduced system numerical solutions representing mixtures of the three su 2 sub systems are found classified by a pair of positive integers

approximation dimension reduction and nonconvex optimization using linear superpositions of gaussians this paper concerns neural network approaches to function approximation and optimization using linear superposition of gaussians or what are popularly known as radial basis function rbf networks the problem of function approximation is one of estimating an underlying function f given samples of the form y sub i x sub i i 1 2 n with y sub i f x sub i when the dimension of the input is high and the number of samples small estimation of the function becomes difficult due to the sparsity of samples in local regions the authors find that this problem of high dimensionality can be overcome to some extent by using linear transformations of the input in the gaussian kernels such transformations induce intrinsic dimension reduction and can be exploited for identifying key factors of the input and for the phase space reconstruction of dynamical systems without explicitly computing the dimension and delay they present a generalization that uses multiple linear projections onto scalars and successive rbf networks mlprbf that estimate the function based on these scaler values they derive some key properties of rbf networks that provide suitable grounds for implementing efficient search strategies for nonconvex optimization within the same framework

application of projection pursuit learning to boundary detection and deblurring in images abstract projection pursuit learning networks pplns have been used in many fields of research but have not been widely used in image processing in this paper we demonstrate how this highly promising technique may be used to connect edges and produce continuous boundaries we also propose the application of ppln to deblurring a degraded image when little or no a priori information about the blur is available the ppln was successful at developing an inverse blur filter to enhance blurry images theory and background information on projection pursuit regression ppr and ppln are also presented

invariant polynomials and molien functions the molien function associated with a finite dimensional representation of a compact lie group is a useful tool in representation theory because it acts as a generating function for counting the number of invariant polynomials on the representation space the main purpose of this paper is to introduce a more general and apparently new generating function which in the same sense counts not only the number of invariant real polynomials in a real representation or the number of invariant complex polynomials in a complex representation as a function of their degree but encodes the number of invariant real polynomials in a complex representation as a function of their bidegree the first and second component of this bidegree being the number of variables in which the polynomial is holomorphic and antiholomorphic respectively this is obviously an additional and non trivial piece of information for representations which are truly complex i e not self conjugate or are pseudo real but it provides ad

predicting the future advantages of semilocal units in investigating gaussian radial basis function rbf networks for their ability to model nonlinear time series we have found that while rbf networks are much faster than standard sigmoid unit backpropagation for low dimensional problems their advantages diminish in high dimensional input spaces this is particularly troublesome if the input space contains irrelevant variables we suggest that this limitation is due to the localized nature of rbfs to gain the advantages of the highly nonlocal sigmoids and the speed advantages of rbfs we propose a particular class of semilocal activation functions that is a natural interpolation between these two families we present evidence that networks using these gaussian bar units avoid the slow learning problem of sigmoid unit networks and very importantly are more accurate than rbf networks in the presence of irrelevant inputs on the mackey glass and coupled lattice map problems the speedup over sigmoid networks is so dramatic that the difference in training time between rbf and gaussian bar networks is minor gaussian bar architectures that superpose composed gaussians gaussians of gaussians to approximate the unknown function have the best performance we postulate that an interesing behavior displayed by gaussian bar functions under gradient descent dynamics which we call automatic connection pruning is an important factor in the success of this representation

exploratory projection pursuit abstract a new projection pursuit algorithm for exploring multivariate data is presented that has both statistical and computational advantages over previous methods a number of practical issues concerning its application are addressed a connection to multivariate density estimation is established and its properties are investigated through simulation studies and application to real data the goal of exploratory projection pursuit is to use the data to find low one two or three dimensional projections that provide the most revealing views of the full dimensional data with these views the human gift for pattern recognition can be applied to help discover effects that may not have been anticipated in advance since linear effects are directly captured by the covariance structure of the variable pairs which are straightforward to estimate the emphasis here is on the discovery of nonlinear effects such as clustering or other general nonlinear associations among the variables although arbitrary

brief state estimation of continuous time radial basis function networks this paper presents state estimators for previously trained rbf networks based on a minimax approach the uncertainty class is characterized in terms of an approximation error vector minimizing the objective function over this uncertainty class is used to obtain state estimates the initial portion of the paper presents the derivation and properties of such a minimax state estimator for rbf networks in the later portion this is extended to the problem of minimax adaptive state estimation this approach is useful in the cases where the pretrained rbf network is to be used in state estimation since the latest information is used in improving estimates of the rbf weights a special class of estimators for the appended state estimation problem is also considered where a constant gain estimator matrix can be obtained this section gives sufficient conditions for the existence of such an observer and follows directly from the structure of the rbf networks finally example applications of the minimax and adaptive minimax estimation for nonlinear problems are presented the results show the accuracy of state estimation using the proposed minimax estimator

invariants equivariants and characters in symmetric bifurcation theory in the analysis of stability in bifurcation problems it is often assumed that the appropriate reduced equations are in normal form in the presence of symmetry the truncated normal form is an equivariant polynomial map therefore the determination of invariants and equivariants of the group of symmetries of the problem is an important step in general these are hard problems of invariant theory and in most cases they are tractable only through symbolic computer programs nevertheless it is desirable to obtain some of the information about invariants and equivariants without actually computing them for example the number of linearly independent homogeneous invariants or equivariants of a certain degree generating functions for these dimensions are generally known as molien functions we obtain formulae for the number of linearly independent homogeneous invariants or equivariants for hopf bifurcation in terms of characters we also show how to construct molien functions for invariants and equivariants for hopf bifurcation our results are then applied to the computation of the number of invariants and equivariants for hopf bifurcation for several finite groups and the continuous group o 3

robust filtering and prediction for linear systems with uncertain dynamics a game theoretic approach we examine the existence and behavior of game theoretic solutions for robust linear filters and predictors our basic uncertainty class includes m th order time varying discrete time systems with uncertain dynamics uncertain initial state covariance and uncertain nonstationary input and observation noise covariance our results include recursive kalman filter predictor realizations for the resulting robust procedures our approach is based on saddle point theory we emphasize the notion of a least favorable prior distribution for the uncertain parameter values to obtain a worst case design technique in this paper we highlight the role such distributions with finite support play in these decision models in particular we demonstrate that in these decision models the least favorable prior distribution is always discrete

algebraic model for the evolution of the genetic code a search for symmetries based on the compact simple lie algebras is performed to verify to what extent the genetic code is a manifestation of some underlying symmetry an exact continuous symmetry group cannot be found to reproduce the present genetic code however a unique approximate symmetry group is compatible with codon assignment for the fundamental amino acids and the termination codon in order to obtain the actual genetic code the symmetry must be slightly broken

gaussian networks for direct adaptive control a direct adaptive tracking control architecture is proposed and evaluated for a class of continuous time nonlinear dynamic systems for which an explicit linear parameterization of the uncertainty in the dynamics is either unknown or impossible the architecture uses a network of gaussian radial basis functions to adaptively compensate for the plant nonlinearities under mild assumptions about the degree of smoothness exhibit by the nonlinear functions the algorithm is proven to be globally stable with tracking errors converging to a neighborhood of zero a constructive procedure is detailed which directly translates the assumed smoothness properties of the nonlinearities involved into a specification of the network required to represent the plant to a chosen degree of accuracy a stable weight adjustment mechanism is determined using lyapunov theory the network construction and performance of the resulting controller are illustrated through simulations with example systems

game theory approach to finite time horizon optimal estimation in this game a measurement record is given and the first player looks for the best estimate of a prespecified combination of the system states in the presence of a hostile process noise signal and system initial condition that are applied by his adversary it turns out that the game possesses a saddle point solution which leads to an optimal smoothed estimate that is identical to the corresponding l sub 2 optimal estimate a similar game in which the estimate is restricted to be causal is formulated and solved this game provides for the first time a saddle point equilibrium interpretation to finite time h sub infinity optimal filtered estimation the two games are very closely related it is shown that in the first game the first player s strategy which is the optimal smoothed estimate is a linear fractional transformation of the h sub infinity optimal filter which applies a nonzero free contracting q parameter it therefore achieves a unity h sub infinity norm bound for the operator that relates the exogeneous signals to the estimation error

pattern classification using projection pursuit abstract this article discusses the adaptation of recently developed regression techniques to classifier design apart from finite sample effects projection pursuit pp regression may be used to model a desired response class as a sum of ridge functions according to a minimum expected squared error criterion this approach can be shown to furnish an optimal discriminant function which can satisfy the neyman pearson criterion over all possible thresholds basis function expansions are used instead of smoothed histograms to reduce computation since good approximation of a discriminant by a linear combination of moderate number of ridge functions may not be easy we introduce an improved method utilizing a nonlinear weighting function

differential invariants for symplectic lie algebras realized by boson operators the study of algebraic symplectic models in high energy physics and even in molecular genetics has received much attention in the past decades in this paper we first survey one known bosonic realization with n types of bosons operators in a 2j 1 dimensional space with j being the semi integer then using the general theory of differential invariants we determine two sets of sp 2j 1 differential invariants in one set when n 2j 1 all independent invariants of any order are obtained in a simple closed form when n 2j 1 we could present only particular matrix forms in the latter case the differential invariants up to the second order are explicitly presented for the symplectic algebras sp 2 sp 4 and sp 6 realized by up to three types of bosons as applications we present the simplest sp 2 invariant lagrangians null lagrangians new solutions to the helmholtz inverse problem and evolution equations which will be of fundamental importance in the construction of dynamical systems invariant under the symplectic group

on a conjecture of huber concerning the convergence of projection pursuit regression we generalize the projection pursuit procedure of friedman and stuetzle abstract version and prove strong convergence this answers a question of huber

function approximation fast convergence neural approach based on spectral analysis we propose a constructive approach to building single hidden layer neural networks for nonlinear function approximation using frequency domain analysis we introduce a spectrum based learning procedure that minimizes the difference between the spectrum of the training data and the spectrum of the network s estimates the network is built up incrementally during training and automatically determines the appropriate number of hidden units this technique achieves similar or better approximation with faster convergence times than traditional techniques such as backpropagation