growing radial basis neural networks merging supervised and unsupervised learning with network growth techniques this paper proposes a framework for constructing and training radial basis function rbf neural networks the proposed growing radial basis function grbf network begins with a small number of prototypes which determine the locations of radial basis functions in the process of training the grbf network gross by splitting one of the prototypes at each growing cycle two splitting criteria are proposed to determine which prototype to split in each growing cycle the proposed hybrid learning scheme provides a framework for incorporating existing algorithms in the training of grbf networks these include unsupervised algorithms for clustering and learning vector quantization as well as learning algorithms for training single layer linear neural networks a supervised learning scheme based on the minimization of the localized class conditional variance is also proposed and tested grbf neural networks are evaluated and tested on a variety of data sets with very satisfactory results

neural networks for functional approximation and system identification we construct generalized translation networks to uniformly approximate a class of nonlinear continuous functionals defined on l p 1 1 s for integer s 1 1 p 1 or c 1 1 s we obtain lower bounds on the possible order of approximation for such functionals in terms of any approximation process depending continuously upon a given number of parameters our networks almost achieve this order of approximation in terms of the number of parameters neurons involved in the network the training is simple and noniterative in particular we avoid any optimization such as that involved in the usual back propagation

the efficiency and consistency of approximations to the jackknife variance estimators abstract the problem considered is the computation reduction for general delete d jackknife variance estimators the delete d jackknife estimator was proved consistent shao and wu 1986 and in this article its mean squared error is shown to have order o n 2 where n is the sample size these properties are not shared by the traditional delete 1 jackknife in some situations use of the delete d jackknife however requires n d recomputations of a point estimate which increases rapidly as n and d increase using techniques from survey sampling a shortcut can be taken with evaluated only m times m n d the efficiency and consistency of the resulting jackknife sampling hybrid variance estimators jsve s are studied if m is chosen so that n m 0 the increase in mean squared error by using the jsve is relatively negligible for the consistency of jsve m is sufficient hence the jsve with m n can also be used to alleviate the computational burden for the delete 1 jackknife in th

jackknifing r estimators summary sufficient conditions are given for the consistency of the jackknife variance estimator for r estimators of location in the one and two sample problems in particular the jackknife is shown to produce strongly consistent estimates of the variance of the hodges lehmann estimator an efficient algorithm for computing the variance estimator is presented for the two sample sizes considered in this paper calculations for the jackknife estimator of the variance of the hodges lehmann estimator are about 50 times as fast as the bootstrap variance estimator with b 100 some monte carlo evidence of the small sample efficiency of this jackknife variance estimator is reported extensions of the results to r estimators in the linear model context are discussed

approximation analysis of feedforward regular fuzzy neural network with two hidden layers the closure fuzzy mapping is employed to analyze approximation capability of feedforward regular fuzzy neural networks fnns with two hidden layers the bridge to do that is the fuzzy valued bernstein polynomial by the fuzzy polynomial approximation theorem this paper establishes some equivalent conditions for the fuzzy functions that can be approximated by the regular fnns to any degree of accuracy thus universal approximation of regular fnns based on zadeh s extension principle and fuzzy arithmetic can be studied systematically finally a simulation example is used to demonstrate the constructive procedure of approximating fnns

jackknifing l statistics with smooth weight functions abstract the behavior of linear combinations of order statistics l statistics under jackknifing is discussed the asymptotic behavior of a jackknifed l statistic is described and the pseudovalue based variance estimate is shown to be consistent under moderate smoothness and a trimming condition on the weight function the results extend easily to functions of l statistics and thus answer a question posed in miller 1974 monte carlo results support small sample applicability of the large sample results for the construction of approximate confidence intervals

on the partitioning capabilities of feedforward neural networks with sigmoid nodes in this letter the capabilities of feedforward neural networks fnns on the realization and approximation of functions of the form g rl a which partition the rl space into polyhedral sets each one being assigned to one out of the c classes of a are investigated more specifically a constructive proof is given for the fact that fnns consisting of nodes having sigmoid output functions are capable of approximating any function g with arbitrary accuracy also the capabilities of fnns consisting of nodes having the hard limiter as output function are reviewed in both cases the two class as well as the multiclass cases are considered

asymptotic results in jackknifing nonsmooth functions of the sample mean vector the asymptotic behavior of jackknife estimators and jackknife variance estimators is investigated for nonsmooth functions of the sample mean vector an application of jackknifing a suitable estimator of the intrinsic diversity profile is also presented

deterministic error analysis of support vector regression and related regularized kernel methods we introduce a new technique for the analysis of kernel based regression problems the basic tools are sampling inequalities which apply to all machine learning problems involving penalty terms induced by kernels related to sobolev spaces they lead to explicit deterministic results concerning the worst case behaviour of e and svrs using these we show how to adjust regularization parameters to get best possible approximation orders for regression the results are illustrated by some numerical examples

universal approximation using radial basis function networks there have been several recent studies concerning feedforward networks and the problem of approximating arbitrary functionals of a finite number of real variables some of these studies deal with cases in which the hidden layer nonlinearity is not a sigmoid this was motivated by successful applications of feedforward networks with nonsigmoidal hidden layer units this paper reports on a related study of radial basis function rbf networks and it is proved that rbf networks having one hidden layer are capable of universal approximation here the emphasis is on the case of typical rbf networks and the results show that a certain class of rbf networks with the same smoothing factor in each kernel node is broad enough for universal approximation

training ν support vector classifiers theory and algorithms the support vector machine svm for classification proposed by scholkopf smola williamson and bartlett 2000 has the advantage of using a parameter on controlling the number of support vectors in this article we investigate the relation between svm and c svm in detail we show that in general they are two different problems with the same optimal solution set hence we may expect that many numerical aspects of solving them are similar however compared to regular c svm the formulation of svm is more complicated so up to now there have been no effective methods for solving large scale svm we propose a decomposition method for svm that is competitive with existing methods for c svm we also discuss the behavior of svm by some numerical experiments

universal approximation to nonlinear operators by neural networks with arbitrary activation functions and its application to dynamical systems the purpose of this paper is to investigate neural network capability systematically the main results are 1 every tauber wiener function is qualified as an activation function in the hidden layer of a three layered neural network 2 for a continuous function in s r sup 1 to be a tauber wiener function the necessary and sufficient condition is that it is not a polynomial 3 the capability of approximating nonlinear functionals defined on some compact set of a banach space and nonlinear operators has been shown and 4 the possibility by neural computation to approximate the output as a whole not at a fixed point of a dynamical system thus identifying the system

characterization of compact subsets of fuzzy sets a characterization of compact subsets is presented for the metric space of normal fuzzy convex fuzzy sets on the base space rn the metric for which is the supremum over the hausdorff distances between corresponding level sets it is shown that a closed subset is compact if and only if it is uniformly support bounded and the corresponding set of support functions is equileftcontinuous in the membership grade variable

regularization theory and neural networks architectures we had previously shown that regularization principles lead to approximation schemes that are equivalent to networks with one layer of hidden units called regularization networks in particular s

control oriented system identification a worst case deterministic approach in h sub infinity the authors formulate and solve two related control oriented system identification problems for stable linear shift invariant distributed parameter plants in each of these problems the assumed a priori information is minimal consisting only of a lower bound on the relative stability of the plant an upper bound on a certain gain associated with the plant and an upper bound on the noise level the first of these problems involves identification of a point sample of the plant frequency response from a noisy finite output time series obtained in response to an applied sinusoidal input with frequency corresponding to the frequency point of interest this problem leads naturally to the second problem which involves identification of the plant transfer function in h sub infinity from a finite number of noisy point samples of the plant frequency response concrete plans for identification algorithms are provided for each of these two problems

fuzzy input output controllers are universal approximators abstract in this paper we show that a fuzzy input output controller can approximate to any degree of accuracy a continuous fuzzy system this generalizes the known result that fuzzy controllers crisp input output can approximate continuous functions

differentiability of statistical functionals and consistency of the jackknife in statistical applications the unknown parameter of interest can frequently be defined as a functional 0 t f where f is an unknown population statistical inferences about 0 are usually made based on the statistic t fn where fn is the empirical distribution assessing t fn as an estimator of 0 or making large sample inferences usually requires a consistent estimator of the asymptotic variance of t fn asymptotic behaviour of the jackknife variance estimator is closely related to the smoothness of the functional t this paper studies the smoothness of t through the differentiability of t and establishes some general results for the consistency of the jackknife variance estimators the results are applied to some examples in which the statistics t fn are l m estimators and some test statistics

on the approximation by neural networks with bounded number of neurons in hidden layers abstract in the current note we show that a two hidden layer neural network with d inputs d neurons in the first hidden layer 2 d 2 neurons in the second hidden layer and with a specifically constructed sigmoidal and infinitely differentiable activation function can approximate any continuous multivariate function with arbitrary accuracy

stable sequential identification of continuous nonlinear dynamical systems by growing radial basis function networks this paper presents a sequential identification scheme for continuous nonlinear dynamical systems using neural networks the nonlinearities of the dynamical systems are assumed to be unknown the identification model is a gaussian radial basis function neural network that grows gradually to span the appropriate state space and of sufficient complexity to provide an approximation to the dynamical system the sequential identification algorithm for continuous dynamical nonlinear systems is developed in the continuous time framework instead of in discrete time the approach different from the conventional methods of optimizing a cost function attempts to ensure stability of the overall system while the neural network learns the system dynamics the stability and convergence of the overall identification scheme are guaranteed by parameter adjustment laws developed using the lyapunov synthesis approach to ensure that the modelling error can be reduced arbitrarily a one to one mapping is proposed so that th

efficient computation and model selection for the support vector regression in this letter we derive an algorithm that computes the entire solution path of the support vector regression svr we also propose an unbiased estimate for the degrees of freedom of the svr model which allows convenient selection of the regularization parameter

efficient algorithms for function approximation with piecewise linear sigmoidal networks this paper presents a computationally efficient algorithm for function approximation with piecewise linear sigmoidal nodes a one hidden layer network is constructed one node at a time using the well known method of fitting the residual the task of fitting an individual node is accomplished using a new algorithm that searches for the best fit by solving a sequence of quadratic programming problems this approach offers significant advantages over derivative based search algorithms e g backpropagation and its extensions unique characteristics of this algorithm include finite step convergence a simple stopping criterion solutions that are independent of initial conditions good scaling properties and a robust numerical implementation empirical results are included to illustrate these characteristics

universal approximation of multiple nonlinear operators by neural networks recently there has been interest in the observed capabilities of some classes of neural networks with fixed weights to model multiple nonlinear dynamical systems while this property has been observed in simulations open questions exist as to how this property can arise in this article we propose a theory that provides a possible mechanism by which this multiple modeling phenomenon can occur

approximation by neural networks with weights varying on a finite set of directions abstract approximation properties of the mlp multilayer feedforward perceptron model of neural networks have been investigated in a great deal of works over the last 30 years it has been shown that for a large class of activation functions a neural network can approximate arbitrarily well any given continuous function the most significant result on this problem belongs to leshno lin pinkus and schocken they proved that the necessary and sufficient condition for any single hidden layer network to have the u a p universal approximation property is that its activation function not be a polynomial some authors white stinchcombe ito and others showed that a single hidden layer perceptron with some bounded weights can also have the u a p thus the weights required for u a p are not necessary to be of an arbitrarily large magnitude but what if they are too restricted how can one learn approximation properties of networks with arbitrarily restricted set of weights the current paper makes a first step in solving this general problem we consider neural networks with sets of weights consisting of a finite number of directions our purpose is to characterize compact sets x in the d dimensional space such that the network can approximate any continuous function over x in a special case when weights vary only on two directions we give a lower bound for the approximation error and find a sufficient condition for the network to be a best approximation

neural network solution for an inverse problem associated with the dirichlet eigenvalues of the anisotropic laplace operator an innovative numerical method based on an artificial neural network is presented in order to solve an inverse problem associated with the calculation of the dirichlet eigenvalues of the anisotropic laplace operator using a set of predefined eigenvalues obtained by solving repeatedly the direct problem a radial basis neural network is designed with the purpose to find the appropriate components of the anisotropy matrix related to the laplace operator and thus solving the associated inverse problem the finite element method is used to solve the direct problem and to create the training set for the first radial basis neural network a nonlinear map of the dirichlet eigenvalues as a function of the anisotropy matrix is then obtained this nonlinear relationship is later inverted and refined by training a second radial basis neural network solving the aforementioned inverse problem some numerical examples are presented to prove the effectiveness of the introduced method

neural networks for optimal approximation of smooth and analytic functions we prove that neural networks with a single hidden layer are capable of providing an optimal order of approximation for functions assumed to possess a given number of derivatives if the activation function evaluated by each principal element satisfies certain technical conditions under these conditions it is also possible to construct networks that provide a geometric order of approximation for analytic target functions the permissible activation functions include the squashing function 1 e x 1 as well as a variety of radial basis functions our proofs are constructive the weights and thresholds of our networks are chosen independently of the target function we give explicit formulas for the coefficients as simple continuous linear functionals of the target function

rbf based technique for statistical demodulation of pathological tremor this paper presents an innovative technique based on the joint approximation capabilities of radial basis function rbf networks and the estimation capability of the multivariate iterated hilbert transform iht for the statistical demodulation of pathological tremor from electromyography emg signals in patients with parkinson s disease we define a stochastic model of the multichannel high density surface emg by means of the rbf networks applied to the reconstruction of the stochastic process characterizing the disease modeled by the multivariate relationships generated by the karhunen loeve transform in hilbert spaces next we perform a demodulation of the entire random field by means of the estimation capability of the multivariate iht in a statistical setting the proposed method is applied to both simulated signals and data recorded from three parkinsonian patients and the results show that the amplitude modulation components of the tremor oscillation can be estimated with signal to noise ratio close to 30 db with root mean square error for the estimates of the tremor instantaneous frequency additionally the comparisons with a large number of techniques based on all the combinations of the rbf extreme learning machine backpropagation support vector machine used in the first step of the algorithm and iht empirical mode decomposition multiband energy separation algorithm periodic algebraic separation and energy demodulation used in the second step of the algorithm clearly show the effectiveness of our technique these results show that the proposed approach is a potential useful tool for advanced neurorehabilitation technologies that aim at tremor characterization and suppression

analysis of approximation of continuous fuzzy functions by multivariate fuzzy polynomials for the given fuzzy valued function j rn f0 r the fuzzy valued bernstein polynomial is introduced to show the fact that continuous fuzzy valued functions can be approximately represented as the fuzzy valued polynomials on any compact set u rn some equivalent condition for the continuous fuzzy function f f0 r n f0 r that can be approximated to any degree of accuracy by fuzzy polynomials on arbitrary compact set u f0 r n are presented

shape adaptive radial basis functions radial basis functions for discrimination and regression have been used with some success in a wide variety of applications here we investigate the optimal choice for the form of the basis functions and present an iterative strategy for obtaining the function in a regression context using a conjugate gradient based algorithm together with a nonparametric smoother this is developed in a discrimination framework using the concept of optimal scaling results are presented for a range of simulated and real data sets

approximation capabilities of multilayer fuzzy neural networks on the set of fuzzy valued functions this paper first introduces a piecewise linear interpolation method for fuzzy valued functions based on this we present a concrete approximation procedure to show the capability of four layer regular fuzzy neural networks to perform approximation on the set of all d p continuous fuzzy valued functions this approach can also be used to approximate d continuous fuzzy valued functions an example is given to illustrate the approximation procedure

self organizing radial basis function network for real time approximation of continuous time dynamical systems real time approximators for continuous time dynamical systems with many inputs are presented these approximators employ a novel self organizing radial basis function rbf network which varies its structure dynamically to keep the prescribed approximation accuracy the rbfs can be added or removed online in order to achieve the appropriate network complexity for the real time approximation of the dynamical systems and to maintain the overall computational efficiency the performance of this variable structure rbf network approximator with both gaussian rbf grbf and raised cosine rbf rcrbf is analyzed the compact support of rcrbf enables faster training and easier output evaluation of the network than that of the network with grbf the proposed real time self organizing rbf network approximator is then employed to approximate both linear and nonlinear dynamical systems to illustrate the effectiveness of our proposed approximation scheme especially for higher order dynamical systems the uniform ultimate boundedness of the approximation error is proved using the second method of lyapunov

identification using feedforward networks this paper is concerned with the identification of an unknown nonlinear dynamic system when only the inputs and outputs are accessible for measurement specifically we investigate the use of feedforward neural networks as models for the input output behavior of such systems relying on the approximation capabilities of feedforward neural networks and under mild assumptions regarding the properties of the underlying nonlinear system it is shown that there exists a feedforward network that for almost all inputs an open and dense set will display the input output behavior of the system

a general theory for jackknife variance estimation the delete 1 jackknife is known to give inconsistent variance estimators for nonsmooth estimators such as the sample quantiles this well known deficiency can be rectified by using a more general jackknife with d the number of observations deleted depending on a smoothness measure of the point estimator our general theory explains why jackknife works or fails it also shows that i for sufficiently smooth estimators the jackknife variance estimators with bounded d are consistent and asymptotically unbiased and ii for nonsmooth estimators d has to go to infinity at a rate explicitly determined by a smoothness measure to ensure consistency and asymptotic unbiasedness improved results are obtained for several classes of estimators in particular for the sample p quantiles the jackknife variance estimators with d satisfying n 1 2 d rightarrow 0 and n d rightarrow infty are consistent and asymptotically unbiased

metric spaces of fuzzy sets two classes of metrics are introduced for spaces of fuzzy sets their equivalence is discussed and basic properties established a characterisation of compact and locally compact subsets is given in terms of boundedness and p mean equileft continuity and the spaces shown to be locally compact complete and separable metric spaces

universal approximations of continuous fuzzy valued functions by multi layer fuzzy neural networks abstract the fact that four layer feedforward regular fuzzy neural networks with sigmoid function in the first hidden layer are capable of approximately representing continuous fuzzy valued functions on any compact set of r is shown at first bernstein polynomials associated with fuzzy valued functions are employed to approximate continuous fuzzy valued function defined on a compact set secondly by the conclusions related to standard feedforward networks universal approximations of continuous fuzzy valued functions by regular fuzzy neural networks are obtained

using radial basis functions to approximate a function and its error bounds a novel network called the validity index network vi net is presented the vi net derived from radial basis function networks fits functions and calculates confidence intervals for its predictions indicating local regions of poor fit and extrapolation

identification of restoring forces in non linear vibration systems using fuzzy adaptive neural networks the fuzzy adaptive back propagation fabp algorithm which combines fuzzy theory with artificial neural network techniques is applied to the identification of restoring forces in non linear vibration systems simulated results show that the fabp algorithm is effective for the identification of dynamic systems the fabp algorithm not only increases the training speed of the network but also decreases the artificial interference of network parameters to a certain extent based upon the fabp algorithm an improved scheme with a mutation mechanism is presented in this paper the improved fuzzy adaptive bp ifabp algorithm extends the effectiveness and adaptivity of the fabp algorithm still further the successful estimation of simulated systems show that a feasible method of identification is provided which can be used to estimate the restoring forces in non linear vibrating systems quickly and effectively

can neural nets be universal approximators for fuzzy functions abstract we first argue that the extension principle is too computationally involved to be an efficient way for a computer to evaluate fuzzy functions we then suggest using cuts and interval arithmetic to compute the valued of fuzzy functions using the method of computing fuzzy functions we then show that neural nets are universal approximators for computable fuzzy functions when we only input non negative or non positive fuzzy numbers

a constructive method for multivariate function approximation by multilayer perceptrons mathematical theorems establish the existence of feedforward multilayered neural networks based on neurons with sigmoidal transfer functions that approximate arbitrarily well any continuous multivariate function however these theorems do not provide any hint on how to find the network parameters in practice it is shown how to construct a perceptron with two hidden layers for multivariate function approximation such a network can perform function approximation in the same manner as networks based on gaussian potential functions by linear combination of local functions

sugeno type controllers are universal controllers abstract we show that given any continuous real valued function on a compact set there is a sugeno type controller that will approximate this function to any degree of accuracy an application of this result to fuzzy control is presented which shows that this type of sugeno controller is a universal controller

fuzzy neural networks a survey abstract in this paper a fuzzy neural network will be a layered feedforward neural net that has fuzzy signals and or fuzzy weights we survey recent results on learning algorithms and applications for fuzzy neural networks

complete memory structures for approximating nonlinear discrete time mappings this paper introduces a general structure that is capable of approximating input output maps of nonlinear discrete time systems the structure is comprised of two stages a dynamical stage followed by a memoryless nonlinear stage a theorem is presented which gives a simple necessary and sufficient condition for a large set of structures of this form to be capable of modeling a wide class of nonlinear discrete time systems in particular we introduce the concept of a complete memory a structure with a complete memory dynamical stage and a sufficiently powerful memoryless stage is shown to be capable of approximating arbitrarily wide class of continuous causal time invariant approximately finite memory mappings between discrete time signal spaces furthermore we show that any bounded input bounded output time invariant causal memory structure has such an approximation capability if and only if it is a complete memory several examples of linear and nonlinear complete memories are presented the proposed complete memory structure provides a template for designing a wide variety of artificial neural networks for nonlinear spatiotemporal processing

fuzzy hyperbolic neural network with time varying delays in this paper the formula of fuzzy basis functions with time varying delays dfbf is firstly given based on the dfbf a novel fuzzy hyperbolic neural network with time varying delays dfhnn and its approximation properties are presented respectively by constructing an appropriate lyapunov krasovskii functional the stability of dfhnn is discussed and some new criteria concerning the global exponential stability of dfhnn are proposed finally three numerical examples are given to demonstrate the effectiveness of the obtained results

a novel radial basis function neural network for discriminant analysis a novel radial basis function neural network for discriminant analysis is presented in this paper in contrast to many other researches this work focuses on the exploitation of the weight structure of radial basis function neural networks using the bayesian method it is expected that the performance of a radial basis function neural network with a well explored weight structure can be improved as the weight structure of a radial basis function neural network is commonly unknown the bayesian method is therefore used in this paper to study this a priori structure two weight structures are investigated in this study i e a single gaussian structure and a two gaussian structure an expectation maximization learning algorithm is used to estimate the weights the simulation results showed that the proposed radial basis function neural network with a weight structure of two gaussians outperformed the other algorithms

new support vector algorithms we propose a new class of support vector algorithms for regression and classification in these algorithms a parameter lets one effectively control the number of support vectors while this can be useful in its own right the parameterization has the additional benefit of enabling us to eliminate one of the other free parameters of the algorithm the accuracy parameter epsilon in the regression case and the regularization constant c in the classification case we describe the algorithms give some theoretical results concerning the meaning and the choice of and report experimental results

approximation of nonlinear systems with radial basis function neural networks a technique for approximating a continuous function of n variables with a radial basis function rbf neural network is presented the method uses an n dimensional raised cosine type of rbf that is smooth yet has compact support the rbf network coefficients are low order polynomial functions of the input a simple computational procedure is presented which significantly reduces the network training and evaluation time storage space is also reduced by allowing for a nonuniform grid of points about which the rbfs are centered the network output is shown to be continuous and have a continuous first derivative when the network is used to approximate a nonlinear dynamic system the resulting system is bounded input bounded output stable for the special case of a linear system the rbf network representation is exact on the domain over which it is defined and it is optimal in terms of the number of distinct storage parameters required several examples are presented which illustrate the effectiveness of this technique

ridge polynomial networks this paper presents a polynomial connectionist network called ridge polynomial network rpn that can uniformly approximate any continuous function on a compact set in multidimensional input space r sup d with arbitrary degree of accuracy this network provides a more efficient and regular architecture compared to ordinary higher order feedforward networks while maintaining their fast learning property the ridge polynomial network is a generalization of the pi sigma network and uses a special form of ridge polynomials it is shown that any multivariate polynomial can be represented in this form and realized by an rpn approximation capability of the rpn s is shown by this representation theorem and the weierstrass polynomial approximation theorem the rpn provides a natural mechanism for incremental network growth simulation results on a surface fitting problem the classification of high dimensional data and the realization of a multivariate polynomial function are given to highlight the capability of the network in particular a constructive learning algorithm developed for the network is shown to yield smooth generalization and steady learning

analyses of regular fuzzy neural networks for approximation capabilities abstract by introducing a trapezoidal fuzzy function and using the properties of the function we set up a continuously increasing fuzzy function that cannot be arbitrarily closely approximated on a compact set of f 0 r by the regular fuzzy neural network rfnn thus the conclusions in buckley and hayashi fuzzy sets and systems 61 1994 43 51 are improved and the problem if the rfnn is the universal approximator to the class of continuously increasing fuzzy functions is solved finally we obtain the universal approximation to the extended fuzzy functions by rfnn

a measurement fusion method for nonlinear system identification using a cooperative learning algorithm identification of a general nonlinear noisy system viewed as an estimation of a predictor function is studied in this article a measurement fusion method for the predictor function estimate is proposed in the proposed scheme observed data are first fused by using an optimal fusion technique and then the optimal fused data are incorporated in a nonlinear function estimator based on a robust least squares support vector machine ls svm a cooperative learning algorithm is proposed to implement the proposed measurement fusion method compared with related identification methods the proposed method can minimize both the approximation error and the noise error the performance analysis shows that the proposed optimal measurement fusion function estimate has a smaller mean square error than the ls svm function estimate moreover the proposed cooperative learning algorithm can converge globally to the optimal measurement fusion function estimate finally the proposed measurement fusion method is applied to arma signal and spatial temporal signal modeling experimental results show that the proposed measurement fusion method can provide a more accurate model

on resampling methods for variance and bias estimation in linear models let g be a nonlinear function of the regression parameters beta in a heteroscedastic linear model and hat beta be the least squares estimator of beta we consider the estimation of the variance and bias of g hat beta as an estimator of g beta by using three resampling methods the weighted jackknife the unweighted jackknife and the bootstrap the asymptotic orders of the mean squared errors and biases of the resampling variance and bias estimators are given in terms of an imbalance measure of the model consistency of the resampling estimators is also studied the results indicate that the weighted jackknife variance and bias estimators are asymptotically unbiased and consistent and their mean squared errors are of order o n 2 if the imbalance measure converges to zero as the sample size n rightarrow infty furthermore based on large sample properties the weighted jackknife is better than the unweighted jackknife the bootstrap method is shown to be asymptotically correct only under a homoscedastic error model bias reduction a closely related problem is also discussed

approximations of continuous functionals by neural networks with application to dynamic systems the paper gives several strong results on neural network representation in an explicit form under very mild conditions a functional defined on a compact set in c a b or l sup p a b spaces of infinite dimensions can be approximated arbitrarily well by a neural network with one hidden layer the results are a significant development beyond earlier work where theorems of approximating continuous functions defined on a finite dimensional real space by neural networks with one hidden layer were given all the results are shown to be applicable to the approximation of the output of dynamic systems at any particular time

representation of nonlinear random transformations by non gaussian stochastic neural networks the learning capability of neural networks is equivalent to modeling physical events that occur in the real environment several early works have demonstrated that neural networks belonging to some classes are universal approximators of input output deterministic functions recent works extend the ability of neural networks in approximating random functions using a class of networks named stochastic neural networks snn in the language of system theory the approximation of both deterministic and stochastic functions falls within the identification of nonlinear no memory systems however all the results presented so far are restricted to the case of gaussian stochastic processes sps only or to linear transformations that guarantee this property this paper aims at investigating the ability of stochastic neural networks to approximate nonlinear input output random transformations thus widening the range of applicability of these networks to nonlinear systems with memory in particular this study shows that networks belonging to a class named non gaussian stochastic approximate identity neural networks sainns are capable of approximating the solutions of large classes of nonlinear random ordinary differential transformations the effectiveness of this approach is demonstrated and discussed by some application examples

leave one out bounds for support vector regression model selection minimizing bounds of leave one out errors is an important and efficient approach for support vector machine svm model selection past research focuses on their use for classification but not regression in this letter we derive various leave one out bounds for support vector regression svr and discuss the difference from those for classification experiments demonstrate that the proposed bounds are competitive with bayesian svr for parameter selection we also discuss the differentiability of leave one out bounds

accurate on line support vector regression batch implementations of support vector regression svr are inefficient when used in an on line setting because they must be retrained from scratch every time the training set is modified following an incremental support vector classification algorithm introduced by cauwenberghs and poggio 2001 we have developed an accurate on line support vector regression aosvr that efficiently updates a trained svr function whenever a sample is added to or removed from the training set the updated svr function is identical to that produced by a batch algorithm applications of aosvr in both on line and cross validation scenarios are presented in both scenarios numerical experiments indicate that aosvr is faster than batch svr algorithms with both cold and warm start

orthogonal least squares learning algorithm for radial basis function networks the radial basis function network offers a viable alternative to the two layer neural network in many applications of signal processing a common learning algorithm for radial basis function networks is based on first choosing randomly some data points as radial basis function centers and then using singular value decomposition to solve for the weights of the network such a procedure has several drawbacks and in particular an arbitrary selection of centers is clearly unsatisfactory the authors propose an alternative learning procedure based on the orthogonal least squares method the procedure chooses radial basis function centers one by one in a rational way until an adequate network has been constructed in the algorithm each selected center maximizes the increment to the explained variance or energy of the desired output and does not suffer numerical ill conditioning problems the orthogonal least squares learning strategy provides a simple and efficient means for fitting radial basis function networks this is illustrated using examples taken from two different signal processing applications

a note on the jackknife the bootstrap and the delta method estimators of bias and variance three methods for the nonparametric reduction of bias and estimation of variance are in common use the jackknife the delta method i e the method of statistical differentials and the bootstrap in recent work efron 1981 a b 1982 has compared these three methods numerically for the case of the sample correlation coefficient for present purposes we define the procedures as follows consider a statistic tn thought of as an estimator of an unknown parameter 0 we wish to study the possibility of bias reduction by estimating the bias and subtracting off the estimate and variance estimation for g tj considered as an estimator of g 0 throughout we assume that we observe x1 xn independent and identically distributed with common cumulative distribution function f let tn denote the abovementioned statistic computed based on x1 xn we assume tn to be symmetric in its arguments furthermore let t 1 denote the above mentioned statistic computed based upon the sample of size n 1 consisting of x1 xi1 xi 1 xn the jackknife estimator of the bias of g tn is then given as b n n 1 1 g t 1 g tn n and the jackknife variance estimator is just

nonlinear structural response prediction based on support vector machines a support vector machines svm based two stage method is proposed to simulate and predict the nonlinear dynamic response of structures in the first stage an autoregressive moving average with exogenous input armax model is used to represent the acceleration response as the output of a single input single output siso system and the least square method is used to estimate the model parameters with which the linear acceleration response of the system can be simulated and predicted then the linear velocity and displacement are estimated using numerical integration of the predicted acceleration in the second stage by using the predicted linear responses acceleration velocity and displacement and the excitation to construct the input vector the svm is used to approximate nonlinear mapping from the input vector to system output and the trained svm can be used to simulate and predict the nonlinear dynamic response conveniently the nonlinear dynamic responses of a duffing oscillator and a frame structure are simulated and predicted using the proposed method as well as the neural network based method the results demonstrate that the svm based method provides superior performance in generalization and accuracy and can be a powerful tool for nonlinear system simulation and prediction

separating the vertices of n cubes by hyperplanes and its application to artificial neural networks a new sufficient condition that a region be classifiable by a two layer feedforward network using threshold activation functions is found briefly it is either a convex polytope or that minus the removal of convex polytope from its interior or recursively the author refers to these sets as convex recursive deletion regions the proof of implementability exploits the equivalence of this problem with that of characterizing two set partitions of the vertices of a hypercube which are separable by a hyperplane for which a new result is obtained

efficient learning algorithms for three layer regular feedforward fuzzy neural networks a key step of using gradient descend methods to develop learning algorithms of a regular feedforward fuzzy neural network fnn is to differentiate max min functions which contain max spl or and min spl and operations the paper aims at several objectives first investigate further the differentiation of spl or spl and functions second employ general fuzzy numbers which include triangular and trapezoidal fuzzy numbers as special cases to define a three layer regular fnn the general fuzzy numbers related can be approximately determined by their corresponding finite level sets so we can approximately represent the input output i o relationship of the regular fnn as functions of the endpoints of all finite level sets third a fuzzy back propagation algorithm is presented and to speed up the convergence of the learning algorithm a fuzzy conjugate gradient algorithm for fuzzy weights and biases is developed furthermore the convergence of the algorithm is analyzed systematically finally some real simulations demonstrate the efficiency of our learning algorithms the regular fnn is applied to the approximate realization of fuzzy inference rules and fuzzy functions defined on given compact sets

training v support vector regression theory and algorithms we discuss the relation between e support vector regression e svr and support vector regression svr in particular we focus on properties that are different from those of c support vector classification c svc and support vector classification svc we then discuss some issues that do not occur in the case of classification the possible range of e and the scaling of target values a practical decomposition method for svr is implemented and computational experiments are conducted we show some interesting numerical observations specific to regression

face recognition using an enhanced independent component analysis approach this paper is concerned with an enhanced independent component analysis ica and its application to face recognition typically face representations obtained by ica involve unsupervised learning and high order statistics in this paper we develop an enhancement of the generic ica by augmenting this method by the fisher linear discriminant analysis lda hence its abbreviation fica the fica is systematically developed and presented along with its underlying architecture a comparative analysis explores four distance metrics as well as classification with support vector machines svms we demonstrate that the fica approach leads to the formation of well separated classes in low dimension subspace and is endowed with a great deal of insensitivity to large variation in illumination and facial expression the comprehensive experiments are completed for the facial recognition technology feret face database a comparative analysis demonstrates that fica comes with improved classification rates when compared with some other conventional approaches such as eigenface fisherface and the ica itself

existence and uniqueness results for neural network approximations some approximation theoretic questions concerning a certain class of neural networks are considered the networks considered are single input single output single hidden layer feedforward neural networks with continuous sigmoidal activation functions no input weights but with hidden layer thresholds and output layer weights specifically questions of existence and uniqueness of best approximations on a closed interval of the real line under mean square and uniform approximation error measures are studied a by product of this study is a reparametrization of the class of networks considered in terms of rational functions of a single variable this rational reparametrization is used to apply the theory of pade approximation to the class of networks considered in addition a question related to the number of local minima arising in gradient algorithms for learning is examined

orthogonal decomposition of finite population statistics and its applications to distributional asymptotics we study orthogonal decomposition of symmetric statistics based on samples drawn without replacement from finite populations several applications to finite population statistics are given we establish one term edge worth expansions for general asymptotically normal symmetric statistics prove an efron stein inequality and the consistency of the jackknife estimator of variance our expansions provide second order a s approximations to wu s jackknife histogram

nonlinear system modeling via knot optimizing b spline networks in using the b spline network for nonlinear system modeling owing to a lack of suitable theoretical results it is quite difficult to choose an appropriate set of knot points to achieve a good network structure for minimizing say a minimum error criterion in this paper a novel knot optimizing b spline network is proposed to approximate the general nonlinear system behavior the knot points are considered to be independent variables in the b spline network and are optimized together with the b spline expansion coefficients the simulated annealing algorithm with an appropriate search strategy is used as an optimization algorithm for the training process in order to avoid any possible local minima examples involving dynamic systems up to six dimensions in the input space to the network are solved by the proposed method to illustrate the effectiveness of this approach

fuzzy random variables abstract in this paper we define the concepts of fuzzy random variable and the expectation of a fuzzy random variable the new definition of expectation generalizes the integral of a set valued function we derive some properties of these new concepts by considering a suitable generalization of the hausdorff metric we derive the lebesgue dominated convergence type theorem

a jackknife approach to quantifying single trial correlation between covariance based metrics undefined on a single trial basis the quantification of covariance between neuronal activities functional connectivity requires the observation of correlated changes and therefore multiple observations the strength of such neuronal correlations may itself undergo moment by moment fluctuations which might e g lead to fluctuations in single trial metrics such as reaction time rt or may co fluctuate with the correlation between activity in other brain areas yet quantifying the relation between moment by moment co fluctuations in neuronal correlations is precluded by the fact that neuronal correlations are not defined per single observation the proposed solution quantifies this relation by first calculating neuronal correlations for all leave one out subsamples i e the jackknife replications of all observations and then correlating these values because the correlation is calculated between jackknife replications we address this approach as jackknife correlation jc first we demonstrate the equivalence of jc to conventional correlation for simulated paired data that are defined per observation and therefore allow the calculation of conventional correlation while the jc recovers the conventional correlation precisely alternative approaches like sorting and binning result in detrimental effects of the analysis parameters we then explore the case of relating two spectral correlation metrics like coherence that require multiple observation epochs where the only viable alternative analysis approaches are based on some form of epoch subdivision which results in reduced spectral resolution and poor spectral estimators we show that jc outperforms these approaches particularly for short epoch lengths without sacrificing any spectral resolution finally we note that the jc can be applied to relate fluctuations in any smooth metric that is not defined on single observations

approximation of fuzzy functions by regular fuzzy neural networks in this paper we investigate the ability of regular fuzzy neural networks to provide approximations to fuzzy functions since the operation of regular fuzzy neural networks is based on zadeh s extension principle we first present a level characterization of the zadeh s extensions of level continuous fuzzy valued functions and consider the continuity of these extensions on the basis of this we give characterizations of fuzzy functions which can be approximated by a class of four layer regular fuzzy neural networks according to supremum metric and level convergence

can fuzzy neural nets approximate continuous fuzzy functions abstract we first argue that regular fuzzy neural nets based on standard fuzzy arithmetic and the extension principle cannot be universal approximators this negative result is surprising since regular neural nets are universal approximators we then argue that hybrid fuzzy neural nets not necessarily based on standard fuzzy arithmetic can be universal approximators