ridge regression biased estimation for nonorthogonal problems in multiple regression it is shown that parameter estimates based on minimum residual sum of squares have a high probability of being unsatisfactory if not incorrect if the prediction vectors are not orthogonal proposed is an estimation procedure based on adding small positive quantities to the diagonal of x x introduced is the ridge trace a method for showing in two dimensions the effects of nonorthogonality it is then shown how to augment x x to obtain biased estimates with smaller mean square error

extending the functional equivalence of radial basis function networks and fuzzy inference systems we establish the functional equivalence of a generalized class of gaussian radial basis function rbfs networks and the full takagi sugeno model 1983 of fuzzy inference this generalizes an existing result which applies to the standard gaussian rbf network and a restricted form of the takagi sugeno fuzzy system the more general framework allows the removal of some of the restrictive conditions of the previous result

universal approximation using radial basis function networks there have been several recent studies concerning feedforward networks and the problem of approximating arbitrary functionals of a finite number of real variables some of these studies deal with cases in which the hidden layer nonlinearity is not a sigmoid this was motivated by successful applications of feedforward networks with nonsigmoidal hidden layer units this paper reports on a related study of radial basis function rbf networks and it is proved that rbf networks having one hidden layer are capable of universal approximation here the emphasis is on the case of typical rbf networks and the results show that a certain class of rbf networks with the same smoothing factor in each kernel node is broad enough for universal approximation

parsimonious extreme learning machine using recursive orthogonal least squares novel constructive and destructive parsimonious extreme learning machines cp and dp elm are proposed in this paper by virtue of the proposed elms parsimonious structure and excellent generalization of multiinput multioutput single hidden layer feedforward networks slfns are obtained the proposed elms are developed by innovative decomposition of the recursive orthogonal least squares procedure into sequential partial orthogonalization spo the salient features of the proposed approaches are as follows 1 initial hidden nodes are randomly generated by the elm methodology and recursively orthogonalized into an upper triangular matrix with dramatic reduction in matrix size 2 the constructive spo in the cp elm focuses on the partial matrix with the subcolumn of the selected regressor including nonzeros as the first column while the destructive spo in the dp elm operates on the partial matrix including elements determined by the removed regressor 3 termination criteria for cp and dp elm are simplified by the additional residual error reduction method and 4 the output weights of the slfn need not be solved in the model selection procedure and is derived from the final upper triangular equation by backward substitution both single and multi output real world regression data sets are used to verify the effectiveness and superiority of the cp and dp elm in terms of parsimonious architecture and generalization accuracy innovative applications to nonlinear time series modeling demonstrate superior identification results

the equivalence between fuzzy logic systems and feedforward neural networks demonstrates that fuzzy logic systems and feedforward neural networks are equivalent in essence first we introduce the concept of interpolation representations of fuzzy logic systems and several important conclusions we then define mathematical models for rectangular wave neural networks and nonlinear neural networks with this definition we prove that nonlinear neural networks can be represented by rectangular wave neural networks based on this result we prove the equivalence between fuzzy logic systems and feedforward neural networks this result provides us a very useful guideline when we perform theoretical research and applications on fuzzy logic systems neural networks or neuro fuzzy systems

on the optimality of neural network approximation using incremental algorithms the problem of approximating functions by neural networks using incremental algorithms is studied for functions belonging to a rather general class characterized by certain smoothness properties with respect to the l sub 2 norm we compute upper bounds on the approximation error where error is measured by the l sub q norm 1 spl les q spl les spl infin these results extend previous work applicable in the case q 2 and provide an explicit algorithm to achieve the derived approximation error rate in the range q spl les 2 near optimal rates of convergence are demonstrated a gap remains however with respect to a recently established lower bound in the case q 2 although the rates achieved are provably better than those obtained by optimal linear approximation extensions of the results from the l sub 2 norm to l sub p are also discussed a further interesting conclusion from our results is that no loss of generality is suffered using networks with positive hidden to output weights moreover explicit bounds on the size of the hidden to output weights are established which are sufficient to guarantee the established convergence rates

adaptive neural output feedback controller design with reduced order observer for a class of uncertain nonlinear siso systems an adaptive output feedback control is studied for uncertain nonlinear single input single output systems with partial unmeasured states in the scheme a reduced order observer roo is designed to estimate those unmeasured states by employing radial basis function neural networks and incorporating the roo into a new backstepping design an adaptive output feedback controller is constructively developed a prominent advantage is its ability to balance the control action between the state feedback and the output feedback in addition the scheme can be still implemented when all the states are not available the stability of the closed loop system is guaranteed in the sense that all the signals are semiglobal uniformly ultimately bounded and the system output tracks the reference signal to a bounded compact set a simulation example is given to validate the effectiveness of the proposed scheme

heat flow heat content and the isoparametric property let m be a riemannian manifold and omega a compact domain of m with smooth boundary we study the solution of the heat equation on omega having constant unit initial conditions and dirichlet boundary conditions the purpose of this paper is to study the geometry of domains for which at any fixed value of time the normal derivative of the solution heat flow is a constant function on the boundary we express this fact by saying that such domains have the constant flow property in constant curvature spaces known examples of such domains are given by geodesic balls and more generally by domains whose boundary is connected and isoparametric the question is are they all like that this problem is the analogous for the heat equation of the classical serrin s problem for harmonic domains in this paper we give an affirmative answer to the above question in fact we prove more generally that if a domain in an analytic riemannian manifold has the constant flow property then every component of its boundary is an isoparametric hypersurface for space forms we also relate the order of vanishing of the heat content with fixed boundary data with the constancy of the r mean curvatures of the boundary and with the isoparametric property finally we discuss the constant flow property in relation to other well known overdetermined problems involving the laplace operator like the serrin problem or the schiffer problem

smooth function approximation using neural networks an algebraic approach for representing multidimensional nonlinear functions by feedforward neural networks is presented in this paper the approach is implemented for the approximation of smooth batch data containing the function s input output and possibly gradient information the training set is associated to the network adjustable parameters by nonlinear weight equations the cascade structure of these equations reveals that they can be treated as sets of linear systems hence the training process and the network approximation properties can be investigated via linear algebra four algorithms are developed to achieve exact or approximate matching of input output and or gradient based training sets their application to the design of forward and feedback neurocontrollers shows that algebraic training is characterized by faster execution speeds and better generalization properties than contemporary optimization techniques

natural gradient learning algorithms for rbf networks radial basis function rbf networks are one of the most widely used models for function approximation and classification there are many strange behaviors in the learning process of rbf networks such as slow learning speed and the existence of the plateaus the natural gradient learning method can overcome these disadvantages effectively it can accelerate the dynamics of learning and avoid plateaus in this letter we assume that the probability density function pdf of the input and the activation function are gaussian first we introduce natural gradient learning to the rbf networks and give the explicit forms of the fisher information matrix and its inverse second since it is difficult to calculate the fisher information matrix and its inverse when the numbers of the hidden units and the dimensions of the input are large we introduce the adaptive method to the natural gradient learning algorithms finally we give an explicit form of the adaptive natural gradient learning algorithm and compare it to the conventional gradient descent method simulations show that the proposed adaptive natural gradient method which can avoid the plateaus effectively has a good performance when rbf networks are used for nonlinear functions approximation

stochastic choice of basis functions in adaptive function approximation and the functional link net a theoretical justification for the random vector version of the functional link rvfl net is presented in this paper based on a general approach to adaptive function approximation the approach consists of formulating a limit integral representation of the function to be approximated and subsequently evaluating that integral with the monte carlo method two main results are 1 the rvfl is a universal approximator for continuous functions on bounded finite dimensional sets and 2 the rvfl is an efficient universal approximator with the rate of approximation error convergence to zero of order o c spl radic n where n is number of basis functions and with c independent of n similar results are also obtained for neural nets with hidden nodes implemented as products of univariate functions or radial basis functions some possible ways of enhancing the accuracy of multivariate function approximations are discussed

upper bounds on the number of hidden neurons in feedforward networks with arbitrary bounded nonlinear activation functions it is well known that standard single hidden layer feedforward networks slfns with at most n hidden neurons including biases can learn n distinct samples x sub i t sub i with zero error and the weights connecting the input neurons and the hidden neurons can be chosen almost arbitrarily however these results have been obtained for the case when the activation function for the hidden neurons is the signum function this paper rigorously proves that standard single hidden layer feedforward networks slfns with at most n hidden neurons and with any bounded nonlinear activation function which has a limit at one infinity can learn n distinct samples x sub i t sub i with zero error the previous method of arbitrarily choosing weights is not feasible for any slfn the proof of our result is constructive and thus gives a method to directly find the weights of the standard slfns with any such bounded nonlinear activation function as opposed to iterative training algorithms in the literature

a hybrid constructive algorithm for single layer feedforward networks learning single layer feedforward networks slfns have been proven to be a universal approximator when all the parameters are allowed to be adjustable it is widely used in classification and regression problems the slfn learning involves two tasks determining network size and training the parameters most current algorithms could not be satisfactory to both sides some algorithms focused on construction and only tuned part of the parameters which may not be able to achieve a compact network other gradient based optimization algorithms focused on parameters tuning while the network size has to be preset by the user therefore trial and error approach has to be used to search the optimal network size because results of each trial cannot be reused in another trial it costs much computation in this paper a hybrid constructive hc algorithm is proposed for slfn learning which can train all the parameters and determine the network size simultaneously at first by combining levenberg marquardt algorithm and least square method a hybrid algorithm is presented for training slfn with fixed network size then with the hybrid algorithm an incremental constructive scheme is proposed a new randomly initialized neuron is added each time when the training entrapped into local minima because the training continued on previous results after adding new neurons the proposed hc algorithm works efficiently several practical problems were given for comparison with other popular algorithms the experimental results demonstrated that the hc algorithm worked more efficiently than those optimization methods with trial and error and could achieve much more compact slfn than those construction algorithms

generalized single hidden layer feedforward networks for regression problems in this paper traditional single hidden layer feedforward network slfn is extended to novel generalized slfn gslfn by employing polynomial functions of inputs as output weights connecting randomly generated hidden units with corresponding output nodes the significant contributions of this paper are as follows 1 a primal gslfn p gslfn is implemented using randomly generated hidden nodes and polynomial output weights whereby the regression matrix is augmented by full or partial input variables and only polynomial coefficients are to be estimated 2 a simplified gslfn s gslfn is realized by decomposing the polynomial output weights of the p gslfn into randomly generated polynomial nodes and tunable output weights 3 both p and s gslfn are able to achieve universal approximation if the output weights are tuned by ridge regression estimators and 4 by virtue of the developed batch and online sequential ridge elm br elm and osr elm learning algorithms high performance of the proposed gslfns in terms of generalization and learning speed is guaranteed comprehensive simulation studies and comparisons with standard slfns are carried out on real world regression benchmark data sets simulation results demonstrate that the innovative gslfns using br elm and osr elm are superior to standard slfns in terms of accuracy training speed and structure compactness

universal approximation using incremental constructive feedforward networks with random hidden nodes according to conventional neural network theories single hidden layer feedforward networks slfns with additive or radial basis function rbf hidden nodes are universal approximators when all the parameters of the networks are allowed adjustable however as observed in most neural network implementations tuning all the parameters of the networks may cause learning complicated and inefficient and it may be difficult to train networks with nondifferential activation functions such as threshold networks unlike conventional neural network theories this paper proves in an incremental constructive method that in order to let slfns work as universal approximators one may simply randomly choose hidden nodes and then only need to adjust the output weights linking the hidden layer and the output layer in such slfns implementations the activation functions for additive nodes can be any bounded nonconstant piecewise continuous functions g r r and the activation functions for rbf nodes can be any integrable piecewise continuous functions g r r and rg x dx 0 the proposed incremental method is efficient not only for sflns with continuous including nondifferentiable activation functions but also for slfns with piecewise continuous such as threshold activation functions compared to other popular methods such a new network is fully automatic and users need not intervene the learning process by manually tuning control parameters

functional equivalence between radial basis function networks and fuzzy inference systems it is shown that under some minor restrictions the functional behavior of radial basis function networks rbfns and that of fuzzy inference systems are actually equivalent this functional equivalence makes it possible to apply what has been discovered learning rule representational power etc for one of the models to the other and vice versa it is of interest to observe that two models stemming from different origins turn out to be functionally equivalent

generalization of adaptive neuro fuzzy inference systems the adaptive network based fuzzy inference systems anfis of jang 1993 is extended to the generalized anfis ganfis by proposing a generalized fuzzy model gfm and considering a generalized radial basis function grbf network the gfm encompasses both the takagi sugeno ts model and the compositional rule of inference cri model the conditions by which the proposed gfm converts to ts model or the cri model are presented the basis function in grbf is a generalized gaussian function of three parameters the architecture of the grbf network is devised to learn the parameters of gfm where the grbf network and gfm have been proved to be functionally equivalent it is shown that grbf network can be reduced to either the standard rbf or the hunt s rbf network the issue of the normalized versus the non normalized grbf networks is investigated in the context of ganfis an interesting property of symmetry on the error surface of grbf network is investigated the proposed ganfis is applied to the modeling of a multivariable system like stock market

adaptive neural output feedback tracking control for a class of uncertain discrete time nonlinear systems this brief studies an adaptive neural output feedback tracking control of uncertain nonlinear multi input multi output mimo systems in the discrete time form the considered mimo systems are composed of n subsystems with the couplings of inputs and states among subsystems in order to solve the noncausal problem and decouple the couplings it needs to transform the systems into a predictor form the higher order neural networks are utilized to approximate the desired controllers by using lyapunov analysis it is proven that all the signals in the closed loop system is the semi globally uniformly ultimately bounded and the output errors converge to a compact set in contrast to the existing results the advantage of the scheme is that the number of the adjustable parameters is highly reduced the effectiveness of the scheme is verified by a simulation example

polynomial based radial basis function neural networks p rbf nns realized with the aid of particle swarm optimization in this study we design polynomial based radial basis function neural networks p rbf nns based on a fuzzy inference mechanism the essential design parameters including learning rate momentum coefficient and fuzzification coefficient of the underlying clustering method are optimized by means of the particle swarm optimization the proposed p rbf nns dwell upon structural findings about training data that are expressed in terms of a partition matrix resulting from fuzzy clustering in this case being the fuzzy c means fcm the network is of functional nature as the weights between the hidden layer and the output are some polynomials the use of the polynomial weights becomes essential in capturing the nonlinear nature of data encountered in regression or classification problems from the perspective of linguistic interpretation the proposed network can be expressed as a collection of if then fuzzy rules the architecture of the networks discussed here embraces three functional modules reflecting the three phases of input output mapping realized in rule based architectures namely condition formation conclusion creation and aggregation the proposed classifier is applied to some synthetic and machine learning datasets and its results are compared with those reported in the previous studies