a comprehensive evaluation of random vector functional link networks with randomly generated weights between input and hidden layers a random vector functional link network is a universal approximator for continuous functions on compact sets with fast learning property though it was proposed two decades ago the classification ability of this family of networks has not been fully investigated yet through a very comprehensive evaluation by using 121 uci datasets the effect of bias in the output layer direct links from the input layer to the output layer and type of activation functions in the hidden layer scaling of parameter randomization as well as the solution procedure for the output weights are investigated in this work surprisingly we found that the direct link plays an important performance enhancing role in rvfl while the bias term in the output neuron had no significant effect the ridge regression based closed form solution was better than those with moore penrose pseudoinverse instead of using a uniform randomization in 1 1 for all datasets tuning the scaling of the uniform randomization range for each dataset enhances the overall performance six commonly used activation functions were investigated in this work and we found that hardlim and sign activation functions degenerate the overall performance these basic conclusions can serve as general guidelines for designing rvfl networks based classifiers

syndrome decoding of reed solomon codes beyond half the minimum distance based on shift register synthesis in this paper a new approach for decoding low rate reed solomon codes beyond half the minimum distance is considered and analyzed the maximum error correcting radius coincides with the error correcting radius of the sudan algorithm published in 1997 however unlike the sudan algorithm the approach described here is not a list decoding algorithm and is not based on polynomial interpolation the algorithm in this paper is rather syndrome based like classical algebraic decoding algorithms the computational complexity of the new algorithm is of the same order as the complexity of the well known berlekamp massey algorithm to decode errors beyond half the minimum distance the new decoder is allowed to fail for some high weight error patterns with a very small probability

a semi supervised random vector functional link network based on the transductive framework semi supervised learning ssl is the problem of learning a function with only a partially labeled training set it has considerable practical interest in applications where labeled data is costly to obtain while unlabeled data is abundant one approach to ssl in the case of binary classification is inspired by work on transductive learning tl by vapnik it has been applied prevalently using support vector machines svm as the base learning algorithm giving rise to the so called transductive svm tr svm the resulting optimization problem however is highly non convex and complex to solve in this paper we propose an alternative semi supervised training algorithm based on the tl theory namely semi supervised random vector functional link rvfl network which is able to obtain state of the art performance while resulting in a standard convex optimization problem in particular we show that thanks to the characteristics of rvfls networks the resulting optimization problem can be safely approximated with a standard quadratic programming problem solvable in polynomial time a wide range of experiments validate our proposal as a comparison we also propose a semi supervised algorithm for rvfls based on the theory of manifold regularization

a unified view on known algebraic decoding algorithms and new decoding concepts known properties of cyclic codes are used to give a unified description of many classical decoding algorithms for reed solomon codes up to half the minimum distance this description allows also simplified proofs for these decoders further a novel decoding algorithm is derived using these properties directly and variants of a new error erasure decoding algorithm are given for decoding beyond half the minimum distance a basis of all solutions for decoding is derived this basis allows to use side information in order to decode beyond half the minimum distance other methods where this basis can be used are power decoding also known as virtual syndrome extension where additional equations are generated by taking powers of the received symbols and interleaved reed solomon codes the extended euclidean algorithm which calculates the greatest common divisor plays an essential role in many presented methods

valued preference structures abstract an approach to the axiomatics of valued preference modelling is suggested in the framework of this approach general definitions of strict preference indifference and incomparability relations associated with a valued preference relation are established via solving a system of functional equations any solution of this system satisfies most of the classical properties characterizations of important particular solutions are also proved

representability of binary relations through fuzzy numbers we analyse the representability of different classes of binary relations on a set by means of suitable fuzzy numbers in particular we show that symmetric triangular fuzzy numbers can be considered as the best codomain to represent interval orders we also pay attention to the representability of other classes of acyclic binary relations

a recursive algorithm for mixture of densities estimation recursive algorithms for the estimation of mixtures of densities have attracted a lot of attention in the last 10 years here an algorithm for recursive estimation is studied it complements existing approaches in the literature as it is based on conditions that are usually very weak for example the parameter space over which the mixture is taken does not need to be necessarily bounded the essence of the procedure is to combine density estimation via empirical characteristic function together with an iterative hilbert space approximation algorithm the conditions for consistency of the estimator are verified for three important statistical problems a simulation study is also included

social choice and lukasiewicz logic abstract an approach to the social choice problem based on multiple valued logic is suggested it is assumed that individual preference are classical binary relations whereas the social preference is a valued binary relation the role of lukasiewicz logic in modelling valued transitive relations is discussed we argue that this logic is a natural choice for contructing valued social preference the paper is concerned with arrow s theory of social welfare functions standard conditions of unrestricted domain pareto independence of irrelevant alternatives and nondictatorship are employed we show that certain means in binary relations spaces are social welfare functions satisfying conditions of the classical arrow s possibility theorem it is also shown that collective choice rules given by means are completely characterized by the conditions of unrestrict domain anomymity neutrality and positive responsiveness

on the optimality of neural network approximation using incremental algorithms the problem of approximating functions by neural networks using incremental algorithms is studied for functions belonging to a rather general class characterized by certain smoothness properties with respect to the l sub 2 norm we compute upper bounds on the approximation error where error is measured by the l sub q norm 1 spl les q spl les spl infin these results extend previous work applicable in the case q 2 and provide an explicit algorithm to achieve the derived approximation error rate in the range q spl les 2 near optimal rates of convergence are demonstrated a gap remains however with respect to a recently established lower bound in the case q 2 although the rates achieved are provably better than those obtained by optimal linear approximation extensions of the results from the l sub 2 norm to l sub p are also discussed a further interesting conclusion from our results is that no loss of generality is suffered using networks with positive hidden to output weights moreover explicit bounds on the size of the hidden to output weights are established which are sufficient to guarantee the established convergence rates

the unreasonable effectiveness of neural network approximation results concerning the approximation rates of neural networks are of particular interest to engineers the results reported in the literature have slow approximation rates o 1 spl radic m where m is the number of parameters in the neural network however many empirical studies report that neural network approximation is quite effective in practice we give an explanation of this unreasonable effectiveness by proving the existence of approximation schemes that converge at a rate of the order of 1 m sup 2 by using methods from number theory

efficient agnostic learning of neural networks with bounded fan in we show that the class of two layer neural networks with bounded fan in is efficiently learnable in a realistic extension to the probably approximately correct pac learning model in this model a joint probability distribution is assumed to exist on the observations and the learner is required to approximate the neural network which minimizes the expected quadratic error as special cases the model allows learning real valued functions with bounded noise learning probabilistic concepts and learning the best approximation to a target function that cannot be well approximated by the neural network the networks we consider have real valued inputs and outputs an unlimited number of threshold hidden units with bounded fan in and a bound on the sum of the absolute values of the output weights the number of computation steps of the learning algorithm is bounded by a polynomial in 1 spl epsiv 1 spl delta n and b where spl epsiv is the desired accuracy spl delta is the probability that the algorithm fails n is the input dimension and b is the bound on both the absolute value of the target which may be a random variable and the sum of the absolute values of the output weights in obtaining the result we also extended some results on iterative approximation of functions in the closure of the convex hull of a function class and on the sample complexity of agnostic learning with the quadratic loss function

error estimates for approximate optimization by the extended ritz method an alternative to the classical ritz method for approximate optimization is investigated in the extended ritz method sets of admissible solutions are approximated by their intersections with sets of linear combinations of all n tuples of functions from a given basis this alternative scheme called variable basis approximation includes functions computable by trigonometric polynomials with free frequencies free node splines neural networks and other nonlinear approximating families estimates of rates of approximate optimization by the extended ritz method are derived upper bounds on rates of convergence of suboptimal solutions to the optimal one are expressed in terms of the degree n of variable basis functions the modulus of continuity of the functional to be minimized the modulus of tikhonov well posedness of the problem and certain norms tailored to the type of basis the results are applied to convex best approximation and to kernel methods in machine learning

comments on stochastic choice of basis functions in adaptive function approximation and the functional link net with reply this paper includes some comments and amendments of the above mentioned paper by igelnik et al 1995 subsequently theorem 1 in the above mentioned paper has been revised the significant change of the original theorem is the space of the thresholds in the hidden layer the revised theorem says that the thresholds of hidden b sub 0 should be w sub 0 spl middot y sub 0 u sub 0 where w sub 0 spl alpha w spl circ sub 0 w spl circ sub 0 w spl circ sub 01 spl middot spl middot spl middot y sub 0d and u sub 0 be independent and uniformly distributed in v sup d 0 spl omega spl times spl omega spl omega sup d 1 i sup d and 2d spl omega 2d spl omega respectively in reply igelnik et al acknowledge that a factor of two was omitted in the statement of a trigonometric identity however the validity of the essential point of theorem 1 is unaltered

a generalization of the berlekamp massey algorithm for multisequence shift register synthesis with applications to decoding cyclic codes a generalization of the berlekamp massey algorithm is presented for synthesizing minimum length linear feedback shift registers for generating prescribed multiple sequences a more general problem is first considered that of finding the smallest initial set of linearly dependent columns in a matrix over an arbitrary field which includes the multisequence problem as a special case a simple iterative algorithm the fundamental iterative algorithm fia is presented for solving this problem the generalized algorithm is then derived through a refinement of the fia application of this generalized algorithm to decoding cyclic codes up to the hartmann tzeng ht bound and roos bound making use of multiple syndrome sequences is considered conditions for guaranteeing that the connection polynomial of the shortest linear feedback shift register obtained by the algorithm will be the error locator polynomial are determined with respect to decoding up to the ht bound and special cases of the roos bound

new list decoding algorithms for reed solomon and bch codes in this paper we devise a rational curve fitting algorithm and apply it to the list decoding of reed solomon and bose chaudhuri hocquenghen bch codes the resulting list decoding algorithms exhibit the following significant properties

a simple lemma on greedy approximation in hilbert space and convergence rates for projection pursuit regression and neural network training a general convergence criterion for certain iterative sequences in hilbert space is presented for an important subclass of these sequences estimates of the rate of convergence are given under very mild assumptions these results establish an 0 1 f4n nonsampling convergence rate for projection pursuit regression and neural network training where n represents the number of ridge functions neurons or coefficients in a greedy basis expansion

list decoding of crisscross errors in this paper list decoding of crisscross errors in arrays over finite fields is considered for this purpose the so called cover metric is used where the cover of a matrix is a set of rows and columns which contains all non zero elements of the matrix a johnson like upper bound on the maximum list size in the cover metric is derived showing that the list of codewords has polynomial size up to a certain radius furthermore a simple list decoding algorithm for a known optimal code construction is presented which decodes errors in the cover metric up to our upper bound these results reveal significant differences between the cover metric and the rank metric and show that the cover metric is more suitable for correcting crisscross errors

a constrained backpropagation approach for the adaptive solution of partial differential equations this paper presents a constrained backpropagation cprop methodology for solving nonlinear elliptic and parabolic partial differential equations pdes adaptively subject to changes in the pde parameters or external forcing unlike existing methods based on penalty functions or lagrange multipliers cprop solves the constrained optimization problem associated with training a neural network to approximate the pde solution by means of direct elimination as a result cprop reduces the dimensionality of the optimization problem while satisfying the equality constraints associated with the boundary and initial conditions exactly at every iteration of the algorithm the effectiveness of this method is demonstrated through several examples including nonlinear elliptic and parabolic pdes with changing parameters and nonhomogeneous terms

use of bias term in projection pursuit learning improves approximation and convergence properties in a regression problem one is given a multidimensional random vector x the components of which are called predictor variables and a random variable y called response a regression surface describes a general relationship between x and y a nonparametric regression technique that has been successfully applied to high dimensional data is projection pursuit regression ppr the regression surface is approximated by a sum of empirically determined univariate functions of linear combinations of the predictors projection pursuit learning ppl formulates ppr using a 2 layer feedforward neural network the smoothers in ppr are nonparametric whereas those in ppl are based on hermite functions of some predefined highest order r we demonstrate that ppl networks in the original form do not have the universal approximation property for any finite r and thus cannot converge to the desired function even with an arbitrarily large number of hidden units but by including a bias term in each linear projection of the predictor variables ppl networks can regain these capabilities independent of the exact choice of r experimentally it is shown in this paper that this modification increases the rate of convergence with respect to the number of hidden units improves the generalization performance and makes it less sensitive to the setting of r finally we apply ppl to chaotic time series prediction and obtain superior results compared with the cascade correlation architecture

neural network observer based optimal control for unknown nonlinear systems using adaptive dynamic programming in this paper an observer based optimal control scheme is developed for unknown nonlinear systems using adaptive dynamic programming adp algorithm first a neural network nn observer is designed to estimate system states then based on the observed states a neuro controller is constructed via adp method to obtain the optimal control in this design two nn structures are used a three layer nn is used to construct the observer which can be applied to systems with higher degrees of nonlinearity and without a priori knowledge of system dynamics and a critic nn is employed to approximate the value function the optimal control law is computed using the critic nn and the observer nn uniform ultimate boundedness of the closed loop system is guaranteed the actor critic and observer structures are all implemented in real time continuously and simultaneously finally simulation results are presented to demonstrate the effectiveness of the proposed control scheme

using linear programming to decode binary linear codes a new method is given for performing approximate maximum likelihood ml decoding of an arbitrary binary linear code based on observations received from any discrete memoryless symmetric channel the decoding algorithm is based on a linear programming lp relaxation that is defined by a factor graph or parity check representation of the code the resulting lp decoder generalizes our previous work on turbo like codes a precise combinatorial characterization of when the lp decoder succeeds is provided based on pseudocodewords associated with the factor graph our definition of a pseudocodeword unifies other such notions known for iterative algorithms including stopping sets irreducible closed walks trellis cycles deviation sets and graph covers the fractional distance d sub frac of a code is introduced which is a lower bound on the classical distance it is shown that the efficient lp decoder will correct up to spl lceil d sub frac 2 spl rceil 1 errors and that there are codes with d sub frac spl omega n sup 1 spl epsi an efficient algorithm to compute the fractional distance is presented experimental evidence shows a similar performance on low density parity check ldpc codes between lp decoding and the min sum and sum product algorithms methods for tightening the lp relaxation to improve performance are also provided

efficient estimation of neural weights by polynomial approximation it has been known for some years that the uniform density problem for forward neural networks has a positive answer any real valued continuous function on a compact subset of r sup d can be uniformly approximated by a sigmoidal neural network with one hidden layer we design here algorithms for efficient uniform approximation by a certain class of neural networks with one hidden layer which we call nearly exponential this class contains e g all networks with the activation functions 1 1 e sup t tanh t or e sup t spl and 1 in their hidden layers the algorithms flow from a theorem stating that such networks attain the order of approximation o n sup 1 d d being dimension and n the number of hidden neurons this theorem in turn is a consequence of a close relationship between neural networks of nearly exponential type and multivariate algebraic and exponential polynomials the algorithms need neither a starting point nor learning parameters they do not get stuck in local minima and the gain in execution time relative to the backpropagation algorithm is enormous the size of the hidden layer can be bounded analytically as a function of the precision required

discrete time neural network output feedback control of nonlinear discrete time systems in non strict form an adaptive neural network nn based output feedback controller is proposed to deliver a desired tracking performance for a class of discrete time nonlinear systems which are represented in non strict feedback form the nn backstepping approach is utilized to design the adaptive output feedback controller consisting of 1 an nn observer to estimate the system states and 2 two nns to generate the virtual and actual control inputs respectively the non causal problem encountered during the control design is overcome by using a dynamic nn which is constructed through a feedforward nn with a novel weight tuning law the separation principle is relaxed persistency of excitation condition pe is not needed and certainty equivalence principle is not used the uniformly ultimate boundedness uub of the closed loop tracking error the state estimation errors and the nn weight estimates is demonstrated though the proposed work is applicable for second order nonlinear discrete time systems expressed in non strict feedback form the proposed controller design can be easily extendable to an nth order nonlinear discrete time system

nonparametric estimation and classification using radial basis function nets and empirical risk minimization studies convergence properties of radial basis function rbf networks for a large class of basis functions and reviews the methods and results related to this topic the authors obtain the network parameters through empirical risk minimization the authors show the optimal nets to be consistent in the problem of nonlinear function approximation and in nonparametric classification for the classification problem the authors consider two approaches the selection of the rbf classifier via nonlinear function estimation and the direct method of minimizing the empirical error probability the tools used in the analysis include distribution free nonasymptotic probability inequalities and covering numbers for classes of functions

stochastic choice of basis functions in adaptive function approximation and the functional link net a theoretical justification for the random vector version of the functional link rvfl net is presented in this paper based on a general approach to adaptive function approximation the approach consists of formulating a limit integral representation of the function to be approximated and subsequently evaluating that integral with the monte carlo method two main results are 1 the rvfl is a universal approximator for continuous functions on bounded finite dimensional sets and 2 the rvfl is an efficient universal approximator with the rate of approximation error convergence to zero of order o c spl radic n where n is number of basis functions and with c independent of n similar results are also obtained for neural nets with hidden nodes implemented as products of univariate functions or radial basis functions some possible ways of enhancing the accuracy of multivariate function approximations are discussed

a functional link network with higher order statistics for signal enhancement a functional link network with higher order statistics is introduced for signal enhancement the proposed scheme uses the mean square error mse between higher order statistics of desired signals and filtered output as the learning criterion for training weights in the functional link network this is motivated by the fact that higher order statistics have a natural tolerance to gaussian and symmetrically distributed non gaussian noises results show that the performance of functional link network with higher order statistics is less sensitive to the selection of learning rates than the conventional functional link network and adaptive line enhancement it is also demonstrated that it can enhance signal more effectively under different noise levels for stationary and nonstationary gaussian noises

improved decoding of reed solomon and algebraic geometry codes given an error correcting code over strings of length n and an arbitrary input string also of length n the list decoding problem is that of finding all codewords within a specified hamming distance from the input string we present an improved list decoding algorithm for decoding reed solomon codes the list decoding problem for reed solomon codes reduces to the following curve fitting problem over a field f given n points x sub i spl middot y sub i sub i 1 sup n x sub i y sub i spl isin f and a degree parameter k and error parameter e find all univariate polynomials p of degree at most k such that y sub i p x sub i for all but at most e values of i spl isin 1 n we give an algorithm that solves this problem for e 1 3 where the result yields the first asymptotic improvement in four decades the algorithm generalizes to solve the list decoding problem for other algebraic codes specifically alternant codes a class of codes including bch codes and algebraic geometry codes in both cases we obtain a list decoding algorithm that corrects up to n spl radic n n d errors where n is the block length and d is the designed distance of the code the improvement for the case of algebraic geometry codes extends the methods of shokrollahi and wasserman see in proc 29th annu acm symp theory of computing p 241 48 1998 and improves upon their bound for every choice of n and d we also present some other consequences of our algorithm including a solution to a weighted curve fitting problem which may be of use in soft decision decoding algorithms for reed solomon codes

fuzzy rule based models with randomized development mechanisms abstract fuzzy rule based models have attracted attention because of their modular architectures well developed design methodologies and practices as well as interpretability aspects methods exploiting factors of randomness offer significant efficiency and implementation simplicity that are essential in numerous application areas in this study we propose an original development of fuzzy rule based models established with the aid of concepts of randomization algorithms several design strategies involving different random prototypes generation and basis functions approximation are studied we investigate performance aspects of randomized rule base and look at the performance versus the key components of the models such as the number of rules and the use of the randomized algorithms in the development furthermore a comparative study is offered to quantify the efficiency of randomized algorithms experimental studies are reported for a series of publicly available data sets to illustrate the effectiveness of the proposed method and discuss its main features

neural network controller development and implementation for spark ignition engines with high egr levels past research has shown substantial reductions in the oxides of nitrogen nox concentrations by using 10 25 exhaust gas recirculation egr in spark ignition si engines see dudek and sain 1989 however under high egr levels the engine exhibits strong cyclic dispersion in heat release which may lead to instability and unsatisfactory performance preventing commercial engines to operate with high egr levels a neural network nn based output feedback controller is developed to reduce cyclic variation in the heat release under high levels of egr even when the engine dynamics are unknown by using fuel as the control input a separate control loop was designed for controlling egr levels the stability analysis of the closed loop system is given and the boundedness of the control input is demonstrated by relaxing separation principle persistency of excitation condition certainty equivalence principle and linear in the unknown parameter assumptions online training is used for the adaptive nn and no offline training phase is needed this online learning feature and model free approach is used to demonstrate the applicability of the controller on a different engine with minimal effort simulation results demonstrate that the cyclic dispersion is reduced significantly using the proposed controller when implemented on an engine model that has been validated experimentally for a single cylinder research engine fitted with a modern four valve head ricardo engine experimental results at 15 egr indicate that cyclic dispersion was reduced 33 by the controller an improvement of fuel efficiency by 2 and a 90 drop in nox from stoichiometric operation without egr was observed moreover unburned hydrocarbons uhc drop by 6 due to nn control as compared to the uncontrolled scenario due to the drop in cyclic dispersion similar performance was observed with the controller on a different engine

a modified blahut algorithm for decoding reed solomon codes beyond half the minimum distance a modification of the blahut algorithm is proposed for decoding of reed solomon codes beyond half the minimum distance an effective method is offered for the searching of unknown discrepancies needed for analytical continuation of the berlekamp massey algorithm through two additional iterations this reduces the search time by 2 q 1 n n t 1 n t times compared with the blahut algorithm an architecture of a searcher for unknown discrepancies is given the coding gain of the proposed algorithm is shown for some practical codes

geometric upper bounds on rates of variable basis approximation in this paper approximation by linear combinations of an increasing number n of computational units with adjustable parameters such as perceptrons and radial basis functions is investigated geometric upper bounds on rates of convergence of approximation errors are derived the bounds depend on certain parameters specific for each function to be approximated the results are illustrated by examples of values of such parameters in the case of approximation by linear combinations of orthonormal functions

some extensions of radial basis functions and their applications in artificial intelligence abstract radial basis functions have recently found interesting applications in artificial intelligence and in particular in the problem of learning to perform a particular task from a set of examples however in many practical cases the radial basis functions method cannot be applied in a straight forward manner because it does not take into account some features that are typical of the problem of learning from examples in this paper we show some extensions of the standard theory introduced in order to deal with a large number of examples and with problems in which different variables play very different roles we present some examples and also point out some open problems

projection based approximation and a duality with kernel methods projection pursuit regression and kernel regression are methods for estimating a smooth function of several variables from noisy data obtained at scattered sites methods based on local averaging can perform poorly in high dimensions curse of dimensionality intuition and examples have suggested that projection based approaches can provide better fits for what sorts of regression functions is this true when and by how much do projection methods reduce the curse of dimensionality we make a start by focusing on the two dimensional problem and study the l 2 approximation error bias of the two procedures with respect to gaussian measure let ra stand for a certain ppr type approximation and ka for a particular kernel type approximation building on a simple but striking duality for polynomials we show that ra behaves significantly better than the minimax rate of approximation for radial functions while ka performs significantly better than the minimax rate for harmonic functions in fact the rate improvements carry over to large classes ra behaving very well for functions with enough angular smoothness oscillating slowly with angle while ka behaves very well for functions with enough laplacian smoothness oscillations averaging out locally the rate improvements matter they are equivalent to lowering the dimensionality of the problem for example for functions with nice tail behavior ra behaves as if the dimensionality of the problem were 1 5 rather than its nominal value 2 also ra and ka are complementary for a given function if one method offers a dimensionality reduction the other does not

neural approximations for multistage optimal control of nonlinear stochastic systems two main approximations are used to solve a nonlinear quadratic gaussian lqg optimal control problem the control law is assigned a given structure in which a finite number of parameters have to be determined to minimize the cost function the chosen structure is that of a multilayer feedforward neural network and the control law is given a limited memory the errors resulting front both assumptions are discussed simulation results show that the proposed method may constitute a simple and effective tool for solving to a sufficient degree of accuracy optimal control problems traditionally regarded as difficult ones

lattice dynamical wavelet neural networks implemented using particle swarm optimization for spatio temporal system identification in this brief by combining an efficient wavelet representation with a coupled map lattice model a new family of adaptive wavelet neural networks called lattice dynamical wavelet neural networks ldwnns is introduced for spatio temporal system identification a new orthogonal projection pursuit opp method coupled with a particle swarm optimization pso algorithm is proposed for augmenting the proposed network a novel two stage hybrid training scheme is developed for constructing a parsimonious network model in the first stage by applying the opp algorithm significant wavelet neurons are adaptively and successively recruited into the network where adjustable parameters of the associated wavelet neurons are optimized using a particle swarm optimizer the resultant network model obtained in the first stage however may be redundant in the second stage an orthogonal least squares algorithm is then applied to refine and improve the initially trained network by removing redundant wavelet neurons from the network an example for a real spatio temporal system identification problem is presented to demonstrate the performance of the proposed new modeling framework

an integral upper bound for neural network approximation complexity of one hidden layer networks is studied using tools from nonlinear approximation and integration theory for functions with suitable integral representations in the form of networks with infinitely many hidden units upper bounds are derived on the speed of decrease of approximation error as the number of network units increases these bounds are obtained for various norms using the framework of bochner integration results are applied to perceptron networks

the bayesian evidence scheme for regularizing probability density estimating neural networks training probability density estimating neural networks with the expectation maximization em algorithm aims to maximize the likelihood of the training set and therefore leads to overfitting for sparse data in this article a regularization method for mixture models with generalized linear kernel centers is proposed which adopts the bayesian evidence approach and optimizes the hyperparameters of the prior by type ii maximum likelihood this includes a marginalization over the parameters which is done by laplace approximation and requires the derivation of the hessian of the log likelihood function the incorporation of this approach into the standard training scheme leads to a modified form of the em algorithm which includes a regularization term and adapts the hyperparameters on line after each em cycle the article presents applications of this scheme to classification problems the prediction of stochastic time series and latent space models

h_ infty static output feedback control design for discrete time systems using reinforcement learning this paper provides necessary and sufficient conditions for the existence of the static output feedback opfb solution to the h infty control problem for linear discrete time systems it is shown that the solution of the static opfb h infty control is a nash equilibrium point furthermore a q learning algorithm is developed to find the h infty opfb solution online using data measured along the system trajectories and without knowing the system matrices this is achieved by solving a game algebraic riccati equation online and using the measured data a simulation example shows the effectiveness of the proposed method

on a family of connectives for fuzzy sets abstract in this paper we find the general solution of the functional equation s t x y t x n y x where s is a t conorm t is a t norm and n is a strong negation on the unit interval in particular the result yields a family of connectives for fuzzy sets

generalized single hidden layer feedforward networks for regression problems in this paper traditional single hidden layer feedforward network slfn is extended to novel generalized slfn gslfn by employing polynomial functions of inputs as output weights connecting randomly generated hidden units with corresponding output nodes the significant contributions of this paper are as follows 1 a primal gslfn p gslfn is implemented using randomly generated hidden nodes and polynomial output weights whereby the regression matrix is augmented by full or partial input variables and only polynomial coefficients are to be estimated 2 a simplified gslfn s gslfn is realized by decomposing the polynomial output weights of the p gslfn into randomly generated polynomial nodes and tunable output weights 3 both p and s gslfn are able to achieve universal approximation if the output weights are tuned by ridge regression estimators and 4 by virtue of the developed batch and online sequential ridge elm br elm and osr elm learning algorithms high performance of the proposed gslfns in terms of generalization and learning speed is guaranteed comprehensive simulation studies and comparisons with standard slfns are carried out on real world regression benchmark data sets simulation results demonstrate that the innovative gslfns using br elm and osr elm are superior to standard slfns in terms of accuracy training speed and structure compactness

approximation with random bases in this work we discuss the problem of selecting suitable approximators from families of parameterized elementary functions that are known to be dense in a hilbert space of functions we consider and analyze published procedures both randomized and deterministic for selecting elements from these families that have been shown to ensure the rate of convergence in l2 norm of order o 1 n where n is the number of elements we show that both randomized and deterministic procedures are successful if additional information about the families of functions to be approximated is provided in the absence of such additional information one may observe exponential growth of the number of terms needed to approximate the function and or extreme sensitivity of the outcome of the approximation to parameters implications of our analysis for applications of neural networks in modeling and control are illustrated with examples

fuzzy nonlinear regression analysis using a random weight network modeling a fuzzy in fuzzy out system where both inputs and outputs are uncertain is of practical and theoretical importance fuzzy nonlinear regression fnr is one of the approaches used most widely to model such systems in this study we propose the use of a random weight network rwn to develop a fnr model called fnrrwn where both the inputs and outputs are triangular fuzzy numbers unlike existing fnr models based on back propagation bp and radial basis function rbf networks fnrrwn does not require iterative adjustment of the network weights and biases instead the input layer weights and hidden layer biases of fnrrwn are selected randomly the output layer weights for fnrrwn are calculated analytically based on a derived updating rule which aims to minimize the integrated squared error between cut sets that correspond to the predicted fuzzy outputs and target fuzzy outputs respectively in fnrrwn the integrated squared error is solved approximately by riemann integral theory the experimental results show that the proposed fnrrwn method can effectively approximate a fuzzy in fuzzy out system fnrrwn obtains better prediction accuracy in a lower computational time compared with existing fnr models based on bp and rbf networks

representation of nonlinear random transformations by non gaussian stochastic neural networks the learning capability of neural networks is equivalent to modeling physical events that occur in the real environment several early works have demonstrated that neural networks belonging to some classes are universal approximators of input output deterministic functions recent works extend the ability of neural networks in approximating random functions using a class of networks named stochastic neural networks snn in the language of system theory the approximation of both deterministic and stochastic functions falls within the identification of nonlinear no memory systems however all the results presented so far are restricted to the case of gaussian stochastic processes sps only or to linear transformations that guarantee this property this paper aims at investigating the ability of stochastic neural networks to approximate nonlinear input output random transformations thus widening the range of applicability of these networks to nonlinear systems with memory in particular this study shows that networks belonging to a class named non gaussian stochastic approximate identity neural networks sainns are capable of approximating the solutions of large classes of nonlinear random ordinary differential transformations the effectiveness of this approach is demonstrated and discussed by some application examples

measurement theoretic justification of connectives in fuzzy set theory abstract the problem of representing intersection and union in fuzzy set theory is considered there are various proposals in the literature to model these concepts the possibility of using continuous triangular norms and conorms including min and max are taken up in a measurement theoretic setting the conditions are laid out to arrive at cardinal scales on which addition and multiplication are meaningful and critically discussed these conditions must either be accepted on normative grounds or must be empirically verified before the modeling process in order to see which operations are meaningful it is emphasized that the archimedean axiom and the existence of natural bounds are crucial in arriving at ratio and absolute scale representations

minimum complexity density estimation the authors introduce an index of resolvability that is proved to bound the rate of convergence of minimum complexity density estimators as well as the information theoretic redundancy of the corresponding total description length the results on the index of resolvability demonstrate the statistical effectiveness of the minimum description length principle as a method of inference the minimum complexity estimator converges to true density nearly as fast as an estimator based on prior knowledge of the true subclass of densities interpretations and basic properties of minimum complexity estimators are discussed some regression and classification problems that can be examined from the minimum description length framework are considered

on choosing rationally when preferences are fuzzy abstract in real life exact choices are induced by fuzzy preferences these choices eventually satisfy certain plausible rationality properties several alternative rules for generating exact choices from fuzzy weak preference relations are introduced and the extent to which exact choice sets generated by these rules satisfy fairly weak rationality conditions is studied max min transitivity is shown to be crucial for obtaining a positive result

on fuzzy strict preference indifference and incomparability relations abstract the existence of fuzzy models for strict preference indifference and incomparability satisfying all classical properties is established a particular class of such models is completely described

boosting with early stopping convergence and consistency boosting is one of the most significant advances in machine learning for classification and regression in its original and computationally flexible version boosting seeks to minimize empirically a loss function in a greedy fashion the resulting estimator takes an additive function form and is built iteratively by applying a base estimator or learner to updated samples depending on the previous iterations an unusual regularization technique early stopping is employed based on cv or a test set this paper studies numerical convergence consistency and statistical rates of convergence of boosting with early stopping when it is carried out over the linear span of a family of basis functions for general loss functions we prove the convergence of boosting s greedy optimization to the infinimum of the loss function over the linear span using the numerical convergence result we find early stopping strategies under which boosting is shown to be consistent based on i i d samples and we obtain bounds on the rates of convergence for boosting estimators simulation studies are also presented to illustrate the relevance of our theoretical results for providing insights to practical aspects of boosting as a side product these results also reveal the importance of restricting the greedy search step sizes as known in practice through the work of friedman and others moreover our results lead to a rigorous proof that for a linearly separable problem adaboost with e 0 step size becomes an l 1 margin maximizer when left to run to convergence

adaptive neural control of nonlinear mimo systems with time varying output constraints in this paper adaptive neural control is investigated for a class of unknown multiple input multiple output nonlinear systems with time varying asymmetric output constraints to ensure constraint satisfaction we employ a system transformation technique to transform the original constrained in the sense of the output restrictions system into an equivalent unconstrained one whose stability is sufficient to solve the output constraint problem it is shown that output tracking is achieved without violation of the output constraint more specifically we can shape the system performance arbitrarily on transient and steady state stages with the output evolving in predefined time varying boundaries all the time a single neural network whose weights are tuned online is used in our design to approximate the unknown functions in the system dynamics while the singularity problem of the control coefficient matrix is avoided without assumption on the prior knowledge of control input s bound all the signals in the closed loop system are proved to be semiglobally uniformly ultimately bounded via lyapunov synthesis finally the merits of the proposed controller are verified in the simulation environment

models of knowing and the investigation of dynamical systems abstract we present three distinct concepts of what constitutes a scientific understanding of a dynamical system the development of each of these paradigms has resulted in a significant expansion in the kind of system that can be investigated in particular the recently developed algorithmic modelling paradigm has allowed us to enlarge the domain of discourse to include complex real world processes that cannot necessarily be described by conventional differential equations

on the relationship between generalization error hypothesis complexity and sample complexity for radial basis functions feedforward networks together with their training algorithms are a class of regression techniques that can be used to learn to perform some task from a set of examples the question of generalization of network performance from a finite training set to unseen data is clearly of crucial importance in this article we first show that the generalization error can be decomposed into two terms the approximation error due to the insufficient representational capacity of a finite sized network and the estimation error due to insufficient information about the target function because of the finite number of samples we then consider the problem of learning functions belonging to certain sobolev spaces with gaussian radial basis functions using the above mentioned decomposition we bound the generalization error in terms of the number of basis functions and number of examples while the bound that we derive is specific for radial basis functions a number of observations deriving from it apply to any approximation technique our result also sheds light on ways to choose an appropriate network architecture for a particular problem and the kinds of problems that can be effectively solved with finite resources i e with a finite number of parameters and finite amounts of data

decision theoretic generalizations of the pac model for neural net and other learning applications we describe a generalization of the pac learning model that is based on statistical decision theory in this model the learner receives randomly drawn examples each example consisting of an instance x in x and an outcome y in y and tries to find a hypothesis h x a where h in h that specifies the appropriate action a in a to take for each instance x in order to minimize the expectation of a loss l y a here x y and a are arbitrary sets l is a real valued function and examples are generated according to an arbitrary joint distribution on x times y special cases include the problem of learning a function from x into y the problem of learning the conditional probability distribution on y given x regression and the problem of learning a distribution on x density estimation we give theorems on the uniform convergence of empirical loss estimates to true expected loss rates for certain hypothesis spaces h and show how this implies learnability with bounded sample size disregarding computational complexity as an application we give distribution independent upper bounds on the sample size needed for learning with feedforward neural networks our theorems use a generalized notion of vc dimension that applies to classes of real valued functions adapted from pollard s work and a notion of capacity and metric dimension for classes of functions that map into a bounded metric space supersedes 89 30 and 90 52 also in information and computation vol 100 no 1 september 1992

decentralized optimal control of a class of interconnected nonlinear discrete time systems by using online hamilton jacobi bellman formulation in this paper the direct neural dynamic programming technique is utilized to solve the hamilton jacobi bellman equation forward in time for the decentralized near optimal regulation of a class of nonlinear interconnected discrete time systems with unknown internal subsystem and interconnection dynamics while the input gain matrix is considered known even though the unknown interconnection terms are considered weak and functions of the entire state vector the decentralized control is attempted under the assumption that only the local state vector is measurable the decentralized nearly optimal controller design for each subsystem consists of two neural networks nns an action nn that is aimed to provide a nearly optimal control signal and a critic nn which evaluates the performance of the overall system all nn parameters are tuned online for both the nns by using lyapunov techniques it is shown that all subsystems signals are uniformly ultimately bounded and that the synthesized subsystems inputs approach their corresponding nearly optimal control inputs with bounded error simulation results are included to show the effectiveness of the approach

robust integral of neural network and error sign control of mimo nonlinear systems this paper presents a novel state feedback control scheme for the tracking control of a class of multi input multioutput continuous time nonlinear systems with unknown dynamics and bounded disturbances first the control law consisting of the robust integral of a neural network nn output plus sign of the tracking error feedback multiplied with an adaptive gain is introduced the nn in the control law learns the system dynamics in an online manner while the nn residual reconstruction errors and the bounded disturbances are overcome by the error sign signal since both of the nn output and the error sign signal are included in the integral the continuity of the control input is ensured the controller structure and the nn weight update law are novel in contrast with the previous effort and the semiglobal asymptotic tracking performance is still guaranteed by using the lyapunov analysis in addition the nn weights and all other signals are proved to be bounded simultaneously the proposed approach also relaxes the need for the upper bounds of certain terms which are usually required in the previous designs finally the theoretical results are substantiated with simulations

on a variational norm tailored to variable basis approximation schemes a variational norm associated with sets of computational units and used in function approximation learning from data and infinite dimensional optimization is investigated for sets gk obtained by varying a vector y of parameters in a fixed structure computational unit k y e g the set of gaussians with free centers and widths upper and lower bounds on the gk variation norms of functions having certain integral representations are given in terms of the 1 norms of the weighting functions in such representations families of functions for which the two norms are equal are described

deadzone compensation in motion control systems using neural networks a compensation scheme is presented for general nonlinear actuator deadzones of unknown width the compensator uses two neural networks nns one to estimate the unknown deadzone and another to provide adaptive compensation in the feedforward path the compensator nn has a special augmented form containing extra neurons whose activation functions provide a jump function basis set for approximating piecewise continuous functions rigorous proofs of closed loop stability for the deadzone compensator are provided and yield tuning algorithms for the weights of the two nns the technique provides a general procedure for using nns to determine the preinverse of an unknown right invertible function

on transitivity of strict preference relations abstract a general notion of a strict preference relation associated with a preference relation valued in a semilattice with difference is introduced if a preference relation is valued in a linear scale then the associated strict preference relation is always transitive in contrast for some nonlinear scales transitivity of a preference relation does not imply transitivity of the associated strict preference relation the identity x y z x z y z is characteristic for the class of lower semilattices with difference for which associated strict preference relations preserve transitivity of preference relations

local minimax learning of functions with best finite sample estimation error bounds applications to ridge and lasso regression boosting tree learning kernel machines and inverse problems optimal local estimation is formulated in the minimax sense for inverse problems and nonlinear regression this theory provides best mean squared finite sample error bounds for some popular statistical learning algorithms and also for several optimal improvements of other existing learning algorithms such as smoothing splines and kernel regularization the bounds and improved algorithms are not based on asymptotics or bayesian assumptions and are truly local for each query not depending on cross validating estimates at other queries to optimize modeling parameters results are given for optimal local learning of approximately linear functions with side information context using real algebraic geometry in particular finite sample error bounds are given for ridge regression and for a local version of lasso regression the new regression methods require only quadratic programming with linear or quadratic inequality constraints for implementation greedy additive expansions are then combined with local minimax learning via a change in metric an optimal strategy is presented for fusing the local minimax estimators of a class of experts providing optimal finite sample prediction error bounds from random forests local minimax learning is extended to kernel machines best local prediction error bounds for finite samples are given for tikhonov regularization the geometry of reproducing kernel hilbert space is used to derive improved estimators with finite sample mean squared error mse bounds for class membership probability in two class pattern classification problems a purely local cross validation free algorithm is proposed which uses fisher information with these bounds to determine best local kernel shape in vector machine learning finally a locally quadratic solution to the finite fourier moments problem is presented after reading the first three sections the reader may proceed directly to any of the subsequent applications sections

sub quadratic decoding of one point hermitian codes we present the first two sub quadratic complexity decoding algorithms for one point hermitian codes the first is based on a fast realization of the guruswami sudan algorithm using state of the art algorithms from computer algebra for polynomial ring matrix minimization the second is a power decoding algorithm an extension of classical key equation decoding which gives a probabilistic decoding algorithm up to the sudan radius we show how the resulting key equations can be solved by the matrix minimization algorithms from computer algebra yielding similar asymptotic complexities

reed solomon codes for correcting phased error bursts a code structure is introduced that represents a reed solomon rs code in two dimensional format based on this structure a novel approach to multiple error burst correction using rs codes is proposed for a model of phased error bursts where each burst can affect one of the columns in a two dimensional transmitted word it is shown that the bursts can be corrected using a known multisequence shift register synthesis algorithm it is further shown that the resulting codes posses nearly optimal burst correction capability under certain probability of decoding failure finally low complexity systematic encoding and syndrome computation algorithms for these codes are discussed the proposed scheme may also find use in decoding of different coding schemes based on rs codes such as product or concatenated codes

the unique role of the łukasiewicz triplet in the theory of fuzzified normal forms in fuzzy set theory the fuzzification of a crisp concept is not seldom rooted in a straightforward adjustment of the disjunctive or conjunctive boolean normal form of the underlying mathematical expression however the fuzzified normal forms obtained can rarely be considered as true normal forms in an extended logic or algebra they are to be considered as functions defined on 0 1 n for some n n 0 and taking values in the support 0 1 of a bl algebra 0 1 t i t 0 1 with t a continuous t norm and i t its residual implicator in this paper we clear out some misunderstandings concerning fuzzified normal forms and explore for which continuous de morgan triplets the disjunctive fuzzified normal form is smaller than or equal to the conjunctive fuzzified normal form furthermore we figure out to what extent their mutual distance depends on the original boolean function special attention is drawn to the lukasiewicz triplet as it is the only continuous de morgan triplet for which the difference between both fuzzified normal forms is independent of the underlying boolean function

dependence of computational models on input dimension tractability of approximation and optimization tasks the role of input dimension d is studied in approximating in various norms target sets of d variable functions using linear combinations of adjustable computational units results from the literature which emphasize the number n of terms in the linear combination are reformulated and in some cases improved with particular attention to dependence on d for worst case error upper bounds are given in the factorized form d n where is nonincreasing typically n n 1 2 target sets of functions are described for which the function is a polynomial some important cases are highlighted where decreases to zero as d for target functions extent e g the size of domains in rd where they are defined scale e g maximum norms of target functions and smoothness e g the order of square integrable partial derivatives may depend on d and the influence of such dimension dependent parameters on model complexity is considered results are applied to approximation and solution of optimization problems by neural networks with perceptron and gaussian radial computational units

incomplete interval valued fuzzy preference relations an interval valued preference relation is a preference structure that is used to describe uncertainty in complex decision making problems retrieving complete information from experts is improbable in real life scenarios discarding incomplete information leads to loss of important data in this paper we introduce an upper bound condition to deal with incomplete interval valued fuzzy preference relations with the help of this condition missing preferences are estimated such that they are expressible moreover the resultant complete relation is consistent in case if an expert is unable to abide by the proposed upper bound condition an algorithm is formulated to assist the expert in complying to the upper bound condition

a neural state estimator with bounded errors for nonlinear systems a neural state estimator is described acting on discrete time nonlinear systems with noisy measurement channels a sliding window quadratic estimation cost function is considered and the measurement noise is assumed to be additive no probabilistic assumptions are made on the measurement noise nor on the initial state novel theoretical convergence results are developed for the error bounds of both the optimal and the neural approximate estimators to ensure the convergence properties of the neural estimator a minimax tuning technique is used the approximate estimator can be designed offline in such a way as to enable it to process on line any possible measure pattern almost instantly

interval valued preference structures different languages that are offered to model vague preferences are reviewed and an interval valued language is proposed to resolve a particular difficulty encountered with other languages it is shown that interval valued languages are well defined for de morgan triples constructed by continuous triangular norms conorms and a strong negation function a new transitivity condition for vague preferences is suggested and its relationships to known transitivity conditions are established a complete characterization of intervalvalued preference structures is also provided

adaptive neural control of high order uncertain nonaffine systems a transformation to affine systems approach abstract this brief investigates the adaptive neural network nn control of a class of high order nonaffine nonlinear systems with completely unknown dynamics since the control terms appear within the unknown nonlinearity traditional control schemes and stability analysis are usually rendered extremely complicated our main contribution includes a novel system transformation that converts the nonaffine system into an affine system through a combination of a low pass filter and state transformation as a result the state feedback control of the nonaffine system can be viewed as the output feedback control of an affine system in normal form the transformed system becomes linear with respect to the new input while the traditional backstepping approach is not needed thus allowing the synthesis to be extremely simplified it is theoretically proven that all the signals in the closed loop system are uniformly ultimately bounded uub simulation results are provided to demonstrate the performance of the developed controller

multiple criteria decision making multiattribute utility theory the next ten years management science and decision science have grown exponentially since midcentury two closely related fields central to this growth are multiple criteria decision making mcdm and multiattribute utility theory maut this paper comments on the history of mcdm and maut and discusses topics we believe are important in their continued development and usefulness to management science over the next decade our aim is to identify exciting directions and promising areas for future research

approximate minimization of the regularized expected error over kernel models learning from data under constraints on model complexity is studied in terms of rates of approximate minimization of the regularized expected error functional for kernel models with an increasing number n of kernel functions upper bounds on such rates are derived the bounds are of the form a n b n where a and b depend on the regularization parameter and on properties of the kernel and of the probability measure defining the expected error as a special case estimates of rates of approximate minimization of the regularized empirical error are derived

rotation invariant t norm solutions of a system of functional equations the generalization of the disjunctive and conjunctive boolean normal forms to fuzzy set theory obtained by interpreting as a t norm as a t conorm and as a negator often provides a kind of standard fuzzification procedure a system of functional equations turns up if some functional independence of the difference between both generalizations is demanded in this paper we explore which de morgan triplets based on a left continuous t norm t solve this system imposing some extra continuity conditions on t these t norm solutions can be obtained by the rotation construction of jenei the cauchy equation plays a key role in the reasoning process

on a conjecture of huber concerning the convergence of projection pursuit regression we generalize the projection pursuit procedure of friedman and stuetzle abstract version and prove strong convergence this answers a question of huber

covering number bounds of certain regularized linear function classes recently sample complexity bounds have been derived for problems involving linear functions such as neural networks and support vector machines in many of these theoretical studies the concept of covering numbers played an important role it is thus useful to study covering numbers for linear function classes in this paper we investigate two closely related methods to derive upper bounds on these covering numbers the first method already employed in some earlier studies relies on the so called maurey s lemma the second method uses techniques from the mistake bound framework in online learning we compare results from these two methods as well as their consequences in some learning formulations

facts and figures on fuzzified normal forms the idea behind normal forms is to provide a standard representation or approximation of various kinds of functions in boolean logic for instance this amounts to expressing a given well formed formula wff in terms of the disjunction respectively conjunction of some conjunctions respectively disjunctions of several elementary ones interpreting these well known identical disjunctive and conjunctive normal forms in a poorer structure such as a kleene or de morgan algebra leads to a lower and upper approximation only of the corresponding identical normal forms and hence of the given wff in that poorer structure in this paper we address the question whether a similar interpretation of these normal forms in a bl algebra still provides lower and upper approximations of a given wff this question falls apart in two subquestions first are these interpretations comparable and second if so is the wff located in between the first question can be answered positively for certain bl algebras while the second one is answered negatively