growing radial basis neural networks merging supervised and unsupervised learning with network growth techniques this paper proposes a framework for constructing and training radial basis function rbf neural networks the proposed growing radial basis function grbf network begins with a small number of prototypes which determine the locations of radial basis functions in the process of training the grbf network gross by splitting one of the prototypes at each growing cycle two splitting criteria are proposed to determine which prototype to split in each growing cycle the proposed hybrid learning scheme provides a framework for incorporating existing algorithms in the training of grbf networks these include unsupervised algorithms for clustering and learning vector quantization as well as learning algorithms for training single layer linear neural networks a supervised learning scheme based on the minimization of the localized class conditional variance is also proposed and tested grbf neural networks are evaluated and tested on a variety of data sets with very satisfactory results

time varying radial basis functions we introduce radial basis functions rbfs whose time varying coefficients determine not only the amplitude and position of each rbf but also their shape the intended use of these time varying rbfs tv rbfs is in the local in time representation of low dimensional approximations of functions that arise in solving spatiotemporal evolution problems in particular for time varying spatially localized solutions with a temporal translation component such as traveling waves modulated pulses or soliton like solutions of evolutionary differential equations this paper is restricted to the one dimensional spatial case we also present an algorithm that places the time varying rbfs tv rbfs over spatiotemporal data that may come from experiments from finely discretized pde simulations or even from multiscale particle based simulations it first approximates the function at a single time instant a temporal snapshot as a sum of rbfs using a novel weighted minimization that causes each rbf to primarily approximate one of the localized parts of the function it then extends that approximation to tv rbfs over a sequence of snapshots of the function at different times we conclude by discussing the potential uses of these tv rbfs

divergence statistics for testing uniform association in cross classifications in this paper we consider the problem of testing uniform association in cross classifications having ordered categories taking as test statistic one in the family proposed by conde and salicru j conde m salicru uniform association in contingency tables associated to csiszar divergence statistics and probability letters 37 1998 149 154 we consider two approximations to the null distribution of the test statistics in this family an estimation of the asymptotic null distribution and a bootstrap estimator we prove that both approximations are asymptotically equivalent to study their finite sample performance we carried out two simulation experiments whose results are presented from the simulations it can be concluded that the bootstrap estimator behaves much better than the estimated asymptotic null distribution

alpha divergence is unique belonging to both f divergence and bregman divergence classes a divergence measure between two probability distributions or positive arrays positive measures is a useful tool for solving optimization problems in optimization signal processing machine learning and statistical inference the csiszar f divergence is a unique class of divergences having information monotonicity from which the dual alpha geometrical structure with the fisher metric is derived the bregman divergence is another class of divergences that gives a dually flat geometrical structure different from the alpha structure in general csiszar gave an axiomatic characterization of divergences related to inference problems the kullback leibler divergence is proved to belong to both classes and this is the only such one in the space of probability distributions this paper proves that the alpha divergences constitute a unique class belonging to both classes when the space of positive measures or positive arrays is considered they are the canonical divergences derived from the dually flat geometrical structure of the space of positive measures

descartes rule of signs for radial basis function neural networks we establish versions of descartes rule of signs for radial basis function rbf neural networks the rbf rules of signs provide tight bounds for the number of zeros of univariate networks with certain parameter restrictions moreover they can be used to infer that the vapnik chervonenkis vc dimension and pseudodimension of these networks are no more than linear this contrasts with previous work showing that rbf neural networks with two or more input nodes have superlinear vc dimension the rules also give rise to lower bounds for network sizes thus demonstrating the relevance of network parameters for the complexity of computing with rbf neural networks

on conformable fractional calculus recently the authors khalil et al 2014 introduced a new simple well behaved definition of the fractional derivative called conformable fractional derivative in this article we proceed on to develop the definitions there and set the basic concepts in this new simple interesting fractional calculus the fractional versions of chain rule exponential functions gronwall s inequality integration by parts taylor power series expansions laplace transforms and linear differential systems are proposed and discussed

normal boundary intersection a new method for generating the pareto surface in nonlinear multicriteria optimization problems this paper proposes an alternate method for finding several pareto optimal points for a general nonlinear multicriteria optimization problem such points collectively capture the trade off among the various conflicting objectives it is proved that this method is independent of the relative scales of the functions and is successful in producing an evenly distributed set of points in the pareto set given an evenly distributed set of parameters a property which the popular method of minimizing weighted combinations of objective functions lacks further this method can handle more than two objectives while retaining the computational efficiency of continuation type algorithms this is an improvement over continuation techniques for tracing the trade off curve since continuation strategies cannot easily be extended to handle more than two objectives

universal approximation using radial basis function networks there have been several recent studies concerning feedforward networks and the problem of approximating arbitrary functionals of a finite number of real variables some of these studies deal with cases in which the hidden layer nonlinearity is not a sigmoid this was motivated by successful applications of feedforward networks with nonsigmoidal hidden layer units this paper reports on a related study of radial basis function rbf networks and it is proved that rbf networks having one hidden layer are capable of universal approximation here the emphasis is on the case of typical rbf networks and the results show that a certain class of rbf networks with the same smoothing factor in each kernel node is broad enough for universal approximation

relative information of type s csiszar s f divergence and information inequalities during past years dragomir has contributed a lot of work providing different kinds of bounds on the distance information and divergence measures in this paper we have unified some of his results using the relative information of type s and relating it with the csiszar s f divergence

the method of types information theory the method of types is one of the key technical tools in shannon theory and this tool is valuable also in other fields in this paper some key applications are presented in sufficient detail enabling an interested nonspecialist to gain a working knowledge of the method and a wide selection of further applications are surveyed these range from hypothesis testing and large deviations theory through error exponents for discrete memoryless channels and capacity of arbitrarily varying channels to multiuser problems while the method of types is suitable primarily for discrete memoryless models its extensions to certain models with memory are also discussed

a recursive local polynomial approximation method using dirichlet clouds and radial basis functions we present a recursive function approximation technique that does not require the storage of the arrival data stream our work is motivated by algorithms in stochastic optimization which require approximating functions in a recursive setting such as a stochastic approximation algorithm the unique collection of these features in this technique is essential for nonlinear modeling of large data sets where the storage of the data becomes prohibitively expensive and in circumstances where our knowledge about a given query point increases as new information arrives the algorithm presented here employs radial basis functions rbfs to provide locally adaptive parametric models such as linear models the local models are updated using recursive least squares and only store the statistical representative of the local approximations the resulting scheme is very fast and memory efficient without compromising accuracy in comparison to methods well accepted as the standard and some advanced techniques used for fu

results on residual renyi entropy of order statistics and record values this paper explores properties of the residual renyi entropy of some ordered random variables the residual renyi entropy of the kth order statistic from a continuous distribution function is represented in terms of the residual renyi entropy of the kth order statistic from uniform distribution the monotone behavior of the residual renyi entropy of order statistic under various conditions is discussed analogues results for the residual renyi entropy of record values are also given

measuring the quality of discrete representations of efficient sets in multiple objective mathematical programming one way of solving multiple objective mathematical programming problems is finding discrete representations of the efficient set a modified goal of finding good discrete representations of the efficient set would contribute to the practicality of vector maximization algorithms we define coverage uniformity and cardinality as the three attributes of quality of discrete representations and introduce a framework that includes these attributes in which discrete representations can be evaluated compared to each other and judged satisfactory or unsatisfactory by a decision maker we provide simple mathematical programming formulations that can be used to compute the coverage error of a given discrete representation our formulations are practically implementable when the problem under study is a multiobjective linear programming problem we believe that the interactive algorithms along with the vector maximization methods can make use of our framework and its tools

jensen tsallis divergence and atomic dissimilarity for ionized systems in conjugated spaces abstract a generalization of the original jensen shannon divergence jsd is presented in this work which gives rise to a non extensive one parameter divergence providing a powerful dissimilarity measure between electronic distributions the analysis performed in this study employs the jtd measure to compare one particle densities of neutral and ionized atomic systems that generalizes and improves some previous results based on other measures of divergence such an improvement mainly arises from the capability of jtd to modify by means of its order parameter the relative contribution of specific relevant regions of the atomic densities under comparison in both position and momentum spaces relevant information of the ionization processes attending to structural pattern and periodicity is found as well as the strong correlation between extremal values of the neutral cation jtd and those of the atomic ionization potential similar conclusions are obtained from processes involving an anion the analysis includes a study of the dependence of jtd on its order for fixed atomic couples

entropy and distance of random graphs with application to structural pattern recognition the notion of a random graph is formally defined it deals with both the probabilistic and the structural aspects of relational data by interpreting an ensemble of attributed graphs as the outcomes of a random graph we can use its lower order distribution to characterize the ensemble to reflect the variability of a random graph shannon s entropy measure is used to synthesize an ensemble of attributed graphs into the distribution of a random graph or a set of distributions we propose a distance measure between random graphs based on the minimum change of entropy before and after their merging when the ensemble contains more than one class of pattern graphs the synthesis process yields distributions corresponding to various classes this process corresponds to unsupervised learning in pattern classification using the maximum likelihood rule and the probability computed for the pattern graph based on its matching with the random graph distributions of different classes we can classify the pattern graph to a class

maximum entropy and conditional probability it is well known that maximum entropy distributions subject to appropriate moment constraints arise in physics and mathematics in an attempt to find a physical reason for the appearance of maximum entropy distributions the following theorem is offered the conditional distribution of x l given the empirical observation 1 n sum n i l h x i alpha where x 1 x 2 cdots are independent identically distributed random variables with common density g converges to f lambda x e lambda t h x g x suitably normalized where lambda is chosen to satisfy int f lambda x h x dx alpha thus the conditional distribution of a given random variable x is the normalized product of the maximum entropy distribution and the initial distribution this distribution is the maximum entropy distribution when g is uniform the proof of this and related results relies heavily on the work of zabell and lanford

efficient algorithm for training interpolation rbf networks with equally spaced nodes this brief paper proposes a new algorithm to train interpolation gaussian radial basis function rbf networks in order to solve the problem of interpolating multivariate functions with equally spaced nodes based on an efficient two phase algorithm recently proposed by the authors euclidean norm associated to gaussian rbf is now replaced by a conveniently chosen mahalanobis norm that allows for directly computing the width parameters of gaussian radial basis functions the weighting parameters are then determined by a simple iterative method the original two phase algorithm becomes a one phase one simulation results show that the generality of networks trained by this new algorithm is sensibly improved and the running time significantly reduced especially when the number of nodes is large

wiener s tauberian theorems for the fourier bessel transformation and uniform approximation by rbf networks of delsarte translates abstract we prove analogues of the wiener and wiener pitt tauberian theorems for the delsarte translation and the fourier bessel transformation through similar functional analytic techniques we give conditions on the kernel function or activation function for the family of radial basis function rbf neural networks obtained upon replacing the usual translation by the delsarte one with not necessarily the same smoothing factor in all kernel nodes to have the universal approximation property in a certain space of continuous functions endowed with the uniform norm

the polya information divergence extensions of sanov s theorem and the conditional limit theorem colt are established for a multicolor polya eggenberger pe urn sampling scheme giving the polya information divergence and a polya extension to the maximum relative entropy maxent method polya maxent includes the standard maxent as well as its variants used in bose einstein fermi dirac and intermediate acharya swamy statistics as special cases in the pe setting standard maxent is in general asymptotically inconsistent

a generalization of hh f divergence abstract the discrimination between two probability distributions is an impotent problem in 1991 lin ieee transactions on information theory 37 1 1991 introduced a novel class of information theoretic divergence measures based on the shannon entropy as a generalization of lin s divergence a new divergence called hermite hadamard hh f divergence based on lin s method of constructing the divergence was introduced by shioya and da te in 1995 in this paper we expand the applicability of hh f divergence by combining the properties of fractional calculus with hh f divergence and then introduce the concept of some fractional hh f divergences which are generalizations of the hh f divergence then some inequalities related to fractional hh f divergence are proposed

approximation capabilities of multilayer fuzzy neural networks on the set of fuzzy valued functions this paper first introduces a piecewise linear interpolation method for fuzzy valued functions based on this we present a concrete approximation procedure to show the capability of four layer regular fuzzy neural networks to perform approximation on the set of all d p continuous fuzzy valued functions this approach can also be used to approximate d continuous fuzzy valued functions an example is given to illustrate the approximation procedure

viewpoint based simplification using f divergences we propose a new viewpoint based simplification method for polygonal meshes driven by several f divergences such as kullback leibler hellinger and chi square these distances are a measure of discrimination between probability distributions the kullback leibler distance between the projected and the actual area distributions of the polygons in the scene already has been used as a measure of viewpoint quality in this paper we use the variation in those viewpoint distances to determine the error introduced by an edge collapse we apply the best half edge collapse as a decimation criterion the approximations produced by our method are close to the original model in terms of both visual and geometric criteria unlike many pure visibility driven methods our new approach does not completely remove hidden interiors in order to increase the visual quality of the simplified models this makes our approach more suitable for applications which require exact geometry tolerance but also require high visual quality

the analysis of categorical data from complex sample surveys chi squared tests for goodness of fit and independence in two way tables abstract the effect of stratification and clustering on the asymptotic distributions of standard pearson chi squared test statistics for goodness of fit simple hypothesis and independence in a two way contingency table denoted as x 2 and xi 2 respectively is investigated it is shown that both x 2 and xi 2 are asymptotically distributed as weighted sums of independent 1 2 random variables the weights are then related to the familiar design effects deffs used by survey samplers a simple correction to x 2 which requires only the knowledge of variance estimates or deffs for individual cells in the goodness of fit problem is proposed and empirical results on the performance of corrected x 2 provided empirical work on xi 2 indicated that the distortion of nominal significance level is substantially smaller with xi 2 than with x 2 some results under simple models for clustering are also given

geometrical analysis of physically allowed quantum cloning transformations for quantum cryptography the security of quantum key distribution qkd relies on the no cloning theorem which allows no to copy perfectly a quantum system an eavesdropping activity on the quantum channel perturbs the state of the quantum states which results in noise at the receiver the physical layer detection of the eavesdropping activity of the quantum channel requires tomography which is intractable in experiment an adequate and equivalent answer for the problem can be proposed through the logical layer we propose an efficient algorithmical tool to study the eavesdropping activity on the quantum channel and characterize the properties of a quantum cloning based attack for dv and cvqkd protocols the physically allowed quantum cloning transformations on a quantum system can be described in terms of information geometry we propose a computational geometrical method to analyze the cloning activity on the quantum channel and to characterize the noise properties the security analysis studies the dv discrete variable and cv continuous variable qkd schemes through the four state bb84 and six state dvqkd protocols and the two way cvqkd protocol the proposed geometrical method provides a useful tool to analyze the most powerful attacks against quantum cryptography and the effects of the physically allowed quantum cloning transformations

a radial boundary intersection aided interior point method for multi objective optimization abstract we propose a novel multi objective optimization technique combining non convex radial boundary intersection based decomposition with an interior point method which utilizes both line search and trust region steps suitable for non convex nonlinear optimization radial boundary intersection decomposes the multi objective optimization problem into subproblems which are concerned with finding the solutions closest to a reference point along equally spaced lines emanating radially outwards from the latter point the proposed approach is found to be able to generate good approximations of the pareto front including the periphery by generating a sufficiently diverse set of pareto optimal solutions the proposed method is extensively tested on a large number of recent benchmark problems and real world problems and the performance is found to be favorable in comparison to those of some of the cutting edge stochastic evolutionary optimization algorithms that are commonly used to solve non convex multi objective optimization problems

a derivative free trust region method for biobjective optimization we consider unconstrained black box biobjective optimization problems in which analytic forms of the objective functions are not available and function values can be obtained only through computationally expensive simulations we propose a new algorithm to approximate the pareto optimal solutions of such problems based on a trust region approach at every iteration we identify a trust region then sample and evaluate points from it to determine nondominated solutions in the trust region we employ a scalarization method to convert the two objective functions into one we construct and optimize quadratic regression models for the two original objectives and the converted single objective we then remove dominated points from the current pareto approximation and construct a new trust region around the most isolated point in order to explore areas that have not been visited we prove convergence of the method under general regularity conditions and present numerical results suggesting that the method effic

nonparametric estimation and classification using radial basis function nets and empirical risk minimization studies convergence properties of radial basis function rbf networks for a large class of basis functions and reviews the methods and results related to this topic the authors obtain the network parameters through empirical risk minimization the authors show the optimal nets to be consistent in the problem of nonlinear function approximation and in nonparametric classification for the classification problem the authors consider two approaches the selection of the rbf classifier via nonlinear function estimation and the direct method of minimizing the empirical error probability the tools used in the analysis include distribution free nonasymptotic probability inequalities and covering numbers for classes of functions

activation functions defined on higher dimensional spaces for approximation on compact sets with and without scaling let g be a slowly increasing function of locally bounded variation defined on rc 1 c d we investigate when g can be an activation function of the hidden layer units of three layer neural networks that approximate continuous functions on compact sets if the support of the fourier transform of g includes a converging sequence of points with distinct distances from the origin it can be an activation function without scaling if and only if the support of its fourier transform includes a point other than the origin it can be an activation function with scaling we also look for a condition on which an activation function can be used for approximation without rotation any nonpolynomial functions can be activation functions with scaling and many familiar functions such as sigmoid functions and radial basis functions can be activation functions without scaling with or without scaling some of them defined on rd can be used without rotation even if they are not spherically symmetric

neural networks with local receptive fields and superlinear vc dimension local receptive field neurons comprise such well known and widely used unit types as radial basis function rbf neurons and neurons with center surround receptive field we study the vapnik chervonenkis vc dimension of feedforward neural networks with one hidden layer of these units for several variants of local receptive field neurons we show that the vc dimension of these networks is superlinear in particular we establish the bound w log k for any reasonably sized network with w parameters and k hidden nodes this bound is shown to hold for discrete center surround receptive field neurons which are physiologically relevant models of cells in the mammalian visual system for neurons computing a difference of gaussians which are popular in computational vision and for standard rbf neurons a major alternative to sigmoidal neurons in artificial neural networks the resutlt for rbf neural networks is of particular interest since it answers a question that has been open for several years the results also give rise to lower bounds for networks with fixed input dimension regarding constants all bounds are larger than those known thus far for similar architectures with sigmoidal neurons the superlinear lower bounds contrast with linear upper bounds for single local receptive field neurons also derived here

a symmetric information divergence measure of the csiszar s f divergence class and its bounds a symmetric measure of information divergence is proposed this measure belongs to the class of csiszar s f divergences its properties are studied and bounds in terms of well known divergence measures obtained a numerical illustration is carried out to compare this measure with some known divergence measures

quantitative comparison of approximate solution sets for multicriteria optimization problems with weighted tchebycheff preference function we consider the problem of evaluating the quality of solution sets generated by heuristics for multiple objective combinatorial optimization problems we extend previous research on the integrated preference functional ipf which assigns a scalar value to a given discrete set of nondominated points so that the weighted tchebycheff function can be used as the underlying implicit value function this extension is useful because modeling the decision maker s value function with the weighted tchebycheff function reflects the impact of unsupported points when evaluating sets of nondominated points we present an exact calculation method for the ipf measure in this case for an arbitrary number of criteria we show that every nondominated point has its optimal weight interval for the weighted tchebycheff function accordingly all nondominated points and not only the supported points in a set contribute to the value of the ipf measure when using the weighted tchebycheff function two and three criteria numerical examples illustrate the desirable properties of the weighted tchebycheff function providing a richer measure than the original ipf based on a convex combination of objectives

on simultaneous approximations by radial basis function neural networks it is shown in this paper by a constructive method that any multivariate function and all its existing derivatives can be simultaneously approximated by a radial basis function rbf neural network where the assumptions on the rbfs are relatively mild by considering the translations of rbfs it is proved that an o m 1 2 order of simultaneous approximations to a certain class of functions can be achieved by using a translation network with just m neurons

a model to simplify 2d triangle meshes with irregular shapes we present a 2d triangle mesh simplification model which is able to produce high quality approximations of any original planar mesh regardless of the shape of the original mesh this method consists of two phases a self organizing algorithm and a triangulation algorithm the self organizing algorithm is an unsupervised incremental clustering algorithm which provides us a set of nodes representing the best approximation of the original mesh the triangulation algorithm reconstructs the simplified mesh from the planar points obtained by the self organizing training process some examples are detailed with the purpose of demonstrating the ability of the model to perform the task of simplifying an original mesh with irregular shape

layered neural networks with gaussian hidden units as universal approximations a neural network with a single layer of hidden units of gaussian type is proved to be a universal approximator for real valued maps defined on convex compact sets of rn

on some entropy functionals derived from renyi information divergence we consider the maximum entropy problems associated with renyi q entropy subject to two kinds of constraints on expected values the constraints considered are a constraint on the standard expectation and a constraint on the generalized expectation as encountered in nonextensive statistics the optimum maximum entropy probability distributions which can exhibit a power law behaviour are derived and characterized the renyi entropy of the optimum distributions can be viewed as a function of the constraint this defines two families of entropy functionals in the space of possible expected values general properties of these functionals including nonnegativity minimum convexity are documented their relationships as well as numerical aspects are also discussed finally we work out some specific cases for the reference measure q x and recover in a limit case some well known entropies

on global local artificial neural networks for function approximation we present a hybrid radial basis function rbf sigmoid neural network with a three step training algorithm that utilizes both global search and gradient descent training the algorithm used is intended to identify global features of an input output relationship before adding local detail to the approximating function it aims to achieve efficient function approximation through the separate identification of aspects of a relationship that are expressed universally from those that vary only within particular regions of the input space we test the effectiveness of our method using five regression tasks four use synthetic datasets while the last problem uses real world data on the wave overtopping of seawalls it is shown that the hybrid architecture is often superior to architectures containing neurons of a single type in several ways lower mean square errors are often achievable using fewer hidden neurons and with less need for regularization our global local artificial neural network gl ann is also seen to compare favorably with both perceptron radial basis net and regression tree derived rbfs a number of issues concerning the training of gl anns are discussed the use of regularization the inclusion of a gradient descent optimization step the choice of rbf spreads model selection and the development of appropriate stopping criteria

parameter estimation of sigmoid superpositions dynamical system approach superposition of sigmoid function over a finite time interval is shown to be equivalent to the linear combination of the solutions of a linearly parameterized system of logistic differential equations due to the linearity with respect to the parameters of the system it is possible to design an effective procedure for parameter adjustment stability properties of this procedure are analyzed

adaptive estimated maximum entropy distribution model the estimation of distribution algorithm eda model is an optimization procedure through learning and sampling a conditional probabilistic function the use of conditional density function permits multivariate dependency modelling which is not captured in a population based representation like the classical genetic algorithms the gaussian model is a simple and widely used model for density estimation however an assumption of normality is not realistic for many real life problems alternatively the maximum entropy model can be used which makes no assumption of a normal distribution one disadvantage of the maximum entropy model is the learning cost of its parameters this paper proposes an adaptive estimated maximum entropy distribution adaptive meed model which aims to reduce learning complexity of building a model adaptive meed exploits the fact that samples have a low average fitness in the early stage but they gradually converge to an optima towards the end of the search hence it is not necessary to inference the model with a full account of observed constraints in the early stage of the search the proposed model attempts to estimate the density function with a dynamic set of samples and active constraints in addition the proposed model includes a global sampling function to address the issue of a missing mutation operator the ergodic convergence properties of the proposed model are discussed with the markov chain analysis the preliminary experimental evaluation shows that the proposed model performs well against genetic algorithms on several clustering problems

algorithmic superactivation of asymptotic quantum capacity of zero capacity quantum channels the superactivation of zero capacity quantum channels makes it possible to use two zero capacity quantum channels with a positive joint capacity for their output currently we have no theoretical background to describe all possible combinations of superactive zero capacity channels hence there may be many other possible combinations in practice to discover such superactive zero capacity channel pairs we must analyze an extremely large set of possible quantum states channel models and channel probabilities there is still no extremely efficient algorithmic tool for this purpose this paper shows an efficient algorithmical method of finding such combinations our method can be a very valuable tool for improving the results of fault tolerant quantum computation and possible communication techniques over very noisy quantum channels

some properties of renyi entropy and renyi entropy rate in this paper we define the conditional renyi entropy and show that the so called chain rule holds for the renyi entropy then we introduce a relation for the rate of renyi entropy and use it to derive the rate of the renyi entropy for an irreducible aperiodic markov chain we also show that the bound for the renyi entropy rate is simply the shannon entropy rate

approximation with random bases in this work we discuss the problem of selecting suitable approximators from families of parameterized elementary functions that are known to be dense in a hilbert space of functions we consider and analyze published procedures both randomized and deterministic for selecting elements from these families that have been shown to ensure the rate of convergence in l2 norm of order o 1 n where n is the number of elements we show that both randomized and deterministic procedures are successful if additional information about the families of functions to be approximated is provided in the absence of such additional information one may observe exponential growth of the number of terms needed to approximate the function and or extreme sensitivity of the outcome of the approximation to parameters implications of our analysis for applications of neural networks in modeling and control are illustrated with examples

hermite hadamard and hermite hadamard fejer type inequalities for generalized fractional integrals abstract in this paper we obtain the hermite hadamard and hermite hadamard fejer type inequalities for fractional integrals which generalize the two familiar fractional integrals namely the riemann liouville and the hadamard fractional integrals into a single form we prove that in most cases we obtain the riemann liouville and the hadamard equivalence just by taking limits when a parameter 1 and 0 respectively

derivation of an amplitude of information in the setting of a new family of fractional entropies by generalizing the basic functional equation f xy f x f y in the form f b xy f b x f b y b 1 one can derive a family of solutions which are exactly the inverse of the mittag leffler function referred to as mittag leffler logarithm or logarithm of fractional order this result provides a new family of generalized informational entropies which are indexed by a parameter clearly related to fractals via fractional calculus and which is quite relevant in the presence in defect of observation the relation with shannon s entropy renyi s entropy and tsallis entropy is clarified and it is shown that tsallis generalized logarithm has a significance in terms of fractional calculus the case b 2 looks like directly relevant to amplitude of probability in quantum mechanics and provides an approach to the definition of amplitude of informational entropy one examines the kind of result one can so obtain in applying the maximum entropy principle in the presence of uncertain definition or fuzzy definition the mittag leffler function would be more relevant than the gaussian normal law to some extent this new formulation could be fully supported by the derivation of a new family of fractional fisher information

back to the future radial basis function network revisited radial basis function rbf networks are a classical family of algorithms for supervised learning the most popular approach for training rbf networks has relied on kernel methods using regularization based on a norm in a reproducing kernel hilbert space rkhs which is a principled and empirically successful framework in this paper we aim to revisit some of the older approaches to training the rbf networks from a more modern perspective specifically we analyze two common regularization procedures one based on the square norm of the coefficients in the network and another one using centers obtained by k k means clustering we show that both of these rbf methods can be recast as certain data dependent kernels we provide a theoretical analysis of these methods as well as a number of experimental results pointing out very competitive experimental performance as well as certain advantages over the standard kernel methods in terms of both flexibility incorporating of unlabeled data and computational complexity finally our results shed light on some impressive recent successes of using soft k k means features for image recognition and other tasks

contingency matrix theory statistical dependence in a contingency table chance discovery aims at understanding the meaning of functional dependency from the viewpoint of unexpected relations one of the most important observations is that such a chance is hidden under a huge number of coocurrencies extracted from a given data on the other hand conventional data mining methods are strongly dependent on frequencies and statistics rather than interestingness or unexpectedness this paper discusses some limitations of ideas of statistical dependence especially focusing on the formal characteristics of simpson s paradox from the viewpoint of linear algebra theoretical results show that such a simpson s paradox can be observed when a given contingency table as a matrix is not regular in other words the rank of a contingency matrix is not full thus data ordered evidence gives some limitations which should be compensated by human oriented reasoning

solving a system of nonlinear integral equations by an rbf network in this paper a novel learning strategy for radial basis function networks rbfn is proposed by adjusting the parameters of the hidden layer including the rbf centers and widths the weights of the output layer are adapted by local optimization methods a new local optimization algorithm based on a combination of the gradient and newton methods is introduced the efficiency of some local optimization methods to update the weights of rbfn is studied in solving systems of nonlinear integral equations

nonsmooth optimization via quasi newton methods we investigate the behavior of quasi newton algorithms applied to minimize a nonsmooth function f not necessarily convex we introduce an inex act line search that generates a sequence of nested intervals containing a set of points of nonzero measure that satisfy the armijo and wolfe conditions if f is absolutely continuous along the line furthermore the line search is guaranteed to terminate if f is semi algebraic it seems quite difficult to establish a convergence theorem for quasi newton methods applied to such general classes of functions so we give a care ful analysis of a special but illuminating case the euclidean norm in one variable using the inexact line search and in two variables assuming that the line search is exact in practice we find that when f is locally lipschitz and semi algebraic with bounded sublevel sets the bfgs broyden fletcher goldfarb shanno method with the inexact line search almost always generates sequences whose cluster points are clarke stationary and with function values converging r linearly to a clarke station ary value we give references documenting the successful use of bfgs in a variety of nonsmooth applications particularly the design of low order controllers for linear dynamical systems we conclude with a challenging open question

a note on divergences in many areas of neural computation like learning optimization estimation and inference suitable divergences play a key role in this note we study the conjecture presented by amari 2009 and find a counterexample to show that the conjecture does not hold generally moreover we investigate two classes of divergence zhang 2004 weighted f divergence and weighted divergence and prove that if a divergence is a weighted f divergence as well as a bregman divergence then it is a weighted divergence this result reduces in form to the main theorem established by amari 2009 when

on minimum fisher information distributions with restricted support and fixed variance fisher information is of key importance in estimation theory it also serves in inference problems as well as in the interpretation of many physical processes the mean squared estimation error for the location parameter of a distribution is bounded by the inverse of the fisher information associated with this distribution in this paper we look for minimum fisher information distributions with a restricted support more precisely we study the problem of minimizing the fisher information in the set of distributions with fixed variance defined on a bounded subset s of r or on the positive real line we show that the solutions of the underlying differential equation can be expressed in terms of whittaker functions then in the two considered cases we derive the explicit expressions of the solutions and investigate their behavior we also characterize the behavior of the minimum fisher information as a function of the imposed variance

precision and approximate flatness in artificial neural networks several of the major classes of artificial neural networks output functions are linear combinations of elements of approximately flat sets this gives a tool for understanding the precision problem as well as providing a rationale for mixing types of networks approximate flatness also helps explain the power of artificial neural network techniques relative to series regressions series regressions take linear combinations of flat sets while neural networks take linear combinations of the much larger class of approximately flat sets

generalizations of crooks and lin s results on jeffreys csiszar and jensen csiszar f divergences abstract in this paper jeffreys csiszar and jensen csiszar f divergences are introduced and studied some bounds of crooks and lin types for such divergences are provided to this end the concavity of the composition of monotone functions is discussed refinements of the original inequalities by crooks and popescu et al are given

simulation study of the tests of uniform association based on the power divergence in this paper a simulation study is presented to analyze the behavior of the family of test statistics proposed by conde and salicru j conde m salicru uniform association in contingency tables associated to csiszar divergence statistics and probability letters 37 1998 149 154 using the phi divergence measures that include as special case the power divergence n cressie t r c read multinomial goodness of fit tests journal of the royal statistic society series b 46 1984 440 464 for the analysis of uniform association between two classification processes based on the local odd ratios for the above test statistics the significance level and its power are evaluated for different sample sizes when we consider a 3 x 2 contingency table

reformulated radial basis neural networks trained by gradient descent this paper presents an axiomatic approach for constructing radial basis function rbf neural networks this approach results in a broad variety of admissible rbf models including those employing gaussian rbfs the form of the rbfs is determined by a generator function new rbf models can be developed according to the proposed approach by selecting generator functions other than exponential ones which lead to gaussian rbfs this paper also proposes a supervised learning algorithm based on gradient descent for training reformulated rbf neural networks constructed using the proposed approach a sensitivity analysis of the proposed algorithm relates the properties of rbfs with the convergence of gradient descent learning experiments involving a variety of reformulated rbf networks generated by linear and exponential generator functions indicate that gradient descent learning is simple easily implementable and produces rbf networks that perform considerably better than conventional rbf models trained by existing algorithms

divergence based vector quantization supervised and unsupervised vector quantization methods for classification and clustering traditionally use dissimilarities frequently taken as euclidean distances in this article we investigate the applicability of divergences instead focusing on online learning we deduce the mathematical fundamentals for its utilization in gradient based online vector quantization algorithms it bears on the generalized derivatives of the divergences known as freechet derivatives in functional analysis which reduces in finite dimensional problems to partial derivatives in a natural way we demonstrate the application of this methodology for widely applied supervised and unsupervised online vector quantization schemes including self organizing maps neural gas and learning vector quantization additionally principles for hyperparameter optimization and relevance learning for parameterized divergences in the case of supervised vector quantization are given to achieve improved classification accuracy

modeling multivariate time series on manifolds with skew radial basis functions we present an approach for constructing nonlinear empirical mappings from high dimensional domains to multivariate ranges we employ radial basis functions and skew radial basis functions for constructing a model using data that are potentially scattered or sparse the algorithm progresses iteratively adding a new function at each step to refine the model the placement of the functions is driven by a statistical hypothesis test that accounts for correlation in the multivariate range variables the test is applied on training and validation data and reveals nonstatistical or geometric structure when it fails at each step the added function is fit to data contained in a spatiotemporally defined local region to determine the parameters in particular the scale of the local model the scale of the function is determined by the zero crossings of the autocorrelation function of the residuals the model parameters and the number of basis functions are determined automatically from the given data and there is no need to initialize any ad hoc parameters save for the selection of the skew radial basis functions compactly supported skew radial basis functions are employed to improve model accuracy order and convergence properties the extension of the algorithm to higher dimensional ranges produces reduced order models by exploiting the existence of correlation in the range variable data structure is tested not just in a single time series but between all pairs of time series we illustrate the new methodologies using several illustrative problems including modeling data on manifolds and the prediction of chaotic time series

higher order statistics based radial basis function networks for signal enhancement in this paper a higher order statistics hos based radial basis function rbf network for signal enhancement is introduced in the proposed scheme higher order cumulants of the reference signal were used as the input of hos based rbf an hos based supervised learning algorithm with mean square error obtained from higher order cumulants of the desired input and the system output as the learning criterion was used to adapt weights the motivation is that the hos can effectively suppress gaussian and symmetrically distributed non gaussian noise the influence of a gaussian noise on the input of hos based rbf and the hos based learning algorithm can be mitigated simulated results indicate that hos based rbf can provide better performance for signal enhancement under different noise levels and its performance is insensitive to the selection of learning rates moreover the efficiency of hos based rbf under the nonstationary gaussian noise is stable

indexed families of functionals and gaussian radial basis functions we report on results concerning the capabilities of gaussian radial basis function networks in the setting of inner product spaces that need not be finite dimensional specifically we show that important indexed families of functionals can be uniformly approximated with the approximation uniform also with respect to the index applications are described concerning the classification of signals and the synthesis of reconfigurable classifiers

divergence measures based on the shannon entropy a novel class of information theoretic divergence measures based on the shannon entropy is introduced unlike the well known kullback divergences the new measures do not require the condition of absolute continuity to be satisfied by the probability distributions involved more importantly their close relationship with the variational distance and the probability of misclassification error are established in terms of bounds these bounds are crucial in many applications of divergence measures the measures are also well characterized by the properties of nonnegativity finiteness semiboundedness and boundedness

multiobjective evolutionary optimization of the size shape and position parameters of radial basis function networks for function approximation this paper presents a multiobjective evolutionary algorithm to optimize radial basis function neural networks rbfnns in order to approach target functions from a set of input output pairs the procedure allows the application of heuristics to improve the solution of the problem at hand by including some new genetic operators in the evolutionary process these new operators are based on two well known matrix transformations singular value decomposition svd and orthogonal least squares ols which have been used to define new mutation operators that produce local or global modifications in the radial basis functions rbfs of the networks the individuals in the population in the evolutionary procedure after analyzing the efficiency of the different operators we have shown that the global mutation operators yield an improved procedure to adjust the parameters of the rbfnns

approximation and radial basis function networks this paper concerns conditions for the approximation of functions in certain general spaces using radial basis function networks it has been shown in recent papers that certain classes of radial basis function networks are broad enough for universal approximation in this paper these results are considerably extended and sharpened

sup norm approximation bounds for networks through probabilistic methods we consider the problem of approximating a smooth target function and its derivatives by networks involving superpositions and translations of a fixed activation function the approximation is with respect to the sup norm and the rate is shown to be of order o n sup 1 2 that is the rate is independent of the dimension d the results apply to neural and wavelet networks and extend the work of barren see proc 7th yale workshop on adaptive and learning systems may 1992 and ibid vol 39 p 930 1993 the approach involves probabilistic methods based on central limit theorems for empirical processes indexed by classes of functions

approximating gaussian mixture model or radial basis function network with multilayer perceptron gaussian mixture models gmms and multilayer perceptron mlp are both popular pattern classification techniques this brief shows that a multilayer perceptron with quadratic inputs mlpq can accurately approximate gmms with diagonal covariance matrices the mapping equations between the parameters of gmm and the weights of mlpq are presented a similar approach is applied to radial basis function networks rbfns to show that rbfns with gaussian basis functions and euclidean norm can be approximated accurately with mlpq the mapping equations between rbfn and mlpq weights are presented there are well established training procedures for gmms such as the expectation maximization em algorithm the gmm parameters obtained by the em algorithm can be used to generate a set of initial weights of mlpq similarly a trained rbfn can be used to generate a set of initial weights of mlpq mlpq training can be continued further with gradient descent based methods which can lead to improvement in performance compared to the gmm or rbfn from which it is initialized thus the mlpq can always perform as well as or better than the gmm or rbfn

a note on reverse pinsker inequalities a simple method is shown to provide optimal variational bounds on f divergences with possible constraints on relative information extremums the known results are refined or proved to be optimal as particular cases

centroidal voronoi tessellations applications and algorithms a centroidal voronoi tessellation is a voronoi tessellation whose generating points are the centroids centers of mass of the corresponding voronoi regions we give some applications of such tessellations to problems in image compression quadrature finite difference methods distribution of resources cellular biology statistics and the territorial behavior of animals we discuss methods for computing these tessellations provide some analyses concerning both the tessellations and the methods for their determination and finally present the results of some numerical experiments