subgradient based neural networks for nonsmooth nonconvex optimization problems this paper presents a subgradient based neural network to solve a nonsmooth nonconvex optimization problem with a nonsmooth nonconvex objective function a class of affine equality constraints and a class of nonsmooth convex inequality constraints the proposed neural network is modeled with a differential inclusion under a suitable assumption on the constraint set and a proper assumption on the objective function it is proved that for a sufficiently large penalty parameter there exists a unique global solution to the neural network and the trajectory of the network can reach the feasible region in finite time and stay there thereafter it is proved that the trajectory of the neural network converges to the set which consists of the equilibrium points of the neural network and coincides with the set which consists of the critical points of the objective function in the feasible region a condition is given to ensure the convergence to the equilibrium point set in finite time moreover under suitable assumptions the coincidence between the solution to the differential inclusion and the ldquoslow solutionrdquo of it is also proved furthermore three typical examples are given to present the effectiveness of the theoretic results obtained in this paper and the good performance of the proposed neural network

neural networks for functional approximation and system identification we construct generalized translation networks to uniformly approximate a class of nonlinear continuous functionals defined on l p 1 1 s for integer s 1 1 p 1 or c 1 1 s we obtain lower bounds on the possible order of approximation for such functionals in terms of any approximation process depending continuously upon a given number of parameters our networks almost achieve this order of approximation in terms of the number of parameters neurons involved in the network the training is simple and noniterative in particular we avoid any optimization such as that involved in the usual back propagation

mean square dissipative methods for stochastic age dependent capital system with fractional brownian motion and jumps abstract in this paper we analyze mean square dissipativity of numerical methods applied to a class of stochastic age dependent vintage capital system with fractional brownian motion fbm and poisson jumps some sufficient conditions are obtained for ensuring the underlying systems are mean square dissipative it is shown that the mean square dissipativity is preserved by the compensated split step backward euler method and compensated backward euler method without any restriction on stepsize while the split step backward euler method and backward euler method could reproduce mean square dissipativity under a stepsize constraint those results indicate that compensated numerical methods achieve superiority over non compensated numerical methods in terms of mean square dissipativity a numerical example is provided to illustrate the theoretical results

convergence of numerical solutions to stochastic age dependent population equations with markovian switching in this paper a class of stochastic age dependent population equations with markovian switching is considered the main aim of this paper is to investigate the convergence of the numerical approximation of stochastic age dependent population equations with markovian switching it is proved that the numerical approximation solutions converge to the analytic solutions of the equations under the given conditions an example is given for illustration

subconstituents of orthogonal graphs of characteristic 2 abstract the subconstituents of the singular orthogonal graphs over finite fields of characteristic 2 are studied after considering the actions of the orthogonal group on the graphs their parameters are calculated we show that the first subconstituent is strongly regular and that for all but the smallest cases the second subconstituent is quasi strongly regular and edge regular the automorphism group of the second subconstituent is also determined

model selection and akaike s information criterion aic the general theory and its analytical extensions during the last fifteen years akaike s entropy based information criterion aic has had a fundamental impact in statistical model evaluation problems this paper studies the general theory of the aic procedure and provides its analytical extensions in two ways without violating akaike s main principles these extensions make aic asymptotically consistent and penalize overparameterization more stringently to pick only the simplest of the true models these selection criteria are called caic and caicf asymptotic properties of aic and its extensions are investigated and empirical performances of these criteria are studied in choosing the correct degree of a polynomial model in two different monte carlo experiments under different conditions

on the partitioning capabilities of feedforward neural networks with sigmoid nodes in this letter the capabilities of feedforward neural networks fnns on the realization and approximation of functions of the form g rl a which partition the rl space into polyhedral sets each one being assigned to one out of the c classes of a are investigated more specifically a constructive proof is given for the fact that fnns consisting of nodes having sigmoid output functions are capable of approximating any function g with arbitrary accuracy also the capabilities of fnns consisting of nodes having the hard limiter as output function are reviewed in both cases the two class as well as the multiclass cases are considered

dealing with label switching in mixture models summary in a bayesian analysis of finite mixture models parameter estimation and clustering are sometimes less straightforward than might be expected in particular the common practice of estimating parameters by their posterior mean and summarizing joint posterior distributions by marginal distributions often leads to nonsensical answers this is due to the so called label switching problem which is caused by symmetry in the likelihood of the model parameters a frequent response to this problem is to remove the symmetry by using artificial identifiability constraints we demonstrate that this fails in general to solve the problem and we describe an alternative class of approaches relabelling algorithms which arise from attempting to minimize the posterior expected loss under a class of loss functions we describe in detail one particularly simple and general relabelling algorithm and illustrate its success in dealing with the label switching problem on two examples

convergence of numerical solutions to neutral stochastic delay differential equations with markovian switching recently numerical solutions of stochastic differential equations have received a great deal of attention it is surprising that there are not any numerical methods established for neutral stochastic delay differential equations yet in the paper the euler maruyama method for neutral stochastic delay differential equations is developed the key aim is to show that the numerical solutions will converge to the true solutions under the local lipschitz condition

universal approximation to nonlinear operators by neural networks with arbitrary activation functions and its application to dynamical systems the purpose of this paper is to investigate neural network capability systematically the main results are 1 every tauber wiener function is qualified as an activation function in the hidden layer of a three layered neural network 2 for a continuous function in s r sup 1 to be a tauber wiener function the necessary and sufficient condition is that it is not a polynomial 3 the capability of approximating nonlinear functionals defined on some compact set of a banach space and nonlinear operators has been shown and 4 the possibility by neural computation to approximate the output as a whole not at a fixed point of a dynamical system thus identifying the system

convergence analyses on on line weight noise injection based training algorithms for mlps injecting weight noise during training is a simple technique that has been proposed for almost two decades however little is known about its convergence behavior this paper studies the convergence of two weight noise injection based training algorithms multiplicative weight noise injection with weight decay and additive weight noise injection with weight decay we consider that they are applied to multilayer perceptrons either with linear or sigmoid output nodes let w t be the weight vector let v w be the corresponding objective function of the training algorithm let 0 be the weight decay constant and let t be the step size we show that if t 0 then with probability one e w t 22 is bound and limt w t 2 exists based on these two properties we show that if t 0 t t and t t 2 then with probability one these algorithms converge moreover w t converges with probability one to a point where wv w 0

convergence of numerical solutions to stochastic age structured population system with diffusion in this paper stochastic age structured population system with diffusion are studied in general most of stochastic age structured population system with diffusion do not have explicit solutions thus numerical approximation schemes are invaluable tools for exploring their properties the main purpose of this paper is to develop a numerical scheme and show the convergence of the numerical approximation solution to the true solution

neural network for nonsmooth nonconvex constrained minimization via smooth approximation a neural network based on smoothing approximation is presented for a class of nonsmooth nonconvex constrained optimization problems where the objective function is nonsmooth and nonconvex the equality constraint functions are linear and the inequality constraint functions are nonsmooth convex this approach can find a clarke stationary point of the optimization problem by following a continuous path defined by a solution of an ordinary differential equation the global convergence is guaranteed if either the feasible set is bounded or the objective function is level bounded specially the proposed network does not require 1 the initial point to be feasible 2 a prior penalty parameter to be chosen exactly 3 a differential inclusion to be solved numerical experiments and comparisons with some existing algorithms are presented to illustrate the theoretical results and show the efficiency of the proposed network

universal approximation of multiple nonlinear operators by neural networks recently there has been interest in the observed capabilities of some classes of neural networks with fixed weights to model multiple nonlinear dynamical systems while this property has been observed in simulations open questions exist as to how this property can arise in this article we propose a theory that provides a possible mechanism by which this multiple modeling phenomenon can occur

analysis of boundedness and convergence of online gradient method for two layer feedforward neural networks this paper presents a theoretical boundedness and convergence analysis of online gradient method for the training of two layer feedforward neural networks the well known linear difference equation is extended to apply to the general case of linear or nonlinear activation functions based on this extended difference equation we investigate the boundedness and convergence of the parameter sequence of concern which is trained by finite training samples with a constant learning rate we show that the uniform upper bound of the parameter sequence which is very important in the training procedure is the solution of an inequality regarding the bound it is further verified that for the case of linear activation function a solution always exists and moreover the parameter sequence can be uniformly upper bounded while for the case of nonlinear activation function some simple adjustment methods on the training set or the activation function can be derived to improve the boundedness property then for the convergence analysis it is shown that the parameter sequence can converge into a zone around an optimal solution at which the error function attains its global minimum where the size of the zone is associated with the learning rate particularly for the case of perfect modeling a strong global convergence result where the parameter sequence can always converge to an optimal solution is proved

posterior predictive p values extending work of rubin this paper explores a bayesian counterpart of the classical p value namely a tail area probability of a test statistic under a null hypothesis the bayesian formulation using posterior predictive replications of the data allows a test statistic to depend on both data and unknown nuisance parameters and thus permits a direct measure of the discrepancy between sample and population quantities the tail area probability for a test statistic is then found under the joint posterior distribution of replicate data and the nuisance parameters both conditional on the null hypothesis this posterior predictive p value can also be viewed as the posterior mean of a classical p value averaging over the posterior distribution of nuisance parameters under the null hypothesis and thus it provides one general method for dealing with nuisance parameters two classical examples including the behrens fisher problem are used to illustrate the posterior predictive p value and some of its interesting properties which also reveal a new bayesian interpretation for some classical p values an application to multiple imputation inference is also presented a frequency evaluation shows that in general if the replication is defined by new nuisance parameters and new data then the type i frequentist error of an level posterior predictive test is often close to but less than and will never exceed 2

suborbits of a point stabilizer in the orthogonal group on the last subconstituent of orthogonal dual polar graphs abstract as one of the serial papers on suborbits of point stabilizers in classical groups on the last subconstituent of dual polar graphs the corresponding problem for orthogonal dual polar graphs over a finite field of odd characteristic is discussed in this paper we determine all the suborbits of a point stabilizer in the orthogonal group on the last subconstituent and calculate the length of each suborbit moreover we discuss the quasi strongly regular graphs and the association schemes based on the last subconstituent respectively

computing bayes factors by combining simulation and asymptotic approximations abstract the bayes factor is a ratio of two posterior normalizing constants which may be difficult to compute we compare several methods of estimating bayes factors when it is possible to simulate observations from the posterior distributions via markov chain monte carlo or other techniques the methods that we study are all easily applied without consideration of special features of the problem provided that each posterior distribution is well behaved in the sense of having a single dominant mode we consider a simulated version of laplace s method a simulated version of bartlett correction importance sampling and a reciprocal importance sampling technique we also introduce local volume corrections for each of these in addition we apply the bridge sampling method of meng and wong we find that a simulated version of laplace s method with local volume correction furnishes an accurate approximation that is especially useful when likelihood function evaluations are costly a simple bridge sampli

wiener s tauberian theorems for the fourier bessel transformation and uniform approximation by rbf networks of delsarte translates abstract we prove analogues of the wiener and wiener pitt tauberian theorems for the delsarte translation and the fourier bessel transformation through similar functional analytic techniques we give conditions on the kernel function or activation function for the family of radial basis function rbf neural networks obtained upon replacing the usual translation by the delsarte one with not necessarily the same smoothing factor in all kernel nodes to have the universal approximation property in a certain space of continuous functions endowed with the uniform norm

approximation by network operators with logistic activation functions this paper aims to study the construction and multivariate approximation of a class of network operators with logistic sigmoidal functions first a class of even and bell shaped function with support on r is constructed by using appropriate translation and combination of the logistic function then the constructed function is employed as activation function to construct a kind of so called cardaliaguet euvrard type network operators finally these network operators are used to approximate bivariate functions in c 1 1 2 and a jackson type theorem for the approximation errors is established

subconstituents of orthogonal graphs of odd characteristic in this paper the subconstituents of the isotropic orthogonal graphs over finite fields of odd characteristic are studied the first subconstituent is strongly regular while the second subconstituent is edge regular the full automorphism groups of these two subconstituents have also been determined

existence uniqueness and stability of stochastic neutral functional differential equations of sobolev type in this paper we are mainly concerned with a class of stochastic neutral functional differential equations of sobolev type with poisson jumps under two different sets of conditions we establish the existence of the mild solution by applying the leray schauder alternative theory and the sadakovskii s fixed point theorem respectively furthermore we use the bihari s inequality to prove the osgood type uniqueness also the mean square exponential stability is investigated by applying the gronwall inequality finally two examples are given to illustrate the theory results

numerical analysis of the balanced implicit method for stochastic age dependent capital system with poisson jumps abstract the aim of this paper is to construct a numerical method to preserve positivity and mean square dissipativity of stochastic age dependent capital system with poisson jumps we use the balanced implicit numerical techniques to maintain the nonnegative path of the exact solution it is proved that the balanced implicit method bim preserves positivity and converges with order 1 2 under given conditions in addition some sufficient conditions are obtained for ensuring the system and the balanced implicit method bim are mean square dissipative finally a numerical example is simulated to illustrate the efficiency of theoretical results

approximation by neural networks with scattered data in this paper the approximation capability of feed forward neural networks fnns formed as k 1 nc k f x x k is considered where f is an even periodic continuous function and x k s are scattered data the best approximation error of trigonometric polynomials and the continuous modulus of the activation function f are introduced to describe the approximation capability of fnns

modeling a stochastic age structured capital system with poisson jumps using neural networks abstract this paper presents an approach for modeling a stochastic age structured capital system sascs with poisson jumps in which the transfer rate of capital noise intensity and jump intensity are approximated by a type of feedforward neural networks fnns the approximate stock of capital for the system is calculated using euler s scheme in time discretization based on barkholder davis gundys inequality and gronwalls lemma the error between the approximate stock of capital and the actual stock of capital is estimated by using mean squares theoretical analysis of the convergence of the approximate stock of capital is conducted and some numerical results are presented to justify our findings

exponential stability of numerical solutions for a class of stochastic age dependent capital system with poisson jumps recently numerical solutions of stochastic differential equations have received a great deal of attention numerical approximation schemes are invaluable tools for exploring their properties in this paper we introduce a class of stochastic age dependent vintage capital system with poisson jumps we also give the discrete approximate solution with an implicit euler scheme in time discretization using gronwall s lemma and barkholder davis gundy s inequality some criteria are obtained for the exponential stability of numerical solutions to the stochastic age dependent capital system with poisson jumps it is proved that the numerical approximation solutions converge to the analytic solutions of the equations under the given conditions where information on the order of approximation is provided these error bounds imply strong convergence as the timestep tends to zero a numerical example is used to illustrate the theoretical results

convergence analysis of semi implicit euler methods for solving stochastic equations with variable delays and random jump magnitudes in this paper we study the following stochastic equations with variable delays and random jump magnitudes dx t f x t x t t t dt g x t x t t t dw t h x t x t t t c n t 1 dn t 0 t t x t j t r t 0 we establish the semi implicit euler approximate solutions for the above systems and prove that the approximate solutions converge to the analytical solutions in the mean square sense as well as in the probability sense some known results are generalized and improved

upper bounds on the number of hidden neurons in feedforward networks with arbitrary bounded nonlinear activation functions it is well known that standard single hidden layer feedforward networks slfns with at most n hidden neurons including biases can learn n distinct samples x sub i t sub i with zero error and the weights connecting the input neurons and the hidden neurons can be chosen almost arbitrarily however these results have been obtained for the case when the activation function for the hidden neurons is the signum function this paper rigorously proves that standard single hidden layer feedforward networks slfns with at most n hidden neurons and with any bounded nonlinear activation function which has a limit at one infinity can learn n distinct samples x sub i t sub i with zero error the previous method of arbitrarily choosing weights is not feasible for any slfn the proof of our result is constructive and thus gives a method to directly find the weights of the standard slfns with any such bounded nonlinear activation function as opposed to iterative training algorithms in the literature

graph theory based approach for stability analysis of stochastic coupled systems with levy noise on networks in this paper a novel class of stochastic coupled systems with levy noise on networks scslnns is presented both white noise and levy noise are considered in the networks by exploiting graph theory and lyapunov stability theory criteria ensuring p th moment exponential stability and stability in probability of these scslnns are established respectively these principles are closely related to the topology of the network and the perturbation intensity of white noise and levy noise moreover to verify the theoretical results stochastic coupled oscillators with levy noise on a network and stochastic volterra predator prey system with levy noise are performed finally a numerical example about oscillators network is provided to illustrate the feasibility of our analytical results

strong convergence of split step backward euler method for stochastic age dependent capital system with markovian switching abstract in this paper we present and analyze the split step backward euler method for the stochastic capital system with markovian switching under the one sided local lipschitz condition on the drift and local lipschitz condition on the diffusion we prove the split step backward euler method converges with strong order of one half to the true solution a numerical example is provided to illustrate the theoretical results

numerical analysis for stochastic age dependent population equations with poisson jumps in this paper stochastic age dependent population equations with poisson jumps are considered in general most of stochastic age dependent population equations with jumps do not have explicit solutions thus numerical approximation schemes are invaluable tools for exploring their properties the main purpose of this paper is to develop a numerical euler scheme and show the convergence of the numerical approximation solution to the true solution

subconstituents of orthogonal graphs of odd characteristic continued abstract the subconstituents of the orthogonal graph o 2 q where 2 and 1 2 over a finite field of odd characteristic are shown to be quasi strongly regular furthermore the first subconstituent is shown to be co edge regular and when 3 its automorphism group is determined the second subconstituent is shown to be edge regular and when 2 its automorphism group is determined their parameters and chromatic numbers are also determined

the errors of simultaneous approximation of multivariate functions by neural networks there have been many studies on the simultaneous approximation capability of feed forward neural networks fnns most of these however are only concerned with the density or feasibility of performing simultaneous approximations this paper considers the simultaneous approximation of algebraic polynomials employing taylor expansion and an algebraic constructive approach to construct a class of fnns which realize the simultaneous approximation of any smooth multivariate function and all of its derivatives we also present an upper bound on the approximation accuracy of the fnns expressed in terms of the modulus of continuity of the functions to be approximated

suborbits of a point stabilizer in the unitary group on the last subconstituent of hermitean dual polar graphs let be a dual polar graph in a unitary space it is well known that a point stabilizer in the unitary group is transitive on the last subconstituent of in this paper we determine all the suborbits of this action calculate its rank and the length of each suborbit note that the induced subgraph on is quasi strongly regular as an application of our results all its parameters are computed

approximate bayesian inference with the weighted likelihood bootstrap we introduce the weighted likelihood bootstrap wlb as a way to simulate approximately from a posterior distribution this method is often easy to implement requiring only an algorithm for calculating the maximum likelihood estimator such as iteratively reweighted least squares in the generic weighting scheme the wlb is first order correct under quite general conditions inaccuracies can be removed by using the wlb as a source of samples in the sampling importance resampling sir algorithm which also allows incorporation of particular prior information the sir adjusted wlb can be a competitive alternative to other integration methods in certain models asymptotic expansions elucidate the second order properties of the wlb which is a generalization of rubin s bayesian bootstrap the calculation of approximate bayes factors for model comparison is also considered we note that given a sample simulated from the posterior distribution the required marginal likelihood may be simulation consistently estimated by the harmonic mean of the associated likelihood values a modification of this estimator that avoids instability is also noted these methods provide simple ways of calculating approximate bayes factors and posterior model probabilities for a very wide class of models

complete memory structures for approximating nonlinear discrete time mappings this paper introduces a general structure that is capable of approximating input output maps of nonlinear discrete time systems the structure is comprised of two stages a dynamical stage followed by a memoryless nonlinear stage a theorem is presented which gives a simple necessary and sufficient condition for a large set of structures of this form to be capable of modeling a wide class of nonlinear discrete time systems in particular we introduce the concept of a complete memory a structure with a complete memory dynamical stage and a sufficiently powerful memoryless stage is shown to be capable of approximating arbitrarily wide class of continuous causal time invariant approximately finite memory mappings between discrete time signal spaces furthermore we show that any bounded input bounded output time invariant causal memory structure has such an approximation capability if and only if it is a complete memory several examples of linear and nonlinear complete memories are presented the proposed complete memory structure provides a template for designing a wide variety of artificial neural networks for nonlinear spatiotemporal processing

approximation of nonlinear systems with radial basis function neural networks a technique for approximating a continuous function of n variables with a radial basis function rbf neural network is presented the method uses an n dimensional raised cosine type of rbf that is smooth yet has compact support the rbf network coefficients are low order polynomial functions of the input a simple computational procedure is presented which significantly reduces the network training and evaluation time storage space is also reduced by allowing for a nonuniform grid of points about which the rbfs are centered the network output is shown to be continuous and have a continuous first derivative when the network is used to approximate a nonlinear dynamic system the resulting system is bounded input bounded output stable for the special case of a linear system the rbf network representation is exact on the domain over which it is defined and it is optimal in terms of the number of distinct storage parameters required several examples are presented which illustrate the effectiveness of this technique

on different facets of regularization theory this review provides a comprehensive understanding of regularization theory from different perspectives emphasizing smoothness and simplicity principles using the tools of operator theory and fourier analysis it is shown that the solution of the classical tikhonov regularization problem can be derived from the regularized functional defined by a linear differential integral operator in the spatial fourier domain state of the art research relevant to the regularization theory is reviewed covering occam s razor minimum length description bayesian theory pruning algorithms informational entropy theory statistical learning theory and equivalent regularization the universal principle of regularization in terms of kolmogorov complexity is discussed finally some prospective studies on regularization theory and beyond are suggested

convergence of the semi implicit euler method for stochastic age dependent population equations abstract a class of semi implicit methods is introduced for stochastic age dependent population equations it is proved that the semi implicit euler methods are convergent with strong order p 1 2 an example is given to illustrate the efficiency of the semi implicit euler method

solving nonlinear equality constrained multiobjective optimization problems using neural networks this paper develops a neural network architecture and a new processing method for solving in real time the nonlinear equality constrained multiobjective optimization problem necmop where several nonlinear objective functions must be optimized in a conflicting situation in this processing method the necmop is converted to an equivalent scalar optimization problem sop the sop is then decomposed into several separable subproblems processable in parallel and in a reasonable time by multiplexing switched capacitor circuits the approach which we propose makes use of a decomposition coordination principle that allows nonlinearity to be treated at a local level and where coordination is achieved through the use of lagrange multipliers the modularity and the regularity of the neural networks architecture herein proposed make it suitable for very large scale integration implementation an application to the resolution of a physical problem is given to show that the approach used here possesses some advantages of the point of algorithmic view and provides processes of resolution often simpler than the usual techniques

convergence analyses on sparse feedforward neural networks via group lasso regularization in this paper a new variant of feedforward neural networks has been proposed for a class of nonsmooth optimization problems the penalty term of the presented neural networks stems from the group lasso method which selects hidden variables in a grouped manner to deal with the non differentiability of the original penalty term l 1 l 2 norm and avoid oscillations smoothing techniques have been used to approximate the objective function it is assumed that the training samples are supplied to the networks in a specific incremental way during training that is in each cycle samples are supplied in a fixed order then under suitable assumptions on learning rate penalization coefficients and smoothing parameters the weak and strong convergence of the training process for the smoothing neural networks have been proved the convergence analysis shows that the gradient of the smoothing error function approaches zero and the weight sequence converges to a fixed point respectively we demonstrate how the smoothing approximation parameter can be updated in the training procedure so as to guarantee the convergence of the procedure to a clarke stationary point of the original optimization problem in addition we have proved that the original nonsmoothing algorithm with l 1 l 2 norm penalty converges consistently to the same optimum solution with the corresponding smoothed algorithm numerical simulations demonstrate the convergence and effectiveness of the proposed training algorithm

stability analysis for stochastic volterra levin equations with poisson jumps fixed point approach this paper is devoted to investigate a class of stochastic volterra levin equations with poisson jumps to the best of the authors knowledge till now the stability problem for this class of new systems has not yet been solved since poisson jumps are considered the main objective of this paper is to fill the gap by using the fixed point theory we first study the existence and uniqueness of the solution as well as the pth moment exponential stability for the considered system then based on the well known borel cantelli lemma we prove that the solution is almost surely pth moment exponentially stable our results improve and generalize those given in the previous literature finally two simple examples are provided to illustrate the effectiveness of the obtained results

numerical analysis for stochastic age dependent population equations a numerical method proposed to approximate the solution of a class of stochastic age dependent populations we show that under certain conditions weaker than linear growth and global lips chits the euler scheme applied to stochastic age dependent population converges to the analytic solution and provide information on the order of approximation

existence uniqueness and exponential stability for stochastic age dependent population in this paper we introduce a class of stochastic age dependent population dynamic system existence and uniqueness of strong solution are proved for stochastic age dependent population dynamic system in hilbert space using klmogorou s inequality and barkholder davis gundy s lemma some criteria are obtained for the exponential stability of stochastic age dependent population dynamic system

on bayesian analysis of mixtures with an unknown number of components discussion author s reply new methodology for fully bayesian mixture analysis is developed making use of reversible jump markov chain monte carlo methods that are capable of jumping between the parameter subspaces corresponding to different numbers of components in the mixture a sample from the full joint distribution of all unknown variables is thereby generated and this can be used as a basis for a thorough presentation of many aspects of the posterior distribution the methodology is applied here to the analysis of univariate normal mixtures using a hierarchical prior model that offers an approach to dealing with weak prior information while avoiding the mathematical pitfalls of using improper priors in the mixture context

exponential stability of numerical solutions to sddes with markovian switching abstract the main aim of this paper is to investigate the exponential stability of the euler method for a class of autonomous stochastic delay differential equations with markovian switching the definition of exponential mean square stability of numerical method is introduced it is proved that the euler scheme is exponentially stable in mean square sense several examples are given for illustration

convergence of numerical solutions to stochastic age structured population equations with diffusions and markovian switching in this paper it is considered for a class of stochastic age structured population equations with diffusions and markovian switching most kind of equations are nonlinear and cannot be solved explicitly so the construction of efficient computational methods is of great importance the main aim of this paper is to develop a numerical scheme and investigate the convergence of numerical approximation an example is given for illustration

numerical solutions of stochastic differential delay equations under local lipschitz condition in this paper a variant of the euler maruyama method is used to define the numerical solutions for stochastic differential delay equations sddes with variable delay the key aim is to show that the numerical solutions will converge to the true solutions of the sddes under the local lipschitz condition

approximations of continuous functionals by neural networks with application to dynamic systems the paper gives several strong results on neural network representation in an explicit form under very mild conditions a functional defined on a compact set in c a b or l sup p a b spaces of infinite dimensions can be approximated arbitrarily well by a neural network with one hidden layer the results are a significant development beyond earlier work where theorems of approximating continuous functions defined on a finite dimensional real space by neural networks with one hidden layer were given all the results are shown to be applicable to the approximation of the output of dynamic systems at any particular time

generalized linear models with random effects a gibbs sampling approach abstract generalized linear models have unified the approach to regression for a wide variety of discrete continuous and censored response variables that can be assumed to be independent across experimental units in applications such as longitudinal studies genetic studies of families and survey sampling observations may be obtained in clusters responses from the same cluster cannot be assumed to be independent with linear models correlation has been effectively modeled by assuming there are cluster specific random effects that derive from an underlying mixing distribution extensions of generalized linear models to include random effects has thus far been hampered by the need for numerical integration to evaluate likelihoods in this article we cast the generalized linear random effects model in a bayesian framework and use a monte carlo method the gibbs sampler to overcome the current computational limitations the resulting algorithm is flexible to easily accommodate changes in the number

model based clustering of large data sets tracing the development of spelling ability there are two main theories with respect to the development of spelling ability the stage model and the model of overlapping waves in this paper exploratory model based clustering will be used to analyze the responses of more than 3500 pupils to subsets of 245 items to evaluate the two theories the resulting clusters will be ordered along a developmental dimension using an external criterion solutions for three statistical problems will be given 1 an algorithm that can handle large data sets and only renders non degenerate clusters 2 a goodness of fit test that is not affected by the fact that the number of possible response vectors by far out weights the number of observed response vectors and 3 a new technique data expunction that can be used to evaluate goodness of fit tests if the missing data mechanism is known

convergence of the euler scheme for stochastic functional partial differential equations the study of stochastic partial different equations is motivated by the fact that when one wants to model some evolution phenomena arising in physics biology and engineering etc however in many applications the resulting equations have so far proved intractable to direct analytical solution numerical approximations such as the euler scheme are therefore a vital tool in exploring model behaviour unfortunately current results concerning the convergence of such schemes impose conditions on the drift and diffusion coefficients of the stochastic partial differential equation namely the linear growth and global lipschitz conditions which are often not met by systems of interest in this paper these conditions are relaxed and prove that numerical solutions based on the euler scheme will converge to the true solution of a broad class of stochastic partial differential equations several examples are studied to illustrate the results

boundary control of linear uncertain 1 d parabolic pde using approximate dynamic programming this paper develops a near optimal boundary control method for distributed parameter systems governed by uncertain linear 1 d parabolic partial differential equations pde by using approximate dynamic programming a quadratic surface integral is proposed to express the optimal cost functional for the infinite dimensional state space accordingly the hamilton jacobi bellman hjb equation is formulated in the infinite dimensional domain without using any model reduction subsequently a neural network identifier is developed to estimate the unknown spatially varying coefficient in pde dynamics novel tuning law is proposed to guarantee the boundedness of identifier approximation error in the pde domain a radial basis network rbn is subsequently proposed to generate an approximate solution for the optimal surface kernel function online the tuning law for near optimal rbn weights is created such that the hjb equation error is minimized while the dynamics are identified and closed loop system remains stable ultimate boundedness ub of the closed loop system is verified by using the lyapunov theory the performance of the proposed controller is successfully confirmed by simulation on an unstable diffusion reaction process

convergence of numerical solutions for a class of stochastic age dependent capital system with random jump magnitudes in stochastic differential equations sdes there is a class of stochastic functional differential equations with random jump magnitudes which aries in many financial models in general most equations of stochastic age dependent capital system do not have explicit solutions thus numerical approximation schemes are invaluable tools for exploring their properties in this paper the numerical approximation is established for a class of stochastic age dependent capital system with random jump magnitudes the main aim of this paper is to investigate the convergence of the numerical approximation for a class of stochastic age dependent capital system with random jump magnitudes it is proved that the numerical approximate solutions converge to the analytical solutions of the equations under given conditions the numerical approximate results in zhang et al 2011 2 are improved an example is given for illustration

when does online bp training converge the backpropogation bp neural networks have been widely applied in scientific research and engineering the success of the application however relies upon the convergence of the training procedure involved in the neural network learning we settle down the convergence analysis issue through proving two fundamental theorems on the convergence of the online bp training procedure one theorem claims that under mild conditions the gradient sequence of the error function will converge to zero the weak convergence and another theorem concludes the convergence of the weight sequence defined by the procedure to a fixed value at which the error function attains its minimum the strong convergence the weak convergence theorem sharpens and generalizes the existing convergence analysis conducted before while the strong convergence theorem provides new analysis results on convergence of the online bp training procedure the results obtained reveal that with any analytic sigmoid activation function the online bp training procedure is always convergent which then underlies successful application of the bp neural networks

stability of solution to a class of investment system abstract the investment model of nonlinear stochastic neutral technical progress with time delay is given in this paper adopting the theory of stochastic functional differential equation and using ito formula gronwall s lemma and barkholder davis gundy s lemma exponential stability of strong solution is proved for the system of nonlinear stochastic neutral technical progress assets with time delay on hilbert space also a sufficient condition of the exponential stability is obtained the results are the improvement and extension of the existed results of this field

worst case complexity of smoothing quadratic regularization methods for non lipschitzian optimization in this paper we propose a smoothing quadratic regularization sqr algorithm for solving a class of nonsmooth nonconvex perhaps even non lipschitzian minimization problems which has wide applications in statistics and sparse reconstruction the proposed sqr algorithm is a first order method at each iteration the sqr algorithm solves a strongly convex quadratic minimization problem with a diagonal hessian matrix which has a simple closed form solution that is inexpensive to calculate we show that the worst case complexity of reaching an epsilon scaled stationary point is o epsilon 2 moreover if the objective function is locally lipschitz continuous the sqr algorithm with a slightly modified updating scheme for the smoothing parameter and iterate can obtain an epsilon clarke stationary point in at most o epsilon 3 iterations

practical exponential stability of stochastic age dependent capital system with levy noise abstract it is well known that stability is one of most important topics in economy and control in this paper we focus on the practical stability problem for a class of stochastic age dependent vintage capital system with levy noise compared with the classical lyapunov stability theory practical stability can depict not only qualitative behavior but also quantitative properties such as specific trajectory bounds and specific transient behavior with the help of the stochastic analysis theory and ito s formula we establish some novel criteria for the mean square and almost sure practical exponential stability of the addressed system the obtained results improve some previous works given in the literature moreover an example is provided to illustrate the theoretical results

convergence of the split step θ method for stochastic age dependent population equations with poisson jumps in this paper a new split step ss method for stochastic age dependent population equations with poisson jumps is constructed the main aim of this paper is to investigate the convergence of the ss method for stochastic age dependent population equations with poisson jumps it is proved that the proposed method is convergent with strong order 1 2 under given conditions finally an example is simulated to verify the results obtained from theory the results show that the ss method has better accuracy compared to the euler method

exponential stability of numerical solutions to a stochastic age structured population system with diffusion the main aim of this paper is to investigate the exponential stability of the euler method for a stochastic age structured population system with diffusion the definition of exponential mean square stability of numerical method is introduced it is proved that the euler scheme is exponentially stable in mean square sense an example is given for illustration

subconstituents of the orthogonal graph of type m m 1 0 of odd characteristic abstract in this paper the orthogonal graph of type m m 1 0 over f q of odd characteristic is introduced we study the elementary properties of the graph and show that the first subconstituent of it is a generalized strongly regular graph moreover the structure of the second subconstituent and the adjacency relations between different subconstituents are given

a stochastic sirs epidemic model with infectious force under intervention strategies abstract in this paper we extend a classical sirs epidemic model with the infectious forces under intervention strategies from a deterministic framework to a stochastic differential equation sde one through introducing random fluctuations the value of our study lies in two aspects mathematically by using the markov semigroups theory we prove that the reproduction number r 0 s can be used to govern the stochastic dynamics of sde model if r 0 s 1 under mild extra conditions the sde system has a disease free absorbing set which means the extinction of disease with probability one if r 0 s 1 under mild extra conditions it has an endemic stationary distribution which leads to the stochastical persistence of the disease epidemiologically we find that random fluctuations can suppress disease outbreak which can provide us some useful control strategies to regulate disease dynamics

output feedback based boundary control of uncertain coupled semilinear parabolic pde using neurodynamic programming in this paper neurodynamic programming based output feedback boundary control of distributed parameter systems governed by uncertain coupled semilinear parabolic partial differential equations pdes under neumann or dirichlet boundary control conditions is introduced first hamilton jacobi bellman hjb equation is formulated in the original pde domain and the optimal control policy is derived using the value functional as the solution of the hjb equation subsequently a novel observer is developed to estimate the system states given the uncertain nonlinearity in pde dynamics and measured outputs consequently the suboptimal boundary control policy is obtained by forward in time estimation of the value functional using a neural network nn based online approximator and estimated state vector obtained from the nn observer novel adaptive tuning laws in continuous time are proposed for learning the value functional online to satisfy the hjb equation along system trajectories while ensuring the closed loop stability local uniformly ultimate boundedness of the closed loop system is verified by using lyapunov theory the performance of the proposed controller is verified via simulation on an unstable coupled diffusion reaction process

factor analysis and aic the information criterion aic was introduced to extend the method of maximum likelihood to the multimodel situation it was obtained by relating the successful experience of the order determination of an autoregressive model to the determination of the number of factors in the maximum likelihood factor analysis the use of the aic criterion in the factor analysis is particularly interesting when it is viewed as the choice of a bayesian model this observation shows that the area of application of aic can be much wider than the conventional i i d type models on which the original derivation of the criterion was based the observation of the bayesian structure of the factor analysis model leads us to the handling of the problem of improper solution by introducing a natural prior distribution of factor loadings

on bayesian analysis of mixtures with an unknown number of components with discussion new methodology for fully bayesian mixture analysis is developed making use of reversible jump markov chain monte carlo methods that are capable of jumping between the parameter subspaces corresponding to different numbers of components in the mixture a sample from the full joint distribution of all unknown variables is thereby generated and this can be used as a basis for a thorough presentation of many aspects of the posterior distribution the methodology is applied here to the analysis of univariate normal mixtures using a hierarchical prior model that offers an approach to dealing with weak prior information while avoiding the mathematical pitfalls of using improper priors in the mixture context

convergence of numerical solutions to stochastic age dependent population equations in this paper stochastic age dependent population equations one of the important classes of hybrid systems are studied in general most of stochastic age dependent population equations do not have explicit solutions thus numerical approximation schemes are invaluable tools for exploring their properties the main purpose of this paper is to develop a numerical scheme and show the convergence of the numerical approximation solution to the true solution

smoothing neural network for constrained non lipschitz optimization with applications in this paper a smoothing neural network snn is proposed for a class of constrained non lipschitz optimization problems where the objective function is the sum of a nonsmooth nonconvex function and a non lipschitz function and the feasible set is a closed convex subset of using the smoothing approximate techniques the proposed neural network is modeled by a differential equation which can be implemented easily under the level bounded condition on the objective function in the feasible set we prove the global existence and uniform boundedness of the solutions of the snn with any initial point in the feasible set the uniqueness of the solution of the snn is provided under the lipschitz property of smoothing functions we show that any accumulation point of the solutions of the snn is a stationary point of the optimization problem numerical results including image restoration blind source separation variable selection and minimizing condition number are presented to illustrate the theoretical results and show the efficiency of the snn comparisons with some existing algorithms show the advantages of the snn

the approximation operators with sigmoidal functions the aim of this paper is to investigate the error which results from the method of approximation operators with logarithmic sigmoidal function by means of the method of extending functions a class of feed forward neural network operators is introduced using these operators as approximation tools the upper bounds of errors in uniform norm approximating continuous functions are estimated also a class of quasi interpolation operators with logarithmic sigmoidal function is constructed for approximating continuous functions defined on the total real axis

deterministic convergence of an online gradient method for neural networks the online gradient method has been widely used as a learning algorithm for neural networks we establish a deterministic convergence of online gradient methods for the training of a class of nonlinear feedforward neural networks when the training examples are linearly independent we choose the learning rate to be a constant during the training procedure the monotonicity of the error function in the iteration is proved a criterion for choosing the learning rate is also provided to guarantee the convergence under certain conditions similar to those for the classical gradient methods an optimal convergence rate for our online gradient methods is proved

convergence of an online gradient method for feedforward neural networks with stochastic inputs in this paper we study the convergence of an online gradient method for feed forward neural networks the input training examples are permuted stochastically in each cycle of iteration a monotonicity and a weak convergence of deterministic nature are proved

is extreme learning machine feasible a theoretical assessment part i an extreme learning machine elm is a feedforward neural network fnn like learning system whose connections with output neurons are adjustable while the connections with and within hidden neurons are randomly fixed numerous applications have demonstrated the feasibility and high efficiency of elm like systems it has however been open if this is true for any general applications in this two part paper we conduct a comprehensive feasibility analysis of elm in part i we provide an answer to the question by theoretically justifying the following 1 for some suitable activation functions such as polynomials nadaraya watson and sigmoid functions the elm like systems can attain the theoretical generalization bound of the fnns with all connections adjusted i e they do not degrade the generalization capability of the fnns even when the connections with and within hidden neurons are randomly fixed 2 the number of hidden neurons needed for an elm like system to achieve the theoretical bound can be estimated and 3 whenever the activation function is taken as polynomial the deduced hidden layer output matrix is of full column rank therefore the generalized inverse technique can be efficiently applied to yield the solution of an elm like system and furthermore for the nonpolynomial case the tikhonov regularization can be applied to guarantee the weak regularity while not sacrificing the generalization capability in part ii however we reveal a different aspect of the feasibility of elm there also exists some activation functions which makes the corresponding elm degrade the generalization capability the obtained results underlie the feasibility and efficiency of elm like systems and yield various generalizations and improvements of the systems as well

adaptive classification of temporal signals in fixed weight recurrent neural networks an existence proof recurrent neural networks with fixed weights have been shown in practice to successfully classify adaptively signals that vary as a function of time in the presence of additive noise and parametric perturbations we address the question can this ability be explained theoretically we provide a mathematical proof that these networks have this ability even when parametric perturbations enter the signals nonlinearly the restrictions that we impose on the signals to be classified are that they satisfy an assumption of nondegeneracy and that noise amplitude is sufficiently small further we demonstrate that the recurrent neural networks may not only classify uncertain signals adaptively but also can recover the values of uncertain parameters of the signals up to their equivalence classes

smoothing methods for nonsmooth nonconvex minimization we consider a class of smoothing methods for minimization problems where the feasible set is convex but the objective function is not convex not differentiable and perhaps not even locally lipschitz at the solutions such optimization problems arise from wide applications including image restoration signal reconstruction variable selection optimal control stochastic equilibrium and spherical approximations in this paper we focus on smoothing methods for solving such optimization problems which use the structure of the minimization problems and composition of smoothing functions for the plus function x many existing optimization algorithms and codes can be used in the inner iteration of the smoothing methods we present properties of the smoothing functions and the gradient consistency of subdifferential associated with a smoothing function moreover we describe how to update the smoothing parameter in the outer iteration of the smoothing methods to guarantee convergence of the smoothing methods to a stationary point of the original minimization problem

model free control of nonlinear stochastic systems with discrete time measurements consider the problem of developing a controller for general nonlinear and stochastic systems where the equations governing the system are unknown using discrete time measurement this paper presents an approach for estimating a controller without building or assuming a model for the system such an approach has potential advantages in accommodating complex systems with possibly time varying dynamics the controller is constructed through use of a function approximator such as a neural network or polynomial this paper considers the use of the simultaneous perturbation stochastic approximation algorithm which requires only system measurements a convergence result for stochastic approximation algorithms with time varying objective functions and feedback is established it is shown that this algorithm can greatly enhance the efficiency over more standard stochastic approximation algorithms based on finite difference gradient approximations

a theoretical comparison of batch mode on line cyclic and almost cyclic learning we study and compare different neural network learning strategies batch mode learning online learning cyclic learning and almost cyclic learning incremental learning strategies require less storage capacity than batch mode learning however due to the arbitrariness in the presentation order of the training patterns incremental learning is a stochastic process whereas batch mode learning is deterministic in zeroth order i e as the learning parameter spl eta tends to zero all learning strategies approximate the same ordinary differential equation for convenience referred to as the ideal behavior using stochastic methods valid for small learning parameters spl eta we derive differential equations describing the evolution of the lowest order deviations from this ideal behavior we compute how the asymptotic misadjustment measuring the average asymptotic distance from a stable fixed point of the ideal behavior scales as a function of the learning parameter and the number of training patterns knowing the asymptotic misadjustment we calculate the typical number of learning steps necessary to generate a weight within order spl epsiv of this fixed point both with fixed and time dependent learning parameters we conclude that almost cyclic learning learning with random cycles is a better alternative for batch mode learning than cyclic learning learning with a fixed cycle