learning algorithm for boltzmann machines with additive weight and bias noise this brief presents analytical results on the effect of additive weight bias noise on a boltzmann machine bm in which the unit output is in 1 1 instead of 0 1 with such noise it is found that the state distribution is yet another boltzmann distribution but the temperature factor is elevated thus the desired gradient ascent learning algorithm is derived and the corresponding learning procedure is developed this learning procedure is compared with the learning procedure applied to train a bm with noise it is found that these two procedures are identical therefore the learning algorithm for noise free bms is suitable for implementing as an online learning algorithm for an analog circuit implemented bm even if the variances of the additive weight noise and bias noise are unknown

contact structures equivariant spin bordism and periodic fundamental groups using contact surgery and equivariant bordism theory we prove the existence of contact structures on all 5 dimensional spin manifolds with certain finite fundamental groups

sparse kernel regression modeling using combined locally regularized orthogonal least squares and d optimality experimental design the note proposes an efficient nonlinear identification algorithm by combining a locally regularized orthogonal least squares lrols model selection with a d optimality experimental design the proposed algorithm aims to achieve maximized model robustness and sparsity via two effective and complementary approaches the lrols method alone is capable of producing a very parsimonious model with excellent generalization performance the d optimality design criterion further enhances the model efficiency and robustness an added advantage is that the user only needs to specify a weighting for the d optimality cost in the combined model selecting criterion and the entire model construction procedure becomes automatic the value of this weighting does not influence the model selection procedure critically and it can be chosen with ease from a wide range of values

the yamabe invariant for non simply connected manifolds the yamabe invariant is an invariant of a closed smooth manifold defined using conformal geometry and the scalar curvature recently petean showed that the yamabe invariant is nonnegative for all closed simply connected manifolds of dimension 5 we extend this to show that yamabe invariant is nonnegative for all closed manifolds of dimension 5 with fundamental group of odd order having all sylow subgroups abelian the main new geometric input is a way of studying the yamabe invariant on toda brackets a similar method of proof shows that all closed oriented manifolds of dimension 5 with non spin universal cover with finite fundamental group having all sylow subgroups elementary abelian admit metrics of positive scalar curvature once one restricts to the complement of manifolds whose homology classes are toral the exceptional toral homology classes only exist in dimensions not exceeding the rank of the fundamental group so this proves important cases of the gromov lawson rosenberg conjecture once the dimension is sufficiently large

similarities of error regularization sigmoid gain scaling target smoothing and training with jitter the generalization performance of feedforward layered perceptrons can in many cases be improved either by smoothing the target via convolution regularizing the training error with a smoothing constraint decreasing the gain i e slope of the sigmoid nonlinearities or adding noise i e jitter to the input training data in certain important cases the results of these procedures yield highly similar results although at different costs training with jitter for example requires significantly more computation than sigmoid scaling

large sample behaviour of the product limit estimator on the whole line weak convergence results are proved for the product limit estimator on the whole line applications are given to confidence band construction estimation of mean lifetime and to the theory of q functions the results are obtained using stochastic calculus and in probability linear bounds for empirical processes

comparison of some neural network and scattered data approximations the inverse manipulator kinematics example this paper compares the application of five different methods for the approximation of the inverse kinematics of a manipulator arm from a number of joint angle cartesian coordinate training pairs the first method is a standard feedforward neural network with error backpropagation learning the next two methods are derived from an extended kohonen map algorithm that we combine with shepard interpolation for the forward computation we compare the method of ritter et al for the learning of the extended kohonen map to our own scheme based on gradient descent optimization we also study three scattered data approximation algorithms they include two variants of the radial basis function rbf method hardy s multiquadrics and gaussian rbf we further develop our own local polynomial fit method that could be considered as a modification of mclain s method we propose extensions to the considered scattered data approximation algorithms to make them suitable for vector valued multivariable functions such as the mapping of cartesian coordinates into joint angle coordinates

improving the generalization properties of radial basis function neural networks an important feature of radial basis function neural networks is the existence of a fast linear learning algorithm in a network capable of representing complex nonlinear mappings satisfactory generalization in these networks requires that the network mapping be sufficiently smooth we show that a modification to the error functional allows smoothing to be introduced explicitly without significantly affecting the speed of training a simple example is used to demonstrate the resulting improvement in the generalization properties of the network

genetic evolution of radial basis function coverage using orthogonal niches a well performing set of radial basis functions rbfs can emerge from genetic competition among individual rbfs genetic selection of the individual rbfs is based on credit sharing which localizes competition within orthogonal niches these orthogonal niches are derived using singular value decomposition and are used to apportion credit for the overall performance of the rbf network among individual nonorthogonal rbfs niche based credit apportionment facilitates competition to fill each niche and hence to cover the training data the resulting genetic algorithm yields rbf networks with better prediction performance on the mackey glass chaotic time series than rbf networks produced by the orthogonal least squares method and by k means clustering

functional approximation by feed forward networks a least squares approach to generalization this paper considers a least squares approach to function approximation and generalization the particular problem addressed is one in which the training data are noiseless and the requirement is to define a mapping that approximates the data and that generalizes to situations in which data samples are corrupted by noise in the input variables the least squares approach produces a generalizer that has the form of a radial basis function network for a finite number of training samples the finite sample approximation is valid provided that the perturbations due to noise on the expected operating conditions are large compared to the sample spacing in the data space in the other extreme of small noise perturbations a particular parametric form must be assumed for the generalizer it is shown that better generalization will occur if the error criterion used in training the generalizer is modified by the addition of a specific regularization term this is illustrated by an approximator that has a feedforward architecture and is applied to the problem of point source location using the outputs of an array of receivers in the focal plane of a lens

evolving space filling curves to distribute radial basis functions over an input space an evolutionary neural network training algorithm is proposed for radial basis function rbf networks the locations of basis function centers are not directly encoded in a genetic string but are governed by space filling curves whose parameters evolve genetically this encoding causes each group of codetermined basis functions to evolve to fit a region of the input space a network produced from this encoding is evaluated by training its output connections only networks produced by this evolutionary algorithm appear to have better generalization performance on the mackey glass time series than corresponding networks whose centers are determined by k means clustering

methods for choosing the regularization parameter and estimating the noise variance in image restoration and their relation the application of regularization to ill conditioned problems necessitates the choice of a regularization parameter which trades fidelity to the data with smoothness of the solution the value of the regularization parameter depends on the variance of the noise in the data the problem of choosing the regularization parameter and estimating the noise variance in image restoration is examined an error analysis based on an objective mean square error mse criterion is used to motivate regularization two approaches for choosing the regularization parameter and estimating the noise variance are proposed the proposed and existing methods are compared and their relationship to linear minimum mean square error filtering is examined experiments are presented that verify the theoretical results

additive hazards model with multivariate failure time data summary marginal additive hazards models are considered for multivariate survival data in which individuals may experience events of several types and there may also be correlation between individuals estimators are proposed for the parameters of such models and for the baseline hazard functions the estimators of the regression coefficients are shown asymptotically to follow a multivariate normal distribution with a sandwich type covariance matrix that can be consistently estimated the estimated baseline and subject specific cumulative hazard processes are shown to converge weakly to a zero mean gaussian random field the weak convergence properties for the corresponding survival processes are established a resampling technique is proposed for constructing simultaneous confidence bands for the survival curve of a specific subject the method ology is extended to a multivariate version of a class of partly parametric additive hazards model simulation studies are conducted to assess finite sample properties and the method is illustrated with an application to development of coronary heart diseases and cardiovascular accidents in the framingham heart study

using additive noise in back propagation training the possibility of improving the generalization capability of a neural network by introducing additive noise to the training samples is discussed the network considered is a feedforward layered neural network trained with the back propagation algorithm back propagation training is viewed as nonlinear least squares regression and the additive noise is interpreted as generating a kernel estimate of the probability density that describes the training vector distribution two specific application types are considered pattern classifier networks and estimation of a nonstochastic mapping from data corrupted by measurement errors it is not proved that the introduction of additive noise to the training vectors always improves network generalization however the analysis suggests mathematically justified rules for choosing the characteristics of noise if additive noise is used in training results of mathematical statistics are used to establish various asymptotic consistency results for the proposed method numerical simulations support the applicability of the training method

a counterexample to a conjecture concerning the hall wellner band hall and wellner proposed a natural extension of the kolmogorov smirnov simultaneous confidence band for survival curve using the kaplan meier estimator they and gill conjectured that the confidence band holds for all t up to the last observed failure time a counterexample is given herein showing that this may not always be true

on different facets of regularization theory this review provides a comprehensive understanding of regularization theory from different perspectives emphasizing smoothness and simplicity principles using the tools of operator theory and fourier analysis it is shown that the solution of the classical tikhonov regularization problem can be derived from the regularized functional defined by a linear differential integral operator in the spatial fourier domain state of the art research relevant to the regularization theory is reviewed covering occam s razor minimum length description bayesian theory pruning algorithms informational entropy theory statistical learning theory and equivalent regularization the universal principle of regularization in terms of kolmogorov complexity is discussed finally some prospective studies on regularization theory and beyond are suggested

combined learning and use for a mixture model equivalent to the rbf classifier we show that the decision function of a radial basis function rbf classifier is equivalent in form to the bayes optimal discriminant associated with a special kind of mixture based statistical model the relevant mixture model is a type of mixture of experts model for which class labels like continuous valued features are assumed to have been generated randomly conditional on the mixture component of origin the new interpretation shows that rbf classifiers effectively assume a probability model which moreover is easily determined given the designed rbf this interpretation also suggests a statistical learning objective as an alternative to standard methods for designing the rbf equivalent models the statistical objective is especially useful for incorporating unlabeled data to enhance learning finally it is observed that any new data to classify are simply additional unlabeled data thus we suggest a combined learning and use paradigm to be invoked whenever there are new data to classify

noise injection theoretical prospects noise injection consists of adding noise to the inputs during neural network training experimental results suggest that it might improve the generalization ability of the resulting neural network a justification of this improvement remains elusive describing analytically the average perturbed cost function is difficult and controlling the fluctuations of the random perturbed cost function is hard hence recent papers suggest replacing the random perturbed cost by a deterministic taylor approximation of the average perturbed cost function this article takes a different stance when the injected noise is gaussian noise injection is naturally connected to the action of the heat kernel this provides indications on the relevance domain of traditional taylor expansions and shows the dependence of the quality of taylor approximations on global smoothness properties of neural networks under consideration the connection between noise injection and heat kernel also enables controlling the fluctuations of

the image of j in the ehp sequence the ehp sequence is based on a result of james j who showed that there is h a map h such that s q sn s2n 1 is a fibration when localized at 2 the map sn sn is usually labeled e the boundary homomorphism in the homotopy sequence is usually labeled p for our purposes it will be most convenient to combine all the ehp sequences into one system this gives the following filtration of o201 s q s

on the geometry of feedforward neural network error surfaces many feedforward neural network architectures have the property that their overall input output function is unchanged by certain weight permutations and sign flips in this paper the geometric structure of these equioutput weight space transformations is explored for the case of multilayer perceptron networks with tanh activation functions similar results hold for many other types of neural networks it is shown that these transformations form an algebraic group isomorphic to a direct product of weyl groups results concerning the root spaces of the lie algebras associated with these weyl groups are then used to derive sets of simple equations for minimal sufficient search sets in weight space these sets which take the geometric forms of a wedge and a cone occupy only a minute fraction of the volume of weight space a separate analysis shows that large numbers of copies of a network performance function optimum weight vector are created by the action of the equioutput transformation group and that these copies all lie on the same sphere some implications of these results for learning are discussed

a comparison of the von mises and gaussian basis functions for approximating spherical acoustic scatter this paper compares the approximation accuracy of two basis functions that share a common radial basis function rbf neural network architecture used for approximating a known function on the unit sphere the basis function types considered are that of a new spherical basis function the von mises function and the now well known gaussian basis function gradient descent learning rules were applied to optimize learn the solution for both approximating basis functions a benchmark approximation problem was used to compare the performance of the two types of basis functions in this case the mathematical expression for the scattering of an acoustic wave striking a rigid sphere

asmo dan algorithm for adaptive spline modelling of observation data nonlinear system identification by modelling the underlying relationships in observation data is an important application area for artificial neural networks and other learning paradigms splines have been used for scattered data interpolation but the applications have mainly been restricted to low dimensional input spaces this paper describes asmod a new learning paradigm for higher dimensional data 3 based on b spline interpolation the models can be trained online and a method for step wise model refinement is applied during model training for gradually increasing the modelling capability until the desired or best possible accuracy is obtained for every refinement step a number of possible refinement actions are evaluated and the one that gives the highest improvement of the model accuracy is chosen the model structure is hence adapted to the modelling problem giving a model of small size and high accuracy asmod has very efficient implementations on serial computers the scheme has been

eta invariants and their adiabatic limits 0 introduction 1 algebraic preliminaries 2 the finite dimensional case superconnections a odd dimensional base spaces b even dimensional base spaces appendix 1 coupling to general superconnections 3 auxiliary grassman variables and auxiliary adiabatic limits 4 the infinite dimensional case fibrations a elementary geometry of fibrations b odd dimensional base spaces c even dimensional base spaces d the case in which the dirac operator on the fiber is not invertible appendix 2 multifibrations

real connective k theory and the quaternion group let ko be the real connective k theory spectrum we compute ko bg and ko bg for groups g whose sylow 2 subgroup is quaternion of order 8 using this we compute the coefficients t ko g of the g fixed points of the tate spectrum t ko for g s12 3 and g q8 the results provide a counterexample to the optimistic conjecture of greenlees and may 9 conj 13 4 by showing in particular that t ko g is not a wedge of eilenberg mac lane spectra as occurs for groups of prime order