a parameter method for computing highly oscillatory integrals this paper presents a new efficient parameter method for integration of the highly oscillatory integral 0 1f x e i w g x dx with an irregular oscillator the effectiveness and accuracy are tested by means of numerical examples for the case where g x has stationary points

note on the homotopy perturbation method for multivariate vector value oscillatory integrals this paper considers a homotopy perturbation method for approximating multivariate vector value highly oscillatory integrals the asymptotic formulae of the integrals and the asymptotic order of the asymptotic method are presented numerical examples show the efficiency of the approximation method

learning unrealizable tasks from minimum entropy queries in supervised learning learning from queries rather than from random examples can improve generalization performance significantly we study the performance of query learning for unrealizable tasks where the student cannot learn from the perfectly as a simple model scenario of this kind we consider a linear perceptron student learning a general nonlinear perceptron teacher two kinds of queries for maximum information gain i e minimum entropy are investigated minimum student space entropy msse queries which are appropriate if the teacher space is unknown and minimum teacher space entropy mtse queries which can be used if the teacher space is assumed to be known but a student of a simpler form has deliberately been chosen we find that for msse queries the structure of the student space determines the efficacy of query learning mtse queries on the other hand which we investigate for the extreme case of a binary perceptron teacher lead to a higher generalization error than random examples due to a lack of feedback about the progress of the student in the way queries are selected

galerkinlevin method for highly oscillatory integrals in this paper a new method namely galerkinlevin method is proposed to investigate the highly oscillatory integrals indeed we construct an algorithm based on the levin and galerkin methods using the levin method the given integral is transformed into a certain ordinary differential equation then a galerkin method is applied for solving the obtained ode the efficiency of the approach will be shown by applying the procedure on some prototype examples a new algorithm for solving the highly oscillatory integrals is proposed using the levin method a given integral is transformed into a certain ode error and convergence analysis of the proposed algorithm is performed

the effects of adding noise during backpropagation training on a generalization performance we study the effects of adding noise to the inputs outputs weight connections and weight changes of multilayer feedforward neural networks during backpropagation training we rigorously derive and analyze the objective functions that are minimized by the noise affected training processes we show that input noise and weight noise encourage the neural network output to be a smooth function of the input or its weights respectively in the weak noise limit noise added to the output of the neural networks only changes the objective function by a constant hence it cannot improve generalization input noise introduces penalty terms in the objective function that are related to but distinct from those found in the regularization approaches simulations have been performed on a regression and a classification problem to further substantiate our analysis input noise is found to be effective in improving the generalization performance for both problems however weight noise is found to be effective in improving the generalization performance only for the classification problem other forms of noise have practically no effect on generalization

similarities of error regularization sigmoid gain scaling target smoothing and training with jitter the generalization performance of feedforward layered perceptrons can in many cases be improved either by smoothing the target via convolution regularizing the training error with a smoothing constraint decreasing the gain i e slope of the sigmoid nonlinearities or adding noise i e jitter to the input training data in certain important cases the results of these procedures yield highly similar results although at different costs training with jitter for example requires significantly more computation than sigmoid scaling

two low accuracy methods for stiff systems two low accuracy explicit one step methods for stiff ordinary differential equations are extended directly to solve systems of equations some defects of the component form of these methods are avoided to perform these a new set of vector computations are introduced some numerical experiments are presented to show the superiority of the new methods

filon clenshaw curtis rules for a class of highly oscillatory integrals with logarithmic singularities in this work we propose and analyse a numerical method for computing a family of highly oscillatory integrals with logarithmic singularities for these quadrature rules we derive error estimates in terms of n the number of nodes k the rate of oscillations and a sobolev like regularity of the function we prove that the method is not only robust but the error even decreases for fixed n as k increases practical issues about the implementation of the rule are also covered in this paper by a writing down ready to implement algorithms b analysing the numerical stability of the computations and c estimating the overall computational cost we finish by showing some numerical experiments which illustrate the theoretical results presented in this paper

the equivalence of weak strong and complete convergence in l_1 for kernel density estimates theorem 1 let k be a nonnegative borel measurable function on rd with f k x dx 1 then the following conditions are equivalent i j 0 in probability as n oo some f ii jn o 0 in probability as n s oo all f iii jn s 0 almost surely as n oo all f iv jn 0 exponentially as n oc i e for all e 0 there exist r no 0 such that p jne no all f v lim0h 0 and limnnhd oo also v implies iv when k is merely absolutely integrable and f k x dx 1 0

nonparametric density estimation the l 1 view differentiation of integrals consistency lower bounds for rates of convergence rates of convergence in l1 the automatic kernel estimate l1 and pointwise convergence estimates related to the kernel estimate and the histogram estimate simulation inequalities and random variate generation the transformed kernel estimate applications in discrimination operations on density estimates estimators based on orthogonal series index

learning with noise in a linear perceptron the learning of a set of p random patterns in a linear perceptron is studied in the limit of a large n of input units with noise on the weights inputs and output the problem is formulated in continuous time as a langevin equation and the first task is to evaluate the response or green function for the system white noise on the output is shown to correspond to spatially correlated weight noise acting only in a subspace of the weight space it is shown that the input noise acts as a simple weight decay with a size proportional to the load parameter alpha p n with no weight decay the relaxation time diverges at alpha 1 with a weight decay it becomes shorter and finite for alpha 1 but at the cost of a larger asymptotic learning error that is found analytically it is shown that a small weight decay decreases the effect of noise on the weights or outputs

letter to the editor on the filon and levin methods for highly oscillatory integral this paper shows that for any suitably smooth function f x and arbitrarily selected interpolation nodes c1 c2 cv in a b the filon method and the levin method for with the polynomial interpolation approach are identical when g x is a linear function based on this result a new efficient levin quadrature for is presented

on the quadrature of multivariate highly oscillatory integrals over non polytope domains in this paper we present a levin type method for approximating multivariate highly oscillatory integrals subject to a non resonance condition unlike existing methods we do not require the knowledge of moments which enables us to derive an approximation when the oscillator is complicated and when the domain is neither a simplex nor a polytope the accuracy of this method improves as the frequency of oscillations increases a special case of this method has the property that the asymptotic order increases with each additional sample point

generalized hessians of c 1 1 functions and second order viscosity subjets given a c 1 1 function f u rightarrow r where u subset r n open we deal with the question of whether or not at a given point x 0 in u there exists a local minorant varphi of f of class c 2 that satisfies varphi x 0 f x 0 d varphi x 0 df x 0 and d 2 varphi x 0 in mathcal h f x 0 the generalized hessian of f at x 0 this question is motivated by the second order viscosity theory of the pdes since for nonsmooth functions an analogous result between subgradients and first order viscosity subjets is known to hold in every separable asplund space in this work we show that the aforementioned second order result holds true whenever mathcal h f x 0 has a minimum with respect to the positive semidefinite cone thus in particular in one dimension but it fails in two dimensions even for piecewise polynomial functions we extend this result by introducing a new notion of directional minimum of mathcal h f x 0

high order nonlinear volterra fredholm hammerstein integro differential equations and their effective computation in a recent study turkyilmazoglu 2014 high order linear fredholm integro differential equations were solved by means of an elegant and accurate effective technique this approach is extended here to obtain exact and analytic approximate solutions of high order nonlinear volterra fredholm hammerstein integro differential equations difference from the earlier work is that the method involves solution of nonlinear algebraic equations in place of linear ones high accuracy of the introduced technique is also observed for the nonlinear equations via comparisons with some of the available methods

using additive noise in back propagation training the possibility of improving the generalization capability of a neural network by introducing additive noise to the training samples is discussed the network considered is a feedforward layered neural network trained with the back propagation algorithm back propagation training is viewed as nonlinear least squares regression and the additive noise is interpreted as generating a kernel estimate of the probability density that describes the training vector distribution two specific application types are considered pattern classifier networks and estimation of a nonstochastic mapping from data corrupted by measurement errors it is not proved that the introduction of additive noise to the training vectors always improves network generalization however the analysis suggests mathematically justified rules for choosing the characteristics of noise if additive noise is used in training results of mathematical statistics are used to establish various asymptotic consistency results for the proposed method numerical simulations support the applicability of the training method

infinite dimensional generalized jacobian properties and calculus rules the extension to infinite dimensional domains of clarke s generalized jacobian is the focus of this paper first a generalization of a fabian preiss theorem to the infinite dimensional setting is obtained as a consequence a new formula relating the clarke s generalized jacobians corresponding to finite dimensional spaces k l with k l is established furthermore in the infinite dimensional case basic properties pertaining the generalized jacobian are developed and then an identification of this set valued map is produced applications of these results in the form of chain rules including sum and product rules and a computational formula for continuous selections are derived

analysis of a collocation method for integrating rapidly oscillatory functions a collocation method for approximating integrals of rapidly oscillatory functions is analyzed the method is efficient for integrals involving bessel functions j rx with a large oscillation frequency parameter r as well as for many other one and multi dimensional integrals of functions with rapid irregular oscillations the analysis provides a convergence rate and it shows that the relative error of the method is even decreasing as the frequency of the oscillations increases ko words oscillatory integrals collocation analysis

a sparse fractional jacobi galerkin levin quadrature rule for highly oscillatory integrals abstract levin s quadrature rule is well known for its good performance on numerical treatment with highly oscillatory integrals however most existing work treats with sufficiently smooth integrands in this paper we study levin s quadrature rule for a class of singular and oscillatory integrals based on fractional jacobi polynomials a class of differential equation solvers for levin s equation are developed which leads to the fractional jacobi galerkin levin method the discretized equation is turned into a sparse linear system by properly choosing jacobi polynomials and the inner product furthermore convergence analysis with respect to the oscillation is presented by studying coefficients of the fractional jacobi expansion of the error function numerical experiments indicate that in contrast to existing quadrature rules the new method is efficient for computing oscillatory integrals with stationary points and unknown singular parameters

noise injection theoretical prospects noise injection consists of adding noise to the inputs during neural network training experimental results suggest that it might improve the generalization ability of the resulting neural network a justification of this improvement remains elusive describing analytically the average perturbed cost function is difficult and controlling the fluctuations of the random perturbed cost function is hard hence recent papers suggest replacing the random perturbed cost by a deterministic taylor approximation of the average perturbed cost function this article takes a different stance when the injected noise is gaussian noise injection is naturally connected to the action of the heat kernel this provides indications on the relevance domain of traditional taylor expansions and shows the dependence of the quality of taylor approximations on global smoothness properties of neural networks under consideration the connection between noise injection and heat kernel also enables controlling the fluctuations of

anisotropic noise injection for input variables relevance determination there are two archetypal ways to control the complexity of a flexible regressor subset selection and ridge regression in neural networks jargon they are respectively known as pruning and weight decay these techniques may also be adapted to estimate which features of the input space are relevant for predicting the output variables relevance is given by a binary indicator for subset selection and by a continuous rating for ridge regression this paper shows how to achieve such a rating for a multilayer perceptron trained with noise or jitter noise injection ni is modified in order to penalize heavily irrelevant features the proposed algorithm is attractive as it requires the tuning of a single parameter this parameter controls the complexity of the model effective number of parameters together with the rating of feature relevances effective input space dimension bounds on the effective number of parameters support that the stability of this adaptive scheme is enforced by the constraints applied to the admissible set of relevance indices the good properties of the algorithm are confirmed by satisfactory experimental results on simulated data sets

a generalized second order derivative in nonsmooth optimization in this work a new notion of generalized second order directional derivative and generalized hessian for nonsmooth real valued functions is studied the general properties of these mathematical objects are investigated together with some calculus rules that may facilitate their practical computation two applications of these derivatives in optimization theory are considered first to obtaining necessary and sufficient second order optimality conditions for problems with or without constraints and second to extending the newton method for the minimization of a mathcal c 1 1 function

efficient numerical methods for bessel type of oscillatory integrals in this paper new quadrature rules are proposed for numerical evaluation of highly oscillatory integrals containing first kind of bessel functions j x meshless procedure with uniform and scattered nodes is used to cope with frequent irregular oscillations caused by bessel and bessel trigonometric functions in addition multi resolution quadrature rules based on hybrid functions and haar wavelets are used as supporting tools to handle the case of singularity of the meshless collocation method error bounds of the proposed methods are calculated and numerically verified by solving some benchmark test problems

an asymptotically efficient solution to the bandwidth problem of kernel density estimation a data driven method of choosing the bandwidth h of a kernel density estimator is heuristically motivated by considering modifications of the kullback leibler or pseudo likelihood cross validation function it is seen that this means of choosing h is asymptotically equivalent to taking the h that minimizes some compelling error criteria such as the average squared error and the integrated squared error thus for a given kernel function the bandwidth can be chosen optimally without making precise smoothness assumptions on the underlying density

an analysis of noise in recurrent neural networks convergence and generalization concerns the effect of noise on the performance of feedforward neural nets we introduce and analyze various methods of injecting synaptic noise into dynamically driven recurrent nets during training theoretical results show that applying a controlled amount of noise during training may improve convergence and generalization performance we analyze the effects of various noise parameters and predict that best overall performance can be achieved by injecting additive noise at each time step noise contributes a second order gradient term to the error function which can be viewed as an anticipatory agent to aid convergence this term appears to find promising regions of weight space in the beginning stages of training when the training error is large and should improve convergence on error surfaces with local minima the first order term is a regularization term that can improve generalization specifically it can encourage internal representations where the state nodes operate in the saturated regions of the sigmoid discriminant function while this effect can improve performance on automata inference problems with binary inputs and target outputs it is unclear what effect it will have on other types of problems to substantiate these predictions we present simulations on learning the dual parity grammar from temporal strings for all noise models and present simulations on learning a randomly generated six state grammar using the predicted best noise model

generalization in a linear perceptron in the presence of noise the authors study the evolution of the generalization ability of a simple linear perceptron with n inputs which learns to imitate a teacher perceptron the system is trained on p alpha n example inputs drawn from some distribution and the generalization ability is measured by the average agreement with the teacher on test examples drawn from the same distribution the dynamics may be solved analytically and exhibits a phase transition from imperfect to perfect generalization at alpha 1 when there are no errors static noise in the training examples if the examples are produced by an erroneous teacher overfitting is observed i e the generalization error starts to increase after a finite time of training it is shown that a weight decay of the same size as the variance of the noise errors on the teacher improves on the generalization and suppresses the overfitting the generalization error as a function of time is calculated numerically for various values of the parameters finally dynamic noise in the training is considered white noise on the input corresponds on average to a weight decay and can thus improve generalization whereas white noise on the weights or the output degrades generalization generalization is particularly sensitive to noise on the weights for alpha 1 where it makes the error constantly increase with time but this effect is also shown to be damped by a weight decay weight noise and output noise acts similarly above the transition at alpha 1

new quadrature rules for highly oscillatory integrals with stationary points in this paper new algorithms are being proposed for evaluation of highly oscillatory integrals hois with stationary point s the algorithms are based on modified levin quadrature mlq with multiquadric radial basis functions rbfs coupled with quadrature rules based on hybrid functions of order 8 hfq8 and haar wavelets quadrature hwq aziz et al 2011 part of the new procedure presented in this paper is comprised of transplanting monomials which are used in the conventional levin method by the rbfs the linear and hermite polynomials based quadratures xiang 2007 are being replaced by the new methods based on hwq and hfq8 respectively both the methods are merged with mlq to obtain the numerical solution of highly oscillatory integrals having stationary points the accuracy of the new methods is neither dampened by presence of the stationary point s nor by the large value of frequency parameter theoretical facts about the error analysis of the new methods are analyzed and proved numerical examples are included to show efficiency and accuracy of the new methods a new framework for the numerical solution of highly oscillatory integrals is proposed the integrals are bifurcated in the neighborhood of stationary point the integral on the smaller subinterval is solved by hybrid functions and haar wavelets the integral on the longer subinterval is solved by the meshless method with mq radial basis functions convergence analysis of the proposed methods is performed

numerical solutions of highly oscillatory integrals abstract in this paper an application of he s homotopy perturbation method hpm is proposed to numerical solution of highly oscillatory integrals to apply the hpm to the oscillatory integrals we assume that the oscillatory function has not critical point at the endpoints of integration region the results reveal that the method is very effective and simple

support functions of the clarke generalized jacobian and of its plenary hull this paper studies two important mathematical objects which are useful in tackling the first order behaviour of vector valued locally lipschitz functions in a finite dimensional setting the clarke generalized jacobian and its plenary hull we aim at giving analytical expressions of the support functions of these compact convex sets of matrices our study was motivated by earlier works by j b hiriart urruty and recent papers by zs pales and v zeidan the expressions of the support functions are applied for instance to provide a new proof of a chain rule on generalized jacobians of composed locally lipschitz functions without further assumptions applications of our results to the second order behaviour of c 1 functions with locally lipschitz gradients are considered

generalized hessian for c 1 1 functions in infinite dimensional normed spaces the subject of this paper is the systematic study of second order notions concerning differentiable functions with lipschitz derivative the results and notions are motivated by recent papers of cominetti correa and hiriart urruty the first goal of this paper is the comparison of several known second order directional derivatives the second goal is the introduction of a generalized hessian which is a set of certain symmetric bilinear forms the relation of this generalized hessian to other existing second order derivatives is also described