estimation of gradients and coordinate covariation in classification we introduce an algorithm that simultaneously estimates a classification function as well as its gradient in the supervised learning framework the motivation for the algorithm is to find salient variables and estimate how they covary an efficient implementation with respect to both memory and time is given the utility of the algorithm is illustrated on simulated data as well as a gene expression data set an error analysis is given for the convergence of the estimate of the classification function and its gradient to the true classification function and true gradient

volterra discrete equations summability of the fundamental matrix conditions are proven which assure the summability of the first difference of the fundamental matrix of nonconvolution volterra discrete equations these conditions are applied to the stability analysis of some linear methods for solving volterra integral equations of nonconvolution type

variable step size predictor corrector schemes for second kind volterra integral equations in this paper a family of implicit multistep methods for the solution of volterra integral equations is derived these methods together with an explicit euler predictor permit the use of a variable step size when solving integral equations means of controlling the error and stability by varying the step size and the order of the method are described extensive numerical results are presented

nonpolynomial spline collocation for volterra equations with weakly singular kernels volterra integral equations of the second kind with weakly singular kernels possess in general solutions which are not smooth near the left endpoint of the interval of integration since ordinary polynomial spline collocation cannot lead to high order convergence we introduce special nonpolynomial spline spaces which are modelled after the structure of these solutions near the point of nonsmooth behavior collocation in these spaces will once more lead to high order methods analogous results are derived for volterra integro differential equations with weakly singular kernels

artificial boundary conditions for two dimensional incompressible viscous flows around an obstacle abstract in this paper we consider numerical simulation of incompressible viscous flow around an obstacle in velocity pressure formulation two horizontal straight line artificial boundaries are introduced and the original flow is approximated by a flow in an infinite channel with slip boundary condition on the wall then two vertical segment artificial boundaries are introduced to limit the channel to a bounded computational domain in the region of the channel between the vertical boundaries and infinity the velocity of the flow is almost a constant vector in which the navier stokes equations can be linearised by oseen equations and thus a general solution can be derived by using separation of variables artificial boundary conditions on the vertical segments are then designed by imposing the continuity of velocity and normal stress therefore the original problem is reduced to a boundary value problem on a bounded computational domain numerical example shows that our artificial boundary conditions are very effective

artificial boundary conditions for incompressible navier stokes equations a well posed result abstract numerical simulation of two dimensional incompressible viscous flows around an obstacle is considered two horizontal straight line artificial boundaries are introduced and the original flow is approximated by a flow in an infinite channel with slip boundary condition on the wall then two vertical segment artificial boundaries are introduced and a series of approximate artificial boundary conditions on them are derived by imposing the continuity of velocity and the normal stress thus the original problem is reduced to a problem defined in a bounded computational domain the well posedness of the reduced problem is proved the finite element approximation of this problem is given and error estimates are obtained furthermore numerical examples show the accuracy and efficiency of our artificial boundary conditions

implicit runge kutta nystrom methods for general second order volterra integro differential equations abstract in this paper we derive m stage runge kutta nystrom methods for the numerical solution of general second order volterra integro differential equations these implicit methods are based on collocation techniques in certain polynomial spline spaces the question concerning the attainable order of convergence of such one step methods is completely answered

generalization error bounds for bayesian mixture algorithms bayesian approaches to learning and estimation have played a significant role in the statistics literature over many years while they are often provably optimal in a frequentist setting and lead to excellent performance in practical applications there have not been many precise characterizations of their performance for finite sample sizes under general conditions in this paper we consider the class of bayesian mixture algorithms where an estimator is formed by constructing a data dependent mixture over some hypothesis space similarly to what is observed in practice our results demonstrate that mixture approaches are particularly robust and allow for the construction of highly complex estimators while avoiding undesirable overfitting effects our results while being data dependent in nature are insensitive to the underlying model assumptions and apply whether or not these hold at a technical level the approach applies to unbounded functions constrained only by certain moment conditions finally the bounds derived can be directly applied to non bayesian mixture approaches such as boosting and bagging

on the generalization ability of on line learning algorithms in this paper it is shown how to extract a hypothesis with small risk from the ensemble of hypotheses generated by an arbitrary on line learning algorithm run on an independent and identically distributed i i d sample of data using a simple large deviation argument we prove tight data dependent bounds for the risk of this hypothesis in terms of an easily computable statistic m sub n associated with the on line performance of the ensemble via sharp pointwise bounds on m sub n we then obtain risk tail bounds for kernel perceptron algorithms in terms of the spectrum of the empirical kernel matrix these bounds reveal that the linear hypotheses found via our approach achieve optimal tradeoffs between hinge loss and margin size over the class of all linear functions an issue that was left open by previous results a distinctive feature of our approach is that the key tools for our analysis come from the model of prediction of individual sequences i e a model making no probabilistic assumptions on the source generating the data in fact these tools turn out to be so powerful that we only need very elementary statistical facts to obtain our final risk bounds

asymptotic boundary conditions for dissipative waves general theory an outstanding issue in the computational analysis of time dependent problems is the imposition of appropriate radiation boundary conditions at artificial boundaries accurate conditions are developed which are based on the asymptotic analysis of wave propagation over long ranges employing the method of steepest descents dominant wave groups are identified and simple approximations to the dispersion relation are considered in order to derive local boundary operators the existence of a small number of dominant wave groups may be expected for systems with dissipation estimates of the error as a function of domain size are derived under general hypotheses leading to convergence results some practical aspects of the numerical construction of the asymptotic boundary operators are also discussed

online regularized classification algorithms this paper considers online classification learning algorithms based on regularization schemes in reproducing kernel hilbert spaces associated with general convex loss functions a novel capacity independent approach is presented it verifies the strong convergence of the algorithm under a very weak assumption of the step sizes and yields satisfactory convergence rates for polynomially decaying step sizes explicit learning rates with respect to the misclassification error are given in terms of the choice of step sizes and the regularization parameter depending on the sample size error bounds associated with the hinge loss the least square loss and the support vector machine q norm loss are presented to illustrate our method

the numerical treatment of volterra integro differential equations with unbounded delay abstract initial value problems for ordinary and partial volterra integro differential equations with infinite and finite delay arise in the mathematical modelling of various physical and biological phenomena in this paper we give a survey of some recent developments in the numerical treatment of such functional equations with the focus being on equations with infinite delay in addition we will discuss a number of numerical aspects that are not yet understood well

the piecewise polynomial collocation method for nonlinear weakly singular volterra equations second kind volterra integral equations with weakly singular kernels typically have solutions which are nonsmooth near the initial point of the interval of integration using an adaptation of the analysis originally developed for nonlinear weakly singular fredholm integral equations we present a complete discussion of the optimal global and local order of convergence of piecewise polynomial collocation methods on graded grids for nonlinear volterra integral equations with algebraic or logarithmic singularities in their kernels

an open boundary condition for the computation of the steady incompressible navier stokes equations abstract we design a new boundary condition on the stream function at outflow for the steady incompressible navier stokes equations written in the stream function vorticity formulation by using a fourier transform in the direction perpendicular to the flow our open boundary condition is compared to the dirichlet and neumann boundary conditions by computing the flow past an ellipse of excentricity 50 our boundary condition proves to be the most accurate and thus the most time saving

on monotone abel volterra integral equations on the half line in this paper we reanalyze the trapezoidal method for the solution of nonlinear abel volterra integral equations on the half line we prove the convergence of the method in the uniform norm provided the nonlinearity is lipschitz continuous and strictly monotone

learning theory of randomized kaczmarz algorithm a relaxed randomized kaczmarz algorithm is investigated in a least squares regression setting by a learning theory approach when the sampling values are accurate and the regression function conditional means is linear such an algorithm has been well studied in the community of non uniform sampling in this paper we are mainly interested in the different case of either noisy random measurements or a nonlinear regression function in this case we show that relaxation is needed a necessary and sufficient condition on the sequence of relaxation parameters or step sizes for the convergence of the algorithm in expectation is presented moreover polynomial rates of convergence both in expectation and in probability are provided explicitly as a result the almost sure convergence of the algorithm is proved by applying the borel cantelli lemma

improving the accuracy of collocation solutions of volterra integral equations of the first kind by local interpolation we study the numerical solution of volterra integral equations of the first kind by collocation methods using piecewise polynomials of degreep we show that when superconvergence occurs the error in the collocation solution at the node points alternates in sign independent from the kernel or the exact solution evaluation at certain points of interpolating polynomials of degreep 2 annihilates the leading term in the asymptotic expansion of the error thus yielding convergence of orderp 3 the derivation of the asymptotic expansion of the error for the general equation is reduced to the special case of numerical differentiation by the use of an asymptotic continuity result for the projectors associated with the collocation method under compact perturbations we present some numerical results on the improved order of convergence forp 2 and 3

high order methods for the numerical solution of volterra integro differential equations abstract if a first order volterra integro differential equation is solved by collocation in the space of continuous polynomial splines of degree m 1 with collocation occuring at the gauss legendre points then the resulting approximation u converges at its knots like o h 2m while its derivative u exhibits only o h m convergence this paper deals with the question of how to choose the collocation points so that both u and u converge like o h q with q maximal

artificial boundary conditions for the linear advection diffusion equation a family of artificial boundary conditions for the linear advection diffusion equation with small viscosity is developed well posedness for the associated initial boundary value problem is analyzed the error produced by truncating the domain is estimated numerical results are presented 1 introduction when computing the solution of a partial differential equation in an unbounded domain one often introduces artificial boundaries in order to limit the computational cost these boundaries must be chosen not too far from the domain of interest therefore the boundary conditions must be good approxima tions to the so called transparent i boundary condition i e such ithat the solution of the problem in the bounded domain is equal to the solution in the original domain the transparent boundary condition is usually an integral relation in time and space between u and its normal derivative on the boundary which makes it impractical from a numerical point of view one must approximate this relation to get local boundary conditions they are often called absorbing or artificial boundary conditions this question is of crucial interest in such different areas as geophysics plasma physics fluid dynamics 1 2 3 and the use of such conditions is now classical in geophysics our interest for the linear advection diffusion equation comes from the navier stokes equation but it arises also in other fields as for example meteorology 6 the incompressible navier stokes equation can be written as

collocation methods for nonlinear volterra integro differential equations with infinite delay in this paper we study the numerical solution of nonlinear volterra integrodifferential equations with infinite delay by spline collocation and related runge kutta type methods the kernel function in these equations is of the form k t 8 y t y s with a representative example given by volterra s population equation where we have k t a y t y 8 a t s g y t y 8

linearized stability analysis of discrete volterra equations abstract various different types of stability are defined in a unified framework for discrete volterra equations of the type x n f n n j 0 k n j x n n 0 under appropriate assumptions stability results are obtainable from those valid in the linear case k n j x n b n j x j and a linearized stability theory is studied here by using the fundamental and resolvent matrices several necessary and sufficient conditions for stability are obtained for solutions of the linear equation by considering the equations in various choices of banach space b the elements of which are sequences of vectors x n f n e d b n j e d e d n j 0 etc we show that the theory including a number of new results as well as results already known can be presented in a systematic framework in which results parallel corresponding results for classical volterra integral equations of the second kind

order of convergence of one step methods for volterra integral equations of the second kind nice proofs of convergence and asymptotic expansions are known for one step methods for ordinary differential equations it is shown that these proofs can be generalized in a natural way to extended one step methods for volterra integral equations of the second kind furthermore the convergence of mixed one step methods is investigated for both types general volterra runge kutta methods are considered as examples

stability results for one step discretized collocation methods in the numerical treatment of volterra integral equations this paper is concerned with the stability analysis of the discretized collocation method for the second kind volterra integral equation with degenerate kernel a fixed order recurrence relation with variable coefficients is derived and local stability conditions are given independent of the discretization local stability and stability with respect to an isolated perturbation of some methods are proved the reliability of the derived stability conditions is shown by numerical experiments

stability results for collocation methods for volterra integral equations abstract stability behavior is investigated for polynomial spline collocation applied to volterra integral equations with special emphasis on a weakly singular test equation the characterization of the corresponding stability domain is related to stability results for solutions of finite recursion relations as they occur for methods applied to ordinary differential equations estimates of the stability domains are established and as a consequence conditions for a 2 stability are obtained exploiting these estimates collocation approximations with constant linear and continuous splines are considered in more depth and several numerical illustrations are presented

almost periodic solutions of discrete volterra equations for discrete volterra equations with or without delay we obtain several results concerning almost periodic solutions and asymptotically almost periodic solutions under certain conditions we also investigate the relations among solutions of equations discussed and give an example to illustrate our results

qualitative behaviour and stability of solutions of discretised nonlinear volterra integral equations of convolution type abstract in this work we consider equations of the form y t g t t o k t s y s ds te r and their discretised versions obtained through application of quadrature to of the form y n g n h j 0 n w n j j j y j n j e n subject to certain conditions on k g and the weights wj one purpose of the paper is to show how the discussion of qualitative behaviour and stability for can be mimicked in a discussion of we first describe corduneanu s 1973 discussion of stability for re presenting his material in a modified form which lends itself to adaptation for our discussion of we give a stability result for we then demonstrate the stability behaviour of some simple quadrature rules applied to an illustrative example equation and we observe that for a particular family of quadrature rules the qualitative behaviour of solutions to the example equation is preserved under the discretisation

a stability analysis of the trapezoidal method for volterra integral equations with completely positive kernels abstract the solution of the volterra integral equation with completely positive kernel y t 0t b t s y s ds u0 0t b t s g s ds t 0 is nonnegative and nonincreasing provided that g is nonincreasing and 0 g t u0 for any t 0 we prove that under some additional hypotheses this property is inherited by the solution of the recurrence relation resulting from applying the trapezoidal method to this equation

learning with sample dependent hypothesis spaces many learning algorithms use hypothesis spaces which are trained from samples but little theoretical work has been devoted to the study of these algorithms in this paper we show that mathematical analysis for these algorithms is essentially different from that for algorithms with hypothesis spaces independent of the sample or depending only on the sample size the difficulty lies in the lack of a proper characterization of approximation error to overcome this difficulty we propose an idea of using a larger function class not necessarily linear space containing the union of all possible hypothesis spaces varying with the sample to measure the approximation ability of the algorithm we show how this idea provides error analysis for two particular classes of learning algorithms in kernel methods learning the kernel via regularization and coefficient based regularization we demonstrate the power of this approach by its wide applicability

the numerical solution of weakly singular volterra integral equations by collocation on graded meshes since the solution of a second kind volterra integral equation with weakly singular kernel has in general unbounded derivatives at the left endpoint of the interval of integration its numerical solution by polynomial spline collocation on uniform meshes will lead to poor convergence rates in this paper we investigate the convergence rates with respect to graded meshes and we discuss the problem of how to select the quadrature formulas to obtain the fully discretized collocation equation

stability of difference volterra equations direct liapunov method and numerical procedure abstract a procedure to construct liapunov functionals for discrete volterra equations is proposed using this procedure stability conditions are derived for general volterra difference equations some applications of the proposed procedure for obtaining stability conditions for linear multistep methods for volterra integro differential equations are presented

generalization bounds for ranking algorithms via algorithmic stability the problem of ranking in which the goal is to learn a real valued ranking function that induces a ranking or ordering over an instance space has recently gained much attention in machine learning we study generalization properties of ranking algorithms using the notion of algorithmic stability in particular we derive generalization bounds for ranking algorithms that have good stability properties we show that kernel based ranking algorithms that perform regularization in a reproducing kernel hilbert space have such stability properties and therefore our bounds can be applied to these algorithms this is in contrast with generalization bounds based on uniform convergence which in many cases cannot be applied to these algorithms our results generalize earlier results that were derived in the special setting of bipartite ranking agarwal and niyogi 2005 to a more general setting of the ranking problem that arises frequently in applications

online pairwise learning algorithms with convex loss functions online pairwise learning algorithms with general convex loss functions without regularization in a reproducing kernel hilbert space rkhs are investigated under mild conditions on loss functions and the rkhs upper bounds for the expected excess generalization error are derived in terms of the approximation error when the stepsize sequence decays polynomially in particular for lipschitz loss functions such as the hinge loss the logistic loss and the absolute value loss the bounds can be of order o t13logt after t iterations while for the least squares loss the bounds can be of order o t14logt in comparison with previous works for these algorithms a broader family of convex loss functions is studied here and refined upper bounds are obtained

on ranking and generalization bounds the problem of ranking is to predict or to guess the ordering between objects on the basis of their observed features in this paper we consider ranking estimators that minimize the empirical convex risk we prove generalization bounds for the excess risk of such estimators with rates that are faster than 1 n we apply our results to commonly used ranking algorithms for instance boosting or support vector machines moreover we study the performance of considered estimators on real data sets

continuous collocation approximations to solutions of first kind volterra equations in this paper we give necessary and sufficient conditions for convergence of continuous collocation approximations of solutions of first kind volterra integral equations the results close some longstanding gaps in the theory of polynomial spline collocation methods for such equations the convergence analysis is based on a runge kutta or ode approach

online pairwise learning algorithms pairwise learning usually refers to a learning task that involves a loss function depending on pairs of examples among which the most notable ones are bipartite ranking metric learning and auc maximization in this letter we study an online algorithm for pairwise learning with a least square loss function in an unconstrained setting of a reproducing kernel hilbert space rkhs that we refer to as the online pairwise learning algorithm opera in contrast to existing works kar sriperumbudur jain karnick 2013 wang khardon pechyony jones 2012 which require that the iterates are restricted to a bounded domain or the loss function is strongly convex opera is associated with a non strongly convex objective function and learns the target function in an unconstrained rkhs specifically we establish a general theorem that guarantees the almost sure convergence for the last iterate of opera without any assumptions on the underlying distribution explicit convergence rates are derived under the condition of polynomially decaying step sizes we also establish an interesting property for a family of widely used kernels in the setting of pairwise learning and illustrate the convergence results using such kernels our methodology mainly depends on the characterization of rkhss using its associated integral operators and probability inequalities for random variables with values in a hilbert space

uniform error estimates of galerkin methods for monotone abel volterra integral equations on the half line we consider galerkin methods for monotone abel volterra integral equations of the second kind on the half line the l2 theory follows from kolodner s theory of monotone hammerstein equations we derive the loo theory from the l2 theory by relating the l2and loo spectra of operators of the form x b ax to one another here denotes convolution and b e l1 and a e loo as an extra condition we need b t o t a 1 with a 0 we also prove the discrete analogue in particular we verify that the galerkin matrix satisfies the discrete conditions

on the stability of volterra runge kutta methods we investigate the stability of extended runge kutta methods for volterra integral equations of the second kind as test equations we choose linear convolution equations with positive definite l 1 kernels our results are strongly connected with the contractivity of runge kutta methods for nonlinear dissipative differential equations

a perspective on the numerical treatment of volterra equations abstract we discuss the properties and numerical treatment of various types of volterra and abel volterra integral and integro differential equations

local rademacher complexities we propose new bounds on the error of learning algorithms in terms of a data dependent notion of complexity the estimates we establish give optimal rates and are based on a local and empirical version of rademacher averages in the sense that the rademacher averages are computed from the data on a subset of functions with small empirical error we present some applications to classification and prediction with convex function classes and with kernel classes in particular

ranking and empirical minimization of u statistics the problem of ranking ordering instances instead of simply classifying them has recently gained much attention in machine learning in this paper we formulate the ranking problem in a rigorous statistical framework the goal is to learn a ranking rule for deciding among two instances which one is better with minimum ranking risk since the natural estimates of the risk are of the form of a u statistic results of the theory of u processes are required for investigating the consistency of empirical risk minimizers we establish in particular a tail inequality for degenerate u processes and apply it for showing that fast rates of convergence may be achieved under specific noise assumptions just like in classification convex risk minimization methods are also studied

mixed finite element method and high order local artificial boundary conditions for exterior problems of elliptic equation the mixed finite element method is used to solve the exterior elliptic problem with high order local artificial boundary conditions new unknowns are introduced to reduce the order of the derivatives to two this leads to an equivalent mixed variational problem such that the normal finite element can be used and special finite elements are no longer needed on the adjacent layer of the artificial boundary error estimates are obtained for some local artificial boundary conditions with prescribed order numerical examples are presented and the results demonstrate the effectiveness of this method