estimation of gradients and coordinate covariation in classification we introduce an algorithm that simultaneously estimates a classification function as well as its gradient in the supervised learning framework the motivation for the algorithm is to find salient variables and estimate how they covary an efficient implementation with respect to both memory and time is given the utility of the algorithm is illustrated on simulated data as well as a gene expression data set an error analysis is given for the convergence of the estimate of the classification function and its gradient to the true classification function and true gradient

ramification of local fields with imperfect residue fields we define two decreasing filtrations by ramification groups on the absolute galois group of a complete discrete valuation field whose residue field may not be perfect in the classical case where the residue field is perfect we recover the classical upper numbering filtration the definition uses rigid geometry and log structures we also establish some of their properties

ruin probability with claims modeled by a stationary ergodic stable process for a random walk with negative drift we study the exceedance probability ruin probability of a high threshold the steps of this walk claim sizes constitute a stationary ergodic stable process we study how ruin occurs in this situation and evaluate the asymptotic behavior of the ruin probability for a large variety of stationary ergodic stable processes our findings show that the order of magnitude of the ruin probability varies significantly from one model to another in particular ruin becomes much more likely when the claim sizes exhibit long range dependence the proofs exploit large deviation techniques for sums of dependent stable random variables and the series representation of a stable process as a function of a poisson process

gradient iteration with lp norm constraints abstract gradient iterations with l p norm constrains have been studied recently both in the context of inverse problem and of compressed sensing use of l p norm constrains p 2 is often advocated when a sparse solution is expected a constrained gradient iteration is usually implemented via a shrinkage operator in this note we establish the l 2 norm convergence of such a constrained gradient iteration when p 1 2

maxima of branching random walks in recent years several authors have obtained limit theorems for l n the location of the rightmost particle in a supercritical branching random walk but all of these results have been proved under the assumption that the offspring distribution has exp x df x 0 in this paper we investigate what happens when there is a slowly varying function k so that 1 f x x q k x as x and log x f x 0 as x in this case we find that there is a sequence of constants a n which grow exponentially so that l n a n converges weakly to a nondegenerate distribution this result is in sharp contrast to the linear growth of l n observed in the case

null flows positive flows and the structure of stationary symmetric stable processes this paper elucidates the connection between stationary symmetric stable processes with 0 2 and nonsingular flows on measure spaces by describing a new and unique decomposition of stationary stable processes into those corresponding to positive flows and those corresponding to null flows we show that necessary and sufficient for a stationary stable process to be ergodic is that its positive component vanishes

precise large deviations for dependent regularly varying sequences we study a precise large deviation principle for a stationary regularly varying sequence of random variables this principle extends the classical results of nagaev theory probab appl 14 51 64 193 208 1969 and nagaev ann probab 7 745 789 1979 for iid regularly varying sequences the proof uses an idea of jakubowski stoch proc appl 44 291 327 1993 68 1 20 1997 in the context of central limit theorems with infinite variance stable limits we illustrate the principle for stochastic volatility models real valued functions of a markov chain satisfying a polynomial drift condition and solutions of linear and non linear stochastic recurrence equations

statistical analysis of bayes optimal subset ranking the ranking problem has become increasingly important in modern applications of statistical methods in automated decision making systems in particular we consider a formulation of the statistical ranking problem which we call subset ranking and focus on the discounted cumulated gain dcg criterion that measures the quality of items near the top of the rank list similar to error minimization for binary classification direct optimization of natural ranking criteria such as dcg leads to a nonconvex optimization problems that can be np hard therefore a computationally more tractable approach is needed we present bounds that relate the approximate optimization of dcg to the approximate minimization of certain regression errors these bounds justify the use of convex learning formulations for solving the subset ranking problem the resulting estimation methods are not conventional in that we focus on the estimation quality in the top portion of the rank list we further investigate the asymptotic statistical behavior of these formulations under appropriate conditions the consistency of the estimation schemes with respect to the dcg metric can be derived

large deviations for solutions to stochastic recurrence equations under kesten s condition in this paper we prove large deviations results for partial sums constructed from the solution to a stochastic recurrence equation we assume kesten s condition acta math 131 1973 207 248 under which the solution of the stochastic recurrence equation has a marginal distribution with power law tails while the noise sequence of the equations can have light tails the results of the paper are analogs to those obtained by a v nagaev theory probab appl 14 1969 51 64 193 208 and s v nagaev ann probab 7 1979 745 789 in the case of partial sums of i i d random variables in the latter case the large deviation probabilities of the partial sums are essentially determined by the largest step size of the partial sum for the solution to a stochastic recurrence equation the magnitude of the large deviation probabilities is again given by the tail of the maximum summand but the exact asymptotic tail behavior is also influenced by clusters of extreme values due to dependencies in the sequence we apply the large deviation results to study the asymptotic behavior of the ruin probabilities in the model

functional central limit theorem for a class of negatively dependent heavy tailed stationary infinitely divisible processes generated by conservative flows we prove a functional central limit theorem for partial sums of symmetric stationary long range dependent heavy tailed infinitely divisible processes the limiting stable process is particularly interesting due to its long memory which is quantified by a mittag leffler process induced by an associated harris chain at the discrete time level previous results in owada and samorodnitsky ann probab 43 2015 240 285 dealt with positive dependence in the increment process whereas this paper derives the functional limit theorems under negative dependence the negative dependence is due to cancellations arising from gaussian type fluctuations of functionals of the associated harris chain the new types of limiting processes involve stable random measures due to heavy tails mittag leffler processes due to long memory and brownian motions due to the gaussian second order cancellations along the way we prove a function central limit theorem for fluctuations of functionals of harris chains which is of independent interest as it extends a result of chen probab theory related fields 116 2000 89 123

on localizations of the characteristic classes of l adic sheaves and conductor formula in characteristic p 0 abbes kato and saito generalize the grothendieck ogg shafarevich formula to an arbitrary dimension kato and saito in ann math 168 33 96 2008 abbes and saito in invent math 168 567 612 2007 in this paper assuming the strong resolution of singularities we prove a localized version of a formula proved using the characteristic class of an l adic sheaf by abbes and saito invent math 168 567 612 2007 we prove a localized version of the lefschetz verdier trace formula proved in grothendieck formule de lefschetz expose iii sga 5 lect notes math vol 589 pp 372 406 exp x springer berlin 1977 theoreme 4 4 as an application we prove a conductor formula in an arbitrary dimension in the equal characteristic case

learning gradients by a gradient descent algorithm we propose a stochastic gradient descent algorithm for learning the gradient of a regression function from random samples of function values this is a learning algorithm involving mercer kernels by a detailed analysis in reproducing kernel hilbert spaces we provide some error bounds to show that the gradient estimated by the algorithm converges to the true gradient under some natural conditions on the regression function and suitable choices of the step size and regularization parameters

on convergence toward an extreme value distribution in c 0 1 the structure of extreme value distributions in infinite dimensional space is well known we characterize the domain of attraction of such extreme value distributions in the framework of gine hahn and vatan we intend to use the result for statistical applications

the limit distribution of the maximum increment of a random walk with dependent regularly varying jump sizes we investigate the maximum increment of a random walk with heavy tailed jump size distribution here heavy tailedness is understood as regular variation of the finite dimensional distributions the jump sizes constitute a strictly stationary sequence using a continuous mapping argument acting on the point processes of the normalized jump sizes we prove that the maximum increment of the random walk converges in distribution to a frechet distributed random variable

extreme value theory ergodic theory and the boundary between short memory and long memory for stationary stable processes we study the partial maxima of stationary stable processes we relate their asymptotic behavior to the ergodic theoretical properties of the flow we observe a sharp change in the asymptotic behavior of the sequence of partial maxima as flow changes from being dissipative to being conservative and argue that this may indicate a change from a short memory process to a long memory process

large deviations for random walks under subexponentiality the big jump domain for a given one dimensional random walk sn with a subexponential step size distribution we present a unifying theory to study the sequences xn for which as n uniformly for x xn we also investigate the stronger local analogue our theory is self contained and fits well within classical results on domains of partial attraction and local limit theory when specialized to the most important subclasses of subexponential distributions that have been studied in the literature we reproduce known theorems and we supplement them with new results

extremal behavior of stochastic integrals driven by regularly varying levy processes we study the extremal behavior of a stochastic integral driven by a multivariate levy process that is regularly varying with index alpha 0 for predictable integrands with a finite alpha delta moment for some delta 0 we show that the extremal behavior of the stochastic integral is due to one big jump of the driving levy process and we determine its limit measure associated with regular variation on the space of cadlag functions

large deviations and ruin probabilities for solutions to stochastic recurrence equations with heavy tailed innovations in this paper we consider the stochastic recurrence equation y t a t y t 1 b t for an i i d sequence of pairs a t b t of nonnegative random variables where we assume that b t is regularly varying with index k 0 and ea k t x c 1 np y 1 x as x holds for some constant c 1 0 for k 1 we also study the large deviation probabilities p s n es n x x x n for some sequence x n whose growth depends on the heaviness of the tail of the distribution of y 1 we show that the relation p s n es n x c 2 np y 1 x holds uniformly for x x n and some constant c 2 0 then we apply the large deviation results to derive bounds for the ruin probability u p sup n 1 s n es n n u for any 0 we show that u c 3 up y 1 u 1 1 1 for some constant c 3 0 in contrast to the case of i i d regularly varying y t s when the above results hold with c 1 c 2 c 3 1 the constants c 1 c 2 and c 3 are different from 1

generalization performance of regularized ranking with multiscale kernels the regularized kernel method for the ranking problem has attracted increasing attentions in machine learning the previous regularized ranking algorithms are usually based on reproducing kernel hilbert spaces with a single kernel in this paper we go beyond this framework by investigating the generalization performance of the regularized ranking with multiscale kernels a novel ranking algorithm with multiscale kernels is proposed and its representer theorem is proved we establish the upper bound of the generalization error in terms of the complexity of hypothesis spaces it shows that the multiscale ranking algorithm can achieve satisfactory learning rates under mild conditions experiments demonstrate the effectiveness of the proposed method for drug discovery and recommendation tasks

margin based ranking and an equivalence between adaboost and rankboost we study boosting algorithms for learning to rank we give a general margin based bound for ranking based on covering numbers for the hypothesis space our bound suggests that algorithms that maximize the ranking margin will generalize well we then describe a new algorithm smooth margin ranking that precisely converges to a maximum ranking margin solution the algorithm is a modification of rankboost analogous to approximate coordinate ascent boosting finally we prove that adaboost and rankboost are equally good for the problems of bipartite ranking and classification in terms of their asymptotic behavior on the training set under natural conditions adaboost achieves an area under the roc curve that is equally as good as rankboost s furthermore rankboost when given a specific intercept achieves a misclassification error that is as good as adaboost s this may help to explain the empirical observations made by cortes and mohri and caruana and niculescu mizil about the excellent performance of adaboost as a bipartite ranking algorithm as measured by the area under the roc curve

on large delays in multi server queues with heavy tails we present upper and lower bounds for the tail distribution of the stationary waiting time d in the stable gi gi s first come first served fcfs queue these bounds depend on the value of the traffic load which is the ratio of mean service and mean interarrival times for service times with intermediate regularly varying tail distribution the bounds are exact up to a constant and we are able to establish a principle of s k big jumps in this case here k is the integer part of which gives the most probable way for the stationary waiting time to be large another corollary of the bounds obtained is to provide a new proof of necessity and sufficiency of conditions for the existence of moments of the stationary waiting time

a self similar process with nowhere bounded sample paths on considere un processus auto similaire a parametre h et on montre la non existence de trajectoires bornees pour certaines valeurs de et

refined characteristic class and conductor formula in this article we prove a conductor formula in a geometric situation which is an analogue of the grothendieck ogg shafarevich formula our approach uses the ramification theory of abbes and saito and relies on tsushima s refined characteristic class

stability and generalization we define notions of stability for learning algorithms and show how to use these notions to derive generalization error bounds based on the empirical error and the leave one out error the methods we use can be applied in the regression framework as well as in the classification one when the classifier is obtained by thresholding a real valued function we study the stability properties of large classes of learning algorithms such as regularization based algorithms in particular we focus on hilbert space regularization and kullback leibler regularization we demonstrate how to apply the results to svm for regression and classification

a transitivity analysis of bipartite rankings in pairwise multi class classification many multi class classification algorithms in statistics and machine learning typically combine several binary classifiers in order to construct an overall classifier in the popular pairwise ensemble one classifier is built for each pair of classes resulting in pairwise bipartite rankings in contrast ordinal regression algorithms consider a single ranking function for several ordered classes it is known in the literature that pairwise ensembles can be useful for ordinal regression however can single ranking models make a contribution to multi class classification the answer to this question should be affirmative as supported by theoretical results presented in this article we conduct a formal analysis of the consistency of pairwise bipartite rankings by uncovering the conditions under which they can be equivalently expressed in terms of a single ranking similar to the utility representability of pairwise preference relations it turns out that transitivity plays a crucial role in the characterization of the ranking representability of pairwise bipartite rankings to this end we introduce the new concepts of strict ranking representability a restrictive condition that can be verified easily and auc ranking representability a practically more useful condition that is more difficult to verify however the link between pairwise bipartite rankings and dice games allows us to formulate necessary transitivity conditions for auc ranking representability a sufficient condition on the other hand is obtained by introducing a new type of transitivity that can be verified by solving an integer quadratic program

generalization bounds for ranking algorithms via algorithmic stability the problem of ranking in which the goal is to learn a real valued ranking function that induces a ranking or ordering over an instance space has recently gained much attention in machine learning we study generalization properties of ranking algorithms using the notion of algorithmic stability in particular we derive generalization bounds for ranking algorithms that have good stability properties we show that kernel based ranking algorithms that perform regularization in a reproducing kernel hilbert space have such stability properties and therefore our bounds can be applied to these algorithms this is in contrast with generalization bounds based on uniform convergence which in many cases cannot be applied to these algorithms our results generalize earlier results that were derived in the special setting of bipartite ranking agarwal and niyogi 2005 to a more general setting of the ranking problem that arises frequently in applications

generalization performance of bipartite ranking algorithms with convex losses abstract previous works describing the generalization performance of bipartite ranking algorithms are usually based on the assumption of 0 1 loss or the area under the receiver operating characteristic roc curve in this paper we go far beyond this classical framework by investigating the generalization performance of bipartite ranking algorithms with convex losses over reproducing kernel hilbert spaces based on the mcdiarmid inequality and rademacher complexity we establish the upper bound on the generalization error for a bipartite ranking algorithm the theoretical analysis is different from the previous results on error analysis and shows the attractive uniform convergence property of regularized bipartite ranking algorithms

the p norm push a simple convex ranking algorithm that concentrates at the top of the list we are interested in supervised ranking algorithms that perform especially well near the top of the ranked list and are only required to perform sufficiently well on the rest of the list in this work we provide a general form of convex objective that gives high scoring examples more importance this push near the top of the list can be chosen arbitrarily large or small based on the preference of the user we choose lp norms to provide a specific type of push if the user sets p larger the objective concentrates harder on the top of the list we derive a generalization bound based on the p norm objective working around the natural asymmetry of the problem we then derive a boosting style algorithm for the problem of ranking with a push at the top the usefulness of the algorithm is illustrated through experiments on repository data we prove that the minimizer of the algorithm s objective is unique in a specific sense furthermore we illustrate how our objective is related to quality measurements for information retrieval

stable mixed moving averages the class of non gaussian stable moving average processes is extended by introducing an appropriate joint randomization of the filter function and of the stable noise leading to stable mixed moving averages their distribution determines a certain combination of the filter function and the mixing measure leading to a generalization of a theorem of kanter 1973 for usual moving averages stable mixed moving averages contain sums of independent stable moving averages are ergodic and are not harmonizable also a class of stable mixed moving averages is constructed with the reflection positivity property

generalization bounds for the area under the roc curve we study generalization properties of the area under the roc curve auc a quantity that has been advocated as an evaluation criterion for the bipartite ranking problem the auc is a different term than the error rate used for evaluation in classification problems consequently existing generalization bounds for the classification error rate cannot be used to draw conclusions about the auc in this paper we define the expected accuracy of a ranking function analogous to the expected error rate of a classification function and derive distribution free probabilistic bounds on the deviation of the empirical auc of a ranking function observed on a finite data sequence from its expected accuracy we derive both a large deviation bound which serves to bound the expected accuracy of a ranking function in terms of its empirical auc on a test sequence and a uniform convergence bound which serves to bound the expected accuracy of a learned ranking function in terms of its empirical auc on a training sequence our uniform convergence bound is expressed in terms of a new set of combinatorial parameters that we term the bipartite rank shatter coefficients these play the same role in our result as do the standard vc dimension related shatter coefficients also known as the growth function in uniform convergence results for the classification error rate a comparison of our result with a recent uniform convergence result derived by freund et al 2003 for a quantity closely related to the auc shows that the bound provided by our result can be considerably tighter

the characteristic class and ramification of an l adic etale sheaf we introduce the characteristic class of an l adic etale sheaf using a cohomological pairing due to verdier sga5 as a consequence of the lefschetz verdier trace formula its trace computes the euler poincare characteristic of the sheaf we compare the characteristic class to two other invariants arising from ramification theory one is the swan class of kato saito 17 and the other is the 0 cycle class defined by kato for rank 1 sheaves in 16

on the era ranking representability of pairwise bipartite ranking functions in domains like decision theory and social choice theory it is known for a long time that stochastic transitivity properties yield necessary and sufficient conditions for the ranking or utility representability of reciprocal preference relations in this article we extend these results for reciprocal preference relations originating from the pairwise comparison of random vectors in a machine learning context more specifically the expected ranking accuracy era is such a reciprocal relation that occurs in multi class classification problems when ranking or utility functions are fitted to the data in a pairwise manner we establish necessary and sufficient conditions for which these pairwise bipartite ranking functions can be simplified to a single ranking function such that the pairwise expected ranking accuracies of both models coincide similarly as for more common reciprocal preference relations cycle transitivity plays a crucial role in this new setting we first consider the finite sample case for which expected ranking accuracy can be estimated by means of the area under the roc curve auc and subsequently we further generalize these results to the underlying distributions it turns out that the ranking representability of pairwisely compared random vectors can be expressed elegantly in a distribution independent way by means of a specific type of cycle transitivity defined by a conjunctor that is closely related to the algebraic product

a large deviations approach to limit theory for heavy tailed time series in this paper we propagate a large deviations approach for proving limit theory for generally multivariate time series with heavy tails we make this notion precise by introducing regularly varying time series we provide general large deviation results for functionals acting on a sample path and vanishing in some neighborhood of the origin we study a variety of such functionals including large deviations of random walks their suprema the ruin functional and further derive weak limit theory for maxima point processes cluster functionals and the tail empirical process one of the main results of this paper concerns bounds for the ruin probability in various heavy tailed models including garch stochastic volatility models and solutions to stochastic recurrence equations

the maximum of a branching random walk with semiexponential increments we consider an infinite galton watson tree and label the vertices with a collection of i i d random variables y in the case where the upper tail of the distribution of y is semiexponential we then determine the speed of the corresponding tree indexed random walk in contrast to the classical case where the random variables y have finite exponential moments the normalization in the definition of the speed depends on the distribution of y interpreting the random variables y as displacements of the offspring from the parent y describes a branching random walk the result on the speed gives a limit theorem for the maximum of the branching random walk that is for the position of the rightmost particle in our case this maximum grows faster than linear in time

ramification theory for varieties over a perfect field for an l adic sheaf on a variety of arbitrary dimension over a perfect field we define the swan class measuring the wild ramification as a 0 cycle class supported on the ramification locus we prove a lefschetz trace formula for open varieties and a generalization of the grothendieck ogg shararevich formula using the swan class

on the structure of stationary stable processes a connection between structural studies of stationary non gaussian stable processes and the ergodic theory of nonsingular flows is established and exploited using this connection a unique decomposition of a stationary stable process into three independent stationary parts is obtained it is shown that the dissipative part of a flow generates a mixed moving average part of a stationary stable process while the identity part of a flow essentially gives the harmonizable part the third part of a stationary process is determined by a conservative flow without fixed points and by a related cocycle

ranking and empirical minimization of u statistics the problem of ranking ordering instances instead of simply classifying them has recently gained much attention in machine learning in this paper we formulate the ranking problem in a rigorous statistical framework the goal is to learn a ranking rule for deciding among two instances which one is better with minimum ranking risk since the natural estimates of the risk are of the form of a u statistic results of the theory of u processes are required for investigating the consistency of empirical risk minimizers we establish in particular a tail inequality for degenerate u processes and apply it for showing that fast rates of convergence may be achieved under specific noise assumptions just like in classification convex risk minimization methods are also studied

generalization performance of least square regularized regression algorithm with markov chain samples abstract the previously known works describing the generalization of least square regularized regression algorithm are usually based on the assumption of independent and identically distributed i i d samples in this paper we go far beyond this classical framework by studying the generalization of least square regularized regression algorithm with markov chain samples we first establish a novel concentration inequality for uniformly ergodic markov chains then we establish the bounds on the generalization of least square regularized regression algorithm with uniformly ergodic markov chain samples and show that least square regularized regression algorithm with uniformly ergodic markov chains is consistent

deterministic impulse control problems we prove that the optimal cost function of a deterministic impulse control problem is the unique viscosity solution of a first order hamilton jacobi quasi variational inequality in mathbb r n