prediction and classification in nonlinear data analysis something old something new something borrowed something blue prediction and classification are two very active areas in modern data analysis in this paper prediction with nonlinear optimal scaling transformations of the variables is reviewed and extended to the use of multiple additive components much in the spirit of statistical learning techniques that are currently popular among other areas in data mining also a classification clustering method is described that is particularly suitable for analyzing attribute value data from systems biology genomics proteomics and metabolomics and which is able to detect groups of objects that have similar values on small subsets of the attributes

approximately optimizing ndcg using pair wise loss abstract the normalized discounted cumulative gain ndcg is used to measure the performance of ranking algorithms much of the work on learning to rank by optimizing ndcg directly or indirectly is based on list wise approaches in our work we approximately optimize a variant of ndcg called ndcg using pair wise approaches ndcg utilizes the linear discounting function we first prove that the dcg error of ndcg is equal to the weighted pair wise loss then on that basis rankboostndcg and ranksvmndcg are proposed to optimize the upper bound of the pair wise 0 1 loss function the experimental results from applying our approaches and ten other state of the art methods to five public datasets show the superiority of the proposed methods especially ranksvmndcg in addition rankboostndcg are less influenced by the initial weight distribution

fekete points and convergence towards equilibrium measures on complex manifolds building on bb1 we prove a general criterion for convergence of possibly singular bergman measures towards pluripotential theoretic equilibrium measures on complex manifolds the criterion may be formulated in terms of the growth properties of the unit balls of certain norms on holomorphic sections or equivalently as an asymptotic minimization property for generalized donaldson l functionals our result settles in particular a well known conjecture in pluripotential theory concerning the equidistribution of fekete points and it gives the convergence of bergman measures towards the equilibrium measure for bernstein markov measures applications to interpolation of holomorphic sections are also discussed

multitask quantile regression under the transnormal model abstractwe consider estimating multitask quantile regression under the transnormal model with focus on high dimensional setting we derive a surprisingly simple closed form solution through rank based covariance regularization in particular we propose the rank based l1 penalization with positive definite constraints for estimating sparse covariance matrices and the rank based banded cholesky decomposition regularization for estimating banded precision matrices by taking advantage of the alternating direction method of multipliers nearest correlation matrix projection is introduced that inherits sampling properties of the unprojected one our work combines strengths of quantile regression and rank based covariance regularization to simultaneously deal with nonlinearity and nonnormality for high dimensional regression furthermore the proposed method strikes a good balance between robustness and efficiency achieves the oracle like convergence rate and provides the provable prediction interval und

power and sample size for observational studies of point exposure effects inverse probability of treatment weights iptws are commonly used to control for confounding when estimating causal effects of point exposures from observational data when planning a study that will be analyzed with iptws determining the required sample size for a given level of statistical power is challenging because of the effect of weighting on the variance of the estimated causal means this paper considers the utility of the design effect to quantify the effect of weighting on the precision of causal estimates the design effect is defined as the ratio of the variance of the causal mean estimator divided by the variance of a naive estimator if counter to fact no confounding had been present and weights were not needed a simple closed form approximation of the design effect is derived that is outcome invariant and can be estimated during the study design phase once the design effect is approximated for each treatment group sample size calculations are conducted as for a randomized trial but with variances inflated by the design effects to account for weighting simulations demonstrate the accuracy of the design effect approximation and practical considerations are discussed

local existence of the borel transform in euclidean φ 4 4 we bound rigorously the large order behaviour of 44 euclidean perturbative quantum field theory as the simplest example of renormalizable but non super renormalizable theory the needed methods are developed to take into account the structure of renormalization which plays a crucial role in the estimates as a main thorem it is shown that the schwinger functions at ordern are bounded byknn which implies a finite radius of convergence for the borel transform of the perturbation series

recursive construction of operator product expansion coefficients we derive a novel formula for the derivative of operator product expansion ope coefficients with respect to a coupling constant the formula involves just the ope coefficients themselves but no further input and is in this sense self consistent furthermore unlike other formal identities of this general nature in quantum field theory such as the formal expression for the lagrangian perturbation of a correlation function our formula requires no further uv renormalization i e it is completely well defined from the start this feature is a result of a cancelation of uv and ir divergences between various terms in our identity our proof and an analysis of the features of the identity is given for the example of massive euclidean varphi 4 theory in 4 dimensional euclidean space it relies on the renormalization group flow equation method and is valid to arbitrary but finite orders in perturbation theory the final formula however makes neither explicit reference to the renormalization group flow nor to perturbation theory and we conjecture that it also holds non perturbatively our identity can be applied constructively because it gives a novel recursive algorithm for the computation of ope coefficients to arbitrary finite perturbation order in terms of the zeroth order coefficients corresponding to the underlying free field theory which in turn are trivial to obtain we briefly illustrate the relation of this method to more standard methods for computing the ope in some simple examples

on the characterization of a class of fisher consistent loss functions and its application to boosting accurate classification of categorical outcomes is essential in a wide range of applications due to computational issues with minimizing the empirical 0 1 loss fisher consistent losses have been proposed as viable proxies however even with smooth losses direct minimization remains a daunting task to approximate such a minimizer various boosting algorithms have been suggested for example with exponential loss the adaboost algorithm freund and schapire 1995 is widely used for two class problems and has been extended to the multi class setting zhu et al 2009 alternative loss functions such as the logistic and the hinge losses and their corresponding boosting algorithms have also been proposed zou et al 2008 wang 2012 in this paper we demonstrate that a broad class of losses including non convex functions achieve fisher consistency and in addition can be used for explicit estimation of the conditional class probabilities furthermore we provide a generic boosting algorithm that is not loss specific extensive simulation results suggest that the proposed boosting algorithms could outperform existing methods with properly chosen losses and bags of weak learners

on the large order behavior of φ44 we continue the rigorous study of the large order behavior of the perturbation series for the 4 model in 4 dimensions started in 1 in this paper we prove a result announced in 1 we show that the exact radius of convergence of the borel transform of the renormalized perturbation series for 44 is greater than or equal to the expected value given by the position of the first renormalon 2 this result holds for any vector 2 2 model withn components and makes use of the lipatov bound of 1 this result is based on a partial resummation of counterterms similar to the one of 3 but in a phase space analysis of the renormalized series

identifying risk factors for severe childhood malnutrition by boosting additive quantile regression we investigated the risk factors for childhood malnutrition in india based on the 2005 2006 demographic and health survey by applying a novel estimation technique for additive quantile regression ordinary linear and generalized linear regression models relate the mean of a response variable to a linear combination of covariate effects and as a consequence focus on average properties of the response the use of such a regression model for analyzing childhood malnutrition in developing or transition countries implies that the estimated effects describe the average nutritional status however it is of even greater interest to analyze quantiles of the response distribution such as the 5 or 10 quantile which relate to the risk of extreme malnutrition our investigation is based on a semiparametric extension of quantile regression models where different types of nonlinear effects are included in the model equation leading to additive quantile regression we addressed the variable selection and model c

covariate association eliminating weights a unified weighting framework for causal effect estimation weighting methods offer an approach to estimating causal treatment effects in observational studies however if weights are estimated by maximum likelihood misspecification of the treatment assignment model can lead to weighted estimators with substantial bias and variance in this paper we propose a unified framework for constructing weights such that a set of measured pretreatment covariates is unassociated with treatment assignment after weighting we derive conditions for weight estimation by eliminating the associations between these covariates and treatment assignment characterized in a chosen treatment assignment model after weighting the moment conditions in covariate balancing weight methods for binary categorical and continuous treatments in cross sectional settings are special cases of the conditions in our framework which extends to longitudinal settings simulation shows that our method gives treatment effect estimates with smaller biases and variances than the maximum likelihood approach under treatment assignment model misspecification we illustrate our method with an application to systemic lupus erythematosus data

causal inference by using invariant prediction identification and confidence intervals summary what is the difference between a prediction that is made with a causal model and that with a non causal model suppose that we intervene on the predictor variables or change the whole environment the predictions from a causal model will in general work as well under interventions as for observational data in contrast predictions from a non causal model can potentially be very wrong if we actively intervene on variables here we propose to exploit this invariance of a prediction under a causal model for causal inference given different experimental settings e g various interventions we collect all models that do show invariance in their predictive accuracy across settings and interventions the causal model will be a member of this set of models with high probability this approach yields valid confidence intervals for the causal relationships in quite general scenarios we examine the example of structural equation models in more detail and provide sufficient assumptions under which the set of causal predictors becomes identifiable we further investigate robustness properties of our approach under model misspecification and discuss possible extensions the empirical properties are studied for various data sets including large scale gene perturbation experiments

multivariate adaptive regression splines a new method is presented for flexible regression modeling of high dimensional data the model takes the form of an expansion in product spline basis functions where the number of basis functions as well as the parameters associated with each one product degree and knot locations are automatically determined by the data this procedure is motivated by the recursive partitioning approach to regression and shares its attractive properties unlike recursive partitioning however this method produces continuous models with continuous derivatives it has more power and flexibility to model relationships that are nearly additive or involve interactions in at most a few variables in addition the model can be represented in a form that separately identifies the additive contributions and those associated with the different multivariable interactions

a view of margin losses as regularizers of probability estimates regularization is commonly used in classifier design to assure good generalization classical regularization enforces a cost on classifier complexity by constraining parameters this is usually combined with a margin loss which favors large margin decision rules a novel and unified view of this architecture is proposed by showing that margin losses act as regularizers of posterior class probabilities in a way that amplifies classical parameter regularization the problem of controlling the regularization strength of a margin loss is considered using a decomposition of the loss in terms of a link and a binding function the link function is shown to be responsible for the regularization strength of the loss while the binding function determines its outlier robustness a large class of losses is then categorized into equivalence classes of identical regularization strength or outlier robustness it is shown that losses in the same regularization class can be parameterized so as to have tunable regularization strength this parameterization is finally used to derive boosting algorithms with loss regularization boostlr three classes of tunable regularization losses are considered in detail canonical losses can implement all regularization behaviors but have no flexibility in terms of outlier modeling shrinkage losses support equally parameterized link and binding functions leading to boosting algorithms that implement the popular shrinkage procedure this offers a new explanation for shrinkage as a special case of loss based regularization finally tunable losses enable the independent parameterization of link and binding functions leading to boosting algorithms of great exibility this is illustrated by the derivation of an algorithm that generalizes both adaboost and logitboost behaving as either one when that best suits the data to classify various experiments provide evidence of the benefits of probability regularization for both classification and estimation of posterior class probabilities

a cardinal function algorithm for computing multivariate quadrature points we present a new algorithm for numerically computing quadrature formulas for arbitrary domains which exactly integrate a given polynomial space an effective method for constructing quadrature formulas has been to numerically solve a nonlinear set of equations for the quadrature points and their associated weights symmetry conditions are often used to reduce the number of equations and unknowns our algorithm instead relies on the construction of cardinal functions and thus requires that the number of quadrature points n be equal to the dimension of a prescribed lower dimensional polynomial space the cardinal functions allow us to treat the quadrature weights as dependent variables and remove them as well as an equivalent number of equations from the numerical optimization procedure we give results for the triangle where for all degrees d le 25 we find quadrature formulas of this form which have positive weights and contain no points outside the triangle seven of these quadrature formulas improve on previously known results

an algorithm for computing fekete points in the triangle on the line and its tensor products fekete points are known to be the gauss lobatto quadrature points but unlike high order quadrature fekete points generalize to non tensor product domains such as the triangle thus fekete points might serve as an alternative to the gauss lobatto points for certain applications in this work we present a new algorithm to compute fekete points and give results up to degree 19 for the triangle for degree d 10 these points have the smallest lebesgue constant currently known the computations validate a conjecture of bos j approx theory 64 1991 pp 271 280 that fekete points along the boundary of the triangle are the one dimensional gauss lobatto points

the operator product expansion converges in perturbative field theory we show within the framework of the massive euclidean varphi 4 quantum field theory in four dimensions that the wilson operator product expansion ope is not only an asymptotic expansion at short distances as previously believed but even converges at arbitrary finite distances our proof rests on a detailed estimation of the remainder term in the ope of an arbitrary product of composite fields inserted as usual into a correlation function with further spectator fields the estimates are obtained using a suitably adapted version of the method of renormalization group flow equations convergence follows because the remainder is seen to become arbitrarily small as the ope is carried out to sufficiently high order i e to operators of sufficiently high dimension our results hold for arbitrary but finite loop orders as an interesting side result of our estimates we can also prove that the gradient expansion of the effective action is convergent

boosting with the l2 loss this article investigates a computationally simple variant of boosting l2boost which is constructed from a functional gradient descent algorithm with the l2 loss function like other boosting algorithms l2boost uses many times in an iterative fashion a prechosen fitting method called the learner based on the explicit expression of refitting of residuals of l2boost the case with symmetric linear learners is studied in detail in both regression and classification in particular with the boosting iteration m working as the smoothing or regularization parameter a new exponential bias variance trade off is found with the variance complexity term increasing very slowly as m tends to infinity when the learner is a smoothing spline an optimal rate of convergence result holds for both regression and classification and the boosted smoothing spline even adapts to higher order unknown smoothness moreover a simple expansion of a smoothed 0 1 loss function is derived to reveal the importance of the d

statistical analysis of bayes optimal subset ranking the ranking problem has become increasingly important in modern applications of statistical methods in automated decision making systems in particular we consider a formulation of the statistical ranking problem which we call subset ranking and focus on the discounted cumulated gain dcg criterion that measures the quality of items near the top of the rank list similar to error minimization for binary classification direct optimization of natural ranking criteria such as dcg leads to a nonconvex optimization problems that can be np hard therefore a computationally more tractable approach is needed we present bounds that relate the approximate optimization of dcg to the approximate minimization of certain regression errors these bounds justify the use of convex learning formulations for solving the subset ranking problem the resulting estimation methods are not conventional in that we focus on the estimation quality in the top portion of the rank list we further investigate the asymptotic statistical behavior of these formulations under appropriate conditions the consistency of the estimation schemes with respect to the dcg metric can be derived

a numerical algorithm for the construction of efficient quadrature rules in two and higher dimensions we present a numerical algorithm for the construction of efficient high order quadratures in two and higher dimensions quadrature rules constructed via this algorithm possess positive weights and interior nodes resembling the gaussian quadratures in one dimension in addition rules can be generated with varying degrees of symmetry adaptable to individual domains we illustrate the performance of our method with numerical examples and report quadrature rules for polynomials on triangles squares and cubes up to degree 50 these formulae are near optimal in the number of nodes used and many of them appear to be new

contrast trees and distribution boosting a method for decision tree induction is presented given a set of predictor variables formula see text and two outcome variables y and z associated with each x the goal is to identify those values of x for which the respective distributions of formula see text and formula see text or selected properties of those distributions such as means or quantiles are most different contrast trees provide a lack of fit measure for statistical models of such statistics or for the complete conditional distribution formula see text as a function of x they are easily interpreted and can be used as diagnostic tools to reveal and then understand the inaccuracies of models produced by any learning method a corresponding contrast boosting strategy is described for remedying any uncovered errors thereby producing potentially more accurate predictions this leads to a distribution boosting strategy for directly estimating the full conditional distribution of y at each x under no assumptions concerning its shape form or parametric representation

on the local borel transform of perturbation theory we prove existence of the local borel transform for the perturbative series of massive varphi 4 4 theory as compared to previous proofs in the literature the present bounds are much sharper as regards the dependence on external momenta they are explicit in the number of external legs and they are obtained quite simply through a judiciously chosen induction hypothesis applied to the wegner wilson polchinski flow equations we pay attention not to generate an astronomically large numerical constant for the inverse radius of convergence of the borel transform

algebraic cubature on polygonal elements with a circular edge abstract we compute low cardinality algebraic cubature formulas on convex or concave polygonal elements with a circular edge by subdivision into circular quadrangles blending formulas via subperiodic trigonometric gaussian quadrature and final compression via caratheodory tchakaloff subsampling of discrete measures we also discuss applications to the vem virtual element method in computational mechanics problems

the lipatov argument for φ 3 4 perturbation theory we extend to 3 4 the work of s breen on the leading behavior at large order of 2 4 perturbation theory using a phase space expansion to obtain new estimates on the high energy behavior of 3 4 feynman graphs and a rigorous semiclassical expansion we prove that the radius of convergence of the borel transform of the pertubative series for 3 4 euclidean field theory is the one computed by the lipatov method

the role of the propensity score in estimating dose response functions summary estimation of average treatment effects in observational studies often requires adjustment for differences in pre treatment variables if the number of pre treatment variables is large standard covariance adjustment methods are often inadequate rosenbaum rubin 1983 propose an alternative method for adjusting for pre treatment variables for the binary treatment case based on the so called propensity score here an extension of the propensity score methodology is proposed that allows for estimation of average casual effects with multi valued treatments estimation of average treatment effects in observational studies often requires adjustment for differences in pre treatment variables if the number of pre treatment variables is large and their distribution varies substantially with treatment status standard adjustment methods such as covari ance adjustment are often inadequate rosenbaum rubin 1983 1984 propose an alternative method for adjusting for pre treatment variables based on the propensity score the conditional probability of receiving the treatment given pre treatment variables they demonstrate that adjusting solely for the propensity score removes all bias associated with differences in the pre treatment variables the rosenbaum rubin proposals deal exclusively with binary valued treat ments in many cases of interest however treatments take on more than two values here an extension of the propensity score methodology is proposed that allows for estimation of average causal effects with multi valued treatments the key insight is that for estimation of average causal effects it is not necessary to divide the population into subpopulations where causal comparisons are valid as the propensity score does it is sufficient to divide the population into subpopulations where average potential outcomes can be estimated

renormalization proof for massive phi_4 4 theory on riemannian manifolds in this paper we present an inductive renormalizability proof for massive phi 4 4 theory on riemannian manifolds based on the wegner wilson flow equations of the wilson renormalization group adapted to perturbation theory the proof goes in hand with bounds on the perturbative schwinger functions which imply tree decay between their position arguments an essential prerequisite is precise bounds on the short and long distance behaviour of the heat kernel on the manifold with the aid of a regularity assumption often taken for granted we also show that for suitable renormalization conditions the bare action takes the minimal form that is to say there appear the same counterterms as in flat space apart from a logarithmically divergent one which is proportional to the scalar curvature

covariance adjustment on propensity parameters for continuous treatment in linear models propensity scores are widely used to control for confounding when estimating the effect of a binary treatment in observational studies they have been generalized to ordinal and continuous treatments in the recent literature following the definition of propensity function and its parameterizations called the propensity parameter in this paper proposed by imai and van dyk we explore sufficient conditions for selecting propensity parameters to control for confounding for continuous treatments in the context of regression based adjustment in linear models typically investigators make parametric assumptions about the form of the dose response function for a continuous treatment such assumptions often allow the analyst to use only a subset of the propensity parameters to control confounding when the treatment is the only predictor in the structural that is causal model it is sufficient to adjust only for the propensity parameters that characterize the expectation of the treatment variable or its functional form when the structural model includes selected baseline covariates other than the treatment variable those baseline covariates in addition to the propensity parameters must also be adjusted in the model we demonstrate these points with an example estimating the dose response relationship for the effect of erythropoietin on hematocrit level in patients with end stage renal disease

the rate of convergence of adaboost the adaboost algorithm was designed to combine many weak hypotheses that perform slightly better than random guessing into a strong hypothesis that has very low error we study the rate at which adaboost iteratively converges to the minimum of the exponential loss unlike previous work our proofs do not require a weak learning assumption nor do they require that minimizers of the exponential loss are finite our first result shows that the exponential loss of adaboost s computed parameter vector will be at most e more than that of any parameter vector of l1 norm bounded by b in a number of rounds that is at most a polynomial in b and 1 e we also provide lower bounds showing that a polynomial dependence is necessary our second result is that within c e iterations adaboost achieves a value of the exponential loss that is at most e more than the best possible value where c depends on the data set we show that this dependence of the rate on e is optimal up to constant factors that is at least 1 e rounds are necessary to achieve within e of the optimal exponential loss

training hard margin support vector machines using greedy stagewise algorithm hard margin support vector machines hm svms suffer from getting overfitting in the presence of noise soft margin svms deal with this problem by introducing a regularization term and obtain a state of the art performance however this disposal leads to a relatively high computational cost in this paper an alternative method greedy stagewise algorithm for svms named gs svms is presented to cope with the overfitting of hm svms without employing the regularization term the most attractive property of gs svms is that its computational complexity in the worst case only scales quadratically with the size of training samples experiments on the large data sets with up to 400 000 training samples demonstrate that gs svms can be faster than libsvm 2 83 without sacrificing the accuracy finally we employ statistical learning theory to analyze the empirical results which shows that the success of gs svms lies in that its early stopping rule can act as an implicit regularization term

construction and borel summability of planar 4 dimensional euclidean field theory we use the methods of 1 to show that the planar part of the renormalized perturbation theory for 4 4 euclidean field theory is borel summable on the asymptotically free side of the theory the borel sum can therefore be taken as a rigorous definition of then limit of a massiven n matrix model with a trg 4 interaction hence with wrong sign ofg our construction is relevant for a solution of the ultra violet problem for planar qcd we also propose a program for studying the structure of the renormalons singularities within the planar world

the adaboost flow we introduce a dynamical system that we call the adaboost flow the flow is defined by a system of odes with control we show that three algorithms of the adaboost family i the adaboost algorithm of schapire and freund ii the arc gv algorithm of breiman and iii the confidence rated prediction of schapire and singer can be embedded in the adaboost flow the nontrivial part of the adaboost flow equations coincides with the equations of dynamics of the nonperiodic toda system written in terms of spectral variables we provide a novel invariant geometrical description of the adaboost algorithm as a gradient flow on a foliation defined by level sets of the potential function we propose a new approach for constructing boosting algorithms as a continuous time gradient flow on measures defined by various metrics and potential functions finally we explain the similarity of the adaboost algorithm with the perelman s construction for the ricci flow 2015 wiley periodicals inc

the central role of the propensity score in observational studies for causal effects abstract the results of observational studies are often disputed because of nonrandom treatment assignment for example patients at greater risk may be overrepresented in some treatment group this paper discusses the central role of propensity scores and balancing scores in the analysis of observational studies the propensity score is the estimated conditional probability of assignment to a particular treatment given a vector of observed covariates both large and small sample theory show that adjustment for the scalar propensity score is sufficient to remove bias due to all observed covariates applications include matched sampling on the univariate propensity score which is equal percent bias reducing under more general conditions than required for discriminant matching multivariate adjustment by subclassification on balancing scores where the same subclasses are used to estimate treatment effects for all outcome variables and in all subpopulations and visual representation of multivariate adjustment by a two dimensional plot author

estimation of conditional and marginal odds ratios using the prognostic score introduced by hansen in 2008 the prognostic score pgs has been presented as the prognostic analogue of the propensity score pps pps based methods are intended to estimate marginal effects most previous studies evaluated the performance of existing pgs based methods adjustment stratification and matching using the pgs in situations in which the theoretical conditional and marginal effects are equal i e collapsible situations to support the use of pgs framework as an alternative to the pps framework applied researchers must have reliable information about the type of treatment effect estimated by each method we propose four new pgs based methods each developed to estimate a specific type of treatment effect we evaluated the ability of existing and new pgs based methods to estimate the conditional treatment effect cte the marginal average treatment effect on the whole population ate and the marginal average treatment effect on the treated population att when the odds ratio a non collapsible estimator is the measure of interest the performance of pgs based methods was assessed by monte carlo simulations and compared with pps based methods and multivariate regression analysis existing pgs based methods did not allow for estimating the ate and showed unacceptable performance when the proportion of exposed subjects was large when estimating marginal effects pps based methods were too conservative whereas the new pgs based methods performed better with low prevalence of exposure and had coverages closer to the nominal value when estimating cte the new pgs based methods performed as well as traditional multivariate regression copyright 2016 john wiley sons ltd

building sparse twin support vector machine classifiers in primal space abstract twin support vector machines tsvm obtain faster training speeds than classical support vector machines svm however tsvm augmented vectors lose sparsity in this paper a rapid sparse twin support vector machine stsvm classifier in primal space is proposed to improve the sparsity and robustness of tsvm based on a simple back fitting strategy the stsvm iteratively builds each nonparallel hyperplanes by adding one support vector sv from the corresponding class at one time this process is terminated using an adaptive and stable stopping criterion stsvm learning is implemented by linear equation computing systems through introducing a quadratic function to approximate the empirical risk the computational results on several synthetic and benchmark datasets indicate that the stsvm obtains a sparse separating hyperplane at a low cost without sacrificing its generalization performance

an l_ 2 boosting algorithm for estimation of a regression function an l 2 boosting algorithm for estimation of a regression function from random design is presented which consists of fitting repeatedly a function from a fixed nonlinear function space to the residuals of the data by least squares and by defining the estimate as a linear combination of the resulting least squares estimates splitting of the sample is used to decide after how many iterations of smoothing of the residuals the algorithm terminates the rate of convergence of the algorithm is analyzed in case of an unbounded response variable the method is used to fit a sum of maxima of minima of linear functions to a given data set and is compared with other nonparametric regression estimates using simulated data

generalized gaussian quadrature rules on arbitrary polygons in this paper we present a numerical algorithm based on group theory and numerical optimization to compute efficient quadrature rules for integration of bivariate polynomials over arbitrary polygons these quadratures have desirable properties such as positivity of weights and interiority of nodes and can readily be used as software libraries where numerical integration within planar polygons is required we have used this algorithm for the construction of symmetric and non symmetric quadrature rules over convex and concave polygons while in the case of symmetric quadratures our results are comparable to available rules the proposed algorithm has the advantage of being flexible enough so that it can be applied to arbitrary planar regions for the integration of generalized classes of functions to demonstrate the efficiency of the new quadrature rules we have tested them for the integration of rational polygonal shape functions over a regular hexagon for a relative error of 10 8 in the computation of stiffness matrix entries one needs at least 198 evaluation points when the region is partitioned whereas 85 points suffice with our quadrature rule copyright 2009 john wiley sons ltd

bounds on renormalized feynman graphs we prove two bounds on the value of renormalized euclidean feynman graphs one is a relatively crude but widely applicable bound the other a finer bound applicable to 4 4 like models

additive logistic regression a statistical view of boosting boosting is one of the most important recent developments in classification methodology boosting works by sequentially applying a classification algorithm to reweighted versions of the training data and then taking a weighted majority vote of the sequence of classifiers thus produced for many classification algorithms this simple strategy results in dramatic improvements in performance we show that this seemingly mysterious phenomenon can be understood in terms of well known statistical principles namely additive modeling and maximum likelihood for the two class problem boosting can be viewed as an approximation to additive modeling on the logistic scale using maximum bernoulli likelihood as a criterion we develop more direct approximations and show that they exhibit nearly identical results to boosting direct multiclass generalizations based on multinomial likelihood are derived that exhibit performance comparable to other recently proposed multiclass generalizations of boosting in most situations and far superior in some we suggest a minor modification to boosting that can reduce computation often by factors of 10 to 50 finally we apply these insights to produce an alternative formulation of boosting decision trees this approach based on best first truncated tree induction often leads to better performance and can provide interpretable descriptions of the aggregate decision rule it is also much faster computationally making it more suitable to large scale data mining applications

computing approximate fekete points by qr factorizations of vandermonde matrices we propose a numerical method implemented in matlab for computing approximate fekete points on compact multivariate domains it relies on the search of maximum volume submatrices of vandermonde matrices computed on suitable discretization meshes and uses a simple greedy algorithm based on qr factorization with column pivoting the method gives also automatically an algebraic cubature formula provided that the moments of the underlying polynomial basis are known numerical tests are presented for the interval and the square which show that approximate fekete points are well suited for polynomial interpolation and cubature

the reconstruction approach from interpolation to regression this article introduces an interpolation based method called the reconstruction approach for nonparametric regression based on the fact that interpolation usually has negligible errors compared

a lipatov bound for φ 4 4 euclidean field theory we bound the large order behavior of these pieces of the renormalized perturbation expansion for 4 4 which do not contain renormalons 1 the bound we obtain has the form of the leading asymptotic behavior computed by the lipatov method with the exact value of the lipatov constant therefore this paper is a step towards the rigorous study of the large order behavior of 4 4 and towards a proof of existence of the first renormalon singularity which should prevent the theory from being borel summable using the results of this paper and the techniques of 15 one can for instance improve 17 the estimate 18 on the finiteness of the radius of convergence of the borel transform of renormalized 4 4 and obtain that this radius is at least the exact value conjectured in 1

rounding error analysis of the classical gram schmidt orthogonalization process this paper provides two results on the numerical behavior of the classical gram schmidt algorithm the first result states that provided the normal equations associated with the initial vectors are numerically nonsingular the loss of orthogonality of the vectors computed by the classical gram schmidt algorithm depends quadratically on the condition number of the initial vectors the second result states that provided the initial set of vectors has numerical full rank two iterations of the classical gram schmidt algorithm are enough for ensuring the orthogonality of the computed vectors to be close to the unit roundoff level

polynomial interpolation and cubature over polygons we have implemented a matlab code to compute discrete extremal sets of fekete and leja type on convex or concave polygons together with the corresponding interpolatory cubature formulas the method works by qr and lu factorizations of rectangular vandermonde matrices on weakly admissible meshes wams of polygons constructed by polygon quadrangulation

propensity score matching and subclassification in observational studies with multi level treatments in this article we develop new methods for estimating average treatment effects in observational studies in settings with more than two treatment levels assuming unconfoundedness given pretreatment variables we emphasize propensity score subclassification and matching methods which have been among the most popular methods in the binary treatment literature whereas the literature has suggested that these particular propensity based methods do not naturally extend to the multi level treatment case we show using the concept of weak unconfoundedness and the notion of the generalized propensity score that adjusting for a scalar function of the pretreatment variables removes all biases associated with observed pretreatment variables we apply the proposed methods to an analysis of the effect of treatments for fibromyalgia we also carry out a simulation study to assess the finite sample performance of the methods relative to previously proposed methods

leading large order asymptotics for φ4 2 perturbation theory we develop a rigorous semiclassical expansion to compute the radius of convergence of the borel transform for the pressure in 4 2 field theory this result gives a partial justification for the lipatov method of finding large order perturbation theory asymptotics in quantum field theory

geometric weakly admissible meshes discrete least squares approximations and approximate fekete points using the concept of geometric weakly admissible meshes see 2 below together with an algorithm based on the classical qr factorization of matrices we compute efficient points for discrete multivariate least squares approximation and lagrange interpolation

boosting as a regularized path to a maximum margin classifier in this paper we study boosting methods from a new perspective we build on recent work by efron et al to show that boosting approximately and in some cases exactly minimizes its loss criterion with an l1 constraint on the coefficient vector this helps understand the success of boosting with early stopping as regularized fitting of the loss criterion for the two most commonly used criteria exponential and binomial log likelihood we further show that as the constraint is relaxed or equivalently as the boosting iterations proceed the solution converges in the separable case to an l1 optimal separating hyper plane we prove that this l1 optimal separating hyper plane has the property of maximizing the minimal l1 margin of the training data as defined in the boosting literature an interesting fundamental similarity between boosting and kernel support vector machines emerges as both can be described as methods for regularized optimization in high dimensional predictor space using a computational trick to make the calculation practical and converging to margin maximizing solutions while this statement describes svms exactly it applies to boosting only approximately

local existence of the borel transform in euclidean massless φ 4 4 we extend the methods of 1 to prove large order estimates on the renormalized feynman amplitudes of massless 4 4 euclidean field theory at non exceptional momenta the borel transform of the perturbative series is analytic in a disk centered at the origin of the complex plane this result is a step towards the rigorous investigation of the infra red singularities in the borel plane for theories containing massless particles like the gauge theories

reducing bias in observational studies using subclassification on the propensity score abstract the propensity score is the conditional probability of assignment to a particular treatment given a vector of observed covariates previous theoretical arguments have shown that subclassification on the propensity score will balance all observed covariates subclassification on an estimated propensity score is illustrated using observational data on treatments for coronary artery disease five subclasses defined by the estimated propensity score are constructed that balance 74 covariates and thereby provide estimates of treatment effects using direct adjustment these subclasses are applied within sub populations and model based adjustments are then used to provide estimates of treatment effects within these sub populations two appendixes address theoretical issues related to the application the effectiveness of subclassification on the propensity score in removing bias and balancing properties of propensity scores with incomplete data

computing almost minimal formulas on the square the main purpose of this paper is to introduce a numerical method for the computation of cubature rules on the square 1 1 2 that are almost minimal i e with few points w r t the degree of precision d in particular for all email protected email protected 55 the sets x d and the weights w k are new new rules are also achieved for degrees 15 d 17 19 and d 23 respectively with 43 54 67 and 96 points

numerical integration over polygons using an eight node quadrilateral spline finite element in this paper a cubature formula over polygons is proposed and analysed it is based on an eight node quadrilateral spline finite element c j li r h wang a new 8 node quadrilateral spline finite element j comp appl math 195 2006 54 65 and is exact for quadratic polynomials on arbitrary convex quadrangulations and for cubic polynomials on rectangular partitions the convergence of sequences of the above cubatures is proved for continuous integrand functions and error bounds are derived some numerical examples are given by comparisons with other known cubatures

boosting with early stopping convergence and consistency boosting is one of the most significant advances in machine learning for classification and regression in its original and computationally flexible version boosting seeks to minimize empirically a loss function in a greedy fashion the resulting estimator takes an additive function form and is built iteratively by applying a base estimator or learner to updated samples depending on the previous iterations an unusual regularization technique early stopping is employed based on cv or a test set this paper studies numerical convergence consistency and statistical rates of convergence of boosting with early stopping when it is carried out over the linear span of a family of basis functions for general loss functions we prove the convergence of boosting s greedy optimization to the infinimum of the loss function over the linear span using the numerical convergence result we find early stopping strategies under which boosting is shown to be consistent based on i i d samples and we obtain bounds on the rates of convergence for boosting estimators simulation studies are also presented to illustrate the relevance of our theoretical results for providing insights to practical aspects of boosting as a side product these results also reveal the importance of restricting the greedy search step sizes as known in practice through the work of friedman and others moreover our results lead to a rigorous proof that for a linearly separable problem adaboost with e 0 step size becomes an l 1 margin maximizer when left to run to convergence

greedy function approximation a gradient boosting machine function estimation approximation is viewed from the perspective of numerical optimization in function space rather than parameter space a connection is made between stagewise additive expansions and steepest descent minimization a general gradient descent boosting paradigm is developed for additive expansions based on any fitting criterion specific algorithms are presented for least squares least absolute deviation and huber m loss functions for regression and multiclass logistic likelihood for classification special enhancements are derived for the particular case where the individual additive components are regression trees and tools for interpreting such treeboost models are presented gradient boosting of regression trees produces competitive highly robust interpretable procedures for both regression and classification especially appropriate for mining less than clean data connections between this approach and the boosting methods of freund and shapire and friedman hastie and tibshirani are discussed

operator product expansion algebra we establish conceptually important properties of the operator product expansion ope in the context of perturbative euclidean 4 quantum field theory first we demonstrate generalizing earlier results and techniques of hep th 1105 3375 that the 3 point ope oa1oa2oa3 cca1a2a3c oc usually interpreted only as an asymptotic short distance expansion actually converges at finite and even large distances we further show that the factorization identity ca1a2a3b cca1a2ccca3b is satisfied for suitable configurations of the spacetime arguments again the infinite sum is shown to be convergent our proofs rely on explicit bounds on the remainders of these expansions obtained using refined versions mostly due to kopper et al of the renormalization group flow equation method these bounds also establish that each ope coefficient is a real analytic function in the spacetime arguments for non coinciding points our results hold for arbitrary but finite loop orders they lend support to proposals for

functional gradient ascent for probit regression this paper proposes two gradient based methods to fit a probit regression model by maximizing the sample log likelihood function using the property of the hessian of the objective function the first method performs weighted least square regression in each iteration of the newton raphson framework resulting in probitboost a boosting like algorithm motivated by the gradient boosting algorithm 10 the second proposed approach maximizes the sample log likelihood function by updating the fitted function a small step in the gradient direction performing gradient ascent in functional space resulting in gradient probitboost we also generalize the algorithms to multi class problems by two strategies one of which is to use the gradient ascent to maximize the multi class sample log likelihood function for fitting all the classifiers simultaneously and the second approach uses the one versus all scheme to reduce the multi class problem to a series of binary classification problems the proposed algorithms are tested on typical classification problems including face detection cancer classification and handwritten digit recognition the results show that compared to the alternative methods the proposed algorithms perform similar or better in terms of testing error rates

on selecting a maximum volume sub matrix of a matrix and related problems given a matrix email protected r m x n n vectors in m dimensions we consider the problem of selecting a subset of its columns such that its elements are as linearly independent as possible this notion turned out to be important in low rank approximations to matrices and rank revealing qr factorizations which have been investigated in the linear algebra community and can be quantified in a few different ways in this paper from a complexity theoretic point of view we propose four related problems in which we try to find a sub matrix email protected r m x k of a given matrix email protected r m x n such that i s m a x c the largest singular value of c is minimum ii s m i n c the smallest singular value of c is maximum iii k c email protected m a x c s m i n c the condition number of c is minimum and iv the volume of the parallelepiped defined by the column vectors of c is maximum we establish the np hardness of these problems and further show that they do not admit ptas we then study a natural greedy heuristic for the maximum volume problem and show that it has approximation ratio 2 o k l o g k our analysis of the greedy heuristic is tight to within a logarithmic factor in the exponent which we show by explicitly constructing an instance for which the greedy heuristic is 2 w k from optimal when a has unit norm columns a related problem is to select the maximum number of vectors with a given volume we show that if the optimal solution selects k columns then greedy will select w k logk columns providing a logk approximation

a tutorial on propensity score estimation for multiple treatments using generalized boosted models the use of propensity scores to control for pretreatment imbalances on observed variables in non randomized or observational studies examining the causal effects of treatments or interventions has become widespread over the past decade for settings with two conditions of interest such as a treatment and a control inverse probability of treatment weighted estimation with propensity scores estimated via boosted models has been shown in simulation studies to yield causal effect estimates with desirable properties there are tools e g the twang package in r and guidance for implementing this method with two treatments however there is not such guidance for analyses of three or more treatments the goals of this paper are twofold 1 to provide step by step guidance for researchers who want to implement propensity score weighting for multiple treatments and 2 to propose the use of generalized boosted models gbm for estimation of the necessary propensity score weights we define the causal quantities that may be of interest to studies of multiple treatments and derive weighted estimators of those quantities we present a detailed plan for using gbm to estimate propensity scores and using those scores to estimate weights and causal effects we also provide tools for assessing balance and overlap of pretreatment variables among treatment groups in the context of multiple treatments a case study examining the effects of three treatment programs for adolescent substance abuse demonstrates the methods

stratified nested and related quadrature rules the stratified nested quadrature procedure due to laurie is discussed together with an alternative computational procedure which leads to the concept of hybrid gkp rules in the context of the approximation of stratified nested sequences the work of krogh and van snyder on the representation of the gkp rules is considered and a generalisation of this employing hybrid rules of special form is discussed

building support vector machines with reduced classifier complexity support vector machines svms though accurate are not preferred in applications requiring great classification speed due to the number of support vectors being large to overcome this problem we devise a primal method with the following properties 1 it decouples the idea of basis functions from the concept of support vectors 2 it greedily finds a set of kernel basis functions of a specified maximum size dmax to approximate the svm primal cost function well 3 it is efficient and roughly scales as o ndmax2 where n is the number of training examples and 4 the number of basis functions it requires to achieve an accuracy close to the svm accuracy is usually far less than the number of svm support vectors

numerical integration over arbitrary polygonal domains based on schwarz christoffel conformal mapping this paper presents a new numerical integration technique oil arbitrary polygonal domains the polygonal domain is mapped conformally to the unit disk using schwarz christoffel mapping and a midpoint quadrature rule defined oil this unit disk is used this method eliminates the need for a two level isoparametric mapping usually required moreover the positivity of the jacobian is guaranteed numerical results presented for a few benchmark problems in the context of polygonal finite elements show that the proposed method yields accurate results

gradient tree boosting for training conditional random fields conditional random fields crfs provide a flexible and powerful model for sequence labeling problems however existing learning algorithms are slow particularly in problems with large numbers of potential input features and feature combinations this paper describes a new algorithm for training crfs via gradient tree boosting in tree boosting the crf potential functions are represented as weighted sums of regression trees which provide compact representations of feature interactions so the algorithm does not explicitly consider the potentially large parameter space as a result gradient tree boosting scales linearly in the order of the markov model and in the order of the feature interactions rather than exponentially as in previous algorithms based on iterative scaling and gradient descent gradient tree boosting also makes it possible to use instance weighting as in c4 5 and surrogate splitting as in cart to handle missing values experimental studies of the effectiveness of these two methods as well as standard imputation and indicator feature methods show that instance weighting is the best method in most cases when feature values are missing at random

taylor type 1 step ahead numerical differentiation rule for first order derivative approximation and znn discretization in order to achieve higher computational precision in approximating the first order derivative and discretize more effectively the continuous time zhang neural network znn a taylor type numerical differentiation rule is proposed and investigated in this paper this rule not only greatly remedies some intrinsic weaknesses of the backward and central numerical differentiation rules but also overcomes the limitation of the lagrange type numerical differentiation rules in znn discretization in addition a formula is proposed to obtain the optimal step length of the taylor type numerical differentiation rule moreover based on the proposed numerical differentiation rule the stability convergence and residual error of the taylor type discrete time znn dtznn are analyzed numerical experimental results further substantiate the efficacy and advantages of the proposed taylor type numerical differentiation rule for first order derivative approximation and znn discretization a formula is proposed to approximate the first order derivative an optimal step length rule for the proposed formula is investigated a taylor type znn model is derived for time varying matrix inversion

statistics and causal inference abstract problems involving causal inference have dogged at the heels of statistics since its earliest days correlation does not imply causation and yet causal conclusions drawn from a carefully designed experiment are often valid what can a statistical model say about causation this question is addressed by using a particular model for causal inference holland and rubin 1983 rubin 1974 to critique the discussions of other writers on causation and causal inference these include selected philosophers medical researchers statisticians econometricians and proponents of causal modeling

tensor product gauss lobatto points are fekete points for the cube tensor products of gauss lobatto quadrature points are frequently used as collocation points in spectral element methods unfortunately it is not known if gauss lobatto points exist in non tensor product domains like the simplex in this work we show that the n dimensional tensor product of gauss lobatto quadrature points are also fekete points this suggests a way to generalize spectral methods based on gauss lobatto points to non tensor product domains since fekete points are known to exist and have been computed in the triangle and tetrahedron in one dimension this result was proved by fejer in 1932 but the extension to higher dimensions in non trivial

functional dissipation microarrays for classification in this article we describe a new method of extracting information from signals called functional dissipation that proves to be very effective for enhancing classification of high resolution texture rich data our algorithm bypasses to some extent the need to have very specialized feature extraction techniques and can potentially be used as an intermediate feature enhancement step in any classification scheme functional dissipation is based on signal transforms but uses the transforms recursively to uncover new features we generate a variety of masking functions and extract features with several generalized matching pursuit iterations in each iteration the recursive process modifies several coefficients of the transformed signal with the largest absolute values according to the specific masking function in this way the greedy pursuit is turned into a slow controlled dissipation of the structure of the signal that for some masking functions enhances separation among classes our case study in this paper is the classification of crystallization patterns of amino acids solutions affected by the addition of small quantities of proteins

prediction games and arcing algorithms the theory behind the success of adaptive reweighting and combining algorithms arcing such as adaboost freund schapire 1996a 1997 and others in reducing generalization error has not been well understood by formulating prediction as a game where one player makes a selection from instances in the training set and the other a convex linear combination of predictors from a finite set existing arcing algorithms are shown to be algorithms for finding good game strategies the minimax theorem is an essential ingredient of the convergence proofs an arcing algorithm is described that converges to the optimal strategy a bound on the generalization error for the combined predictors in terms of their maximum error is proven that is sharper than bounds to date schapire freund bartlett and lee 1997 offered an explanation of why adaboost works in terms of its ability to produce generally high margins the empirical comparison of adaboost to the optimal arcing algorithm shows that their explanation is n

least squares polynomial approximation on weakly admissible meshes disk and triangle we construct symmetric polar wams weakly admissible meshes with low cardinality for least squares polynomial approximation on the disk these are then mapped to an arbitrary triangle numerical tests show that the growth of the least squares projection uniform norm is much slower than the theoretical bound and even slower than that of the lebesgue constant of the best known interpolation points for the triangle as opposed to good interpolation points such meshes are straightforward to compute for any degree the construction can be extended to polygons by triangulation

computing multivariate fekete and leja points by numerical linear algebra we discuss and compare two greedy algorithms that compute discrete versions of fekete like points for multivariate compact sets by basic tools of numerical linear algebra the first gives the so called approximate fekete points by qr factorization with column pivoting of vandermonde like matrices the second computes discrete leja points by lu factorization with row pivoting moreover we study the asymptotic distribution of such points when they are extracted from weakly admissible meshes

local borel summability of euclidean 311 1311 1311 1a simple proof via differential flow equations it is shown how the differential flow equation or equivalently the continous renormalization group method can be employed to give an astonishingly easy proof of the local borel summability of the renormalized perturbative euclidean massive 44