higher order properties of the wild bootstrap under misspecification abstract we examine the higher order properties of the wild bootstrap wu 1986 in a linear regression model with stochastic regressors we find that the ability of the wild bootstrap to provide a higher order refinement is contingent upon whether the errors are mean independent of the regressors or merely uncorrelated with them in the latter case the wild bootstrap may fail to match some of the terms in an edgeworth expansion of the full sample test statistic nonetheless we show that the wild bootstrap still has a lower maximal asymptotic risk as an estimator of the true distribution than a normal approximation in shrinking neighborhoods of properly specified models to assess the practical implications of this result we conduct a monte carlo study contrasting the performance of the wild bootstrap with a normal approximation and the traditional nonparametric bootstrap

fine tuning nesterov s steepest descent algorithm for differentiable convex programming we modify the first order algorithm for convex programming described by nesterov in his book in introductory lectures on convex optimization a basic course kluwer boston 2004 in his algorithms nesterov makes explicit use of a lipschitz constant l for the function gradient which is either assumed known nesterov in introductory lectures on convex optimization a basic course kluwer boston 2004 or is estimated by an adaptive procedure nesterov 2007 we eliminate the use of l at the cost of an extra imprecise line search and obtain an algorithm which keeps the optimal complexity properties and also inherit the global convergence properties of the steepest descent method for general continuously differentiable optimization besides this we develop an adaptive procedure for estimating a strong convexity constant for the function numerical tests for a limited set of toy problems show an improvement in performance when compared with the original nesterov s algorithms

an optimal variant of kelley s cutting plane method we propose a new variant of kelley s cutting plane method for minimizing a nonsmooth convex lipschitz continuous function over the euclidean space we derive the method through a constructive approach and prove that it attains the optimal rate of convergence for this class of problems

quasi aitken estimation for heteroskedasticity of unknown form abstract two modifications to standard aitken estimation of heteroskedastic linear regressions are suggested 1 different criteria for calculating unknown parameters to yield more efficient estimates to the coefficients when the form of heteroskedasticity is misspecified and 2 a different way of estimating the covariance matrix of the feasible aitken estimator a sampling experiment provides evidence that the first adjustment could provide more efficient estimates though not clearly more so than alternatives the second adjustment is crucial for reliable inference even when the form of heteroskedasticity is correctly specified

bootstrapping regression models the regression and correlation models are considered it is shown that the bootstrap approximation to the distribution of the least squares estimates is valid and some error bounds are given

extremes of a class of nonhomogeneous gaussian random fields this contribution establishes exact tail asymptotics of sup s t e x s t for a large class of nonhomogeneous gaussian random fields x on a bounded convex set e r2 with variance function that attains its maximum on a segment on e these findings extend the classical results for homogeneous gaussian random fields and gaussian random fields with unique maximum point of the variance applications of our result include the derivation of the exact tail asymptotics of the shepp statistics for stationary gaussian processes brownian bridge and fractional brownian motion as well as the exact tail asymptotic expansion for the maximum loss and span of stationary gaussian processes

approximate power functions for some robust tests of regression coefficients approximate power functions for some robust tests of regression coefficients thomas j rothenberg department o f economics u n i v e r s i t y o f c a l i f o r n i a berkeley research papers i n economics no 84 1 summary edgeworth expansions are developed f o r the d i s t r i b u t i o n functions o f some t e s t s t a t i s t i c s i n the normal l i n e a r regression model where the e r r o r covariance matrix i s unknown tests based on generalized l e a s t squares estimates and also on ordinary l e a s t squares estimates are considered in both cases adjustments t o the asymptotic c r i t i c a l the approxi values are found and approximate l o c a l power c a l c u l a t e d and a u t o c o r r e l a t i o n mations are applied to a number o f examples i n c l u d i n g h e t e r o s c e d a s t i c i t y

testing inference in heteroskedastic fixed effects models abstract this paper considers the issue of performing testing inference in fixed effects panel data models under heteroskedasticity of unknown form we use numerical integration to compute the exact null distributions of different quasi t test statistics and compare them to their limiting counterpart the test statistics use different heteroskedasticity consistent standard errors our results reveal that the asymptotic approximation is usually poor in small samples when the test statistic is based on the covariance matrix estimator proposed by arellano 1987 the quality of the approximation is greatly increased when the standard error is obtained using other heteroskedasticity consistent estimators most notably the chc4 estimator our results also reveal that the performance of arellano s test improves considerably when standard errors are computed using restricted residuals

backward forward reflected backward splitting for three operator monotone inclusions abstract in this work we propose and analyse two splitting algorithms for finding a zero of the sum of three monotone operators one of which is assumed to be lipschitz continuous each iteration of these algorithms require one forward evaluation of the lipschitz continuous operator and one resolvent evaluation of each of the other two operators by specialising to two operator inclusions we recover the forward reflected backward and the reflected forward backward splitting methods as particular cases the inspiration for the proposed algorithms arises from interpretations of the aforementioned reflected splitting algorithms as discretisations of the continuous time proximal point algorithm

extreme values and high boundary crossings of locally stationary gaussian processes we consider the large values of a locally stationary gaussian process which satisfies berman s condition on the long range dependence we present some limit results on the exceedances of the process above a certain general smooth high boundary this allows deriving the limiting distribution of the maximum up to time t for example in the case of a standardized process with a constant boundary or in the case of a nonstandardized process with a smooth trend

consistent estimates of deformed isotropic gaussian random fields on the plane this paper proves fixed domain asymptotic results for estimating a smooth invertible transformation f r 2 r 2 when observing the deformed random field z o f on a dense grid in a bounded simply connected domain where z is assumed to be an isotropic gaussian random field on r 2 is constructed on a simply connected domain u such that and is defined using kernel smoothed quadratic variations bergman projections and results from quasiconformal theory we show under mild assumptions on the random field z and the deformation f that r f c uniformly on compact subsets of u with probability one as the grid spacing goes to zero where r is an unidentifiable rotation and c is an unidentifiable translation

some heteroskedasticity consistent covariance matrix estimators with improved finite sample properties abstract we examine several modified versions of the heteroskedasticity consistent covariance matrix estimator of hinkley 1977 and white 1980 on the basis of sampling experiments which compare the performance of quasi t statistics we find that one estimator based on the jackknife performs better in small samples than the rest we also examine the finite sample properties of using modified critical values based on edgeworth approximations as proposed by rothenberg 1984 in addition we compare the power of several tests for heteroskedasticity and find that it may be wise to employ the jackknife heteroskedasticity consistent covariance matrix even in the absence of detected heteroskedasticity

second order approximation in the partially linear regression model we examine the second order properties of various quantities of interest in the partially linear regression model we obtain a stochastic expansion with remainder op n 2 4 where u 1 2 for the standardized semiparametric least squares estimator a standard error estimator and a studentized statistic we use the second order expansions to correct the standard error estimates for second order effects and to define a method of bandwidth choice a monte carlo experiment provides favorable evidence on our method of bandwidth choice

extremes of locally stationary gaussian random fields the main result of this contribution is the derivation of the exact asymptotic behavior of the supremum of a class of a t locally stationary gaussian random fields we present two applications of our result the first one deals with the extremes of aggregate multifractional brownian motions whereas the second one establishes the exact asymptotics of the supremum of the x process generated by multifractional brownian motions

performance of first order methods for smooth convex minimization a novel approach we introduce a novel approach for analyzing the worst case performance of first order black box optimization methods we focus on smooth unconstrained convex minimization over the euclidean space our approach relies on the observation that by definition the worst case behavior of a black box optimization method is by itself an optimization problem which we call the performance estimation problem pep we formulate and analyze the pep for two classes of first order algorithms we first apply this approach on the classical gradient method and derive a new and tight analytical bound on its performance we then consider a broader class of first order black box methods which among others include the so called heavy ball method and the fast gradient schemes we show that for this broader class it is possible to derive new bounds on the performance of these methods by solving an adequately relaxed convex semidefinite pep finally we show an efficient procedure for finding optimal step sizes which results in a first order black box method that achieves best worst case performance

primal dual first order methods with mathcal o 1 epsilon iteration complexity for cone programming in this paper we consider the general cone programming problem and propose primal dual convex smooth and or nonsmooth minimization reformulations for it we then discuss first order methods suitable for solving these reformulations namely nesterov s optimal method nesterov in doklady an sssr 269 543 547 1983 math program 103 127 152 2005 nesterov s smooth approximation scheme nesterov in math program 103 127 152 2005 and nemirovski s prox method nemirovski in siam j opt 15 229 251 2005 and propose a variant of nesterov s optimal method which has outperformed the latter one in our computational experiments we also derive iteration complexity bounds for these first order methods applied to the proposed primal dual reformulations of the cone programming problem the performance of these methods is then compared using a set of randomly generated linear programming and semidefinite programming instances we also compare the approach based on the variant of nesterov s optimal method with the low rank method proposed by burer and monteiro math program ser b 95 329 357 2003 math program 103 427 444 2005 for solving a set of randomly generated sdp instances

on the power of tests for superexogeneity and structural invariance abstract this paper examines the finite sample power of tests of structural invariance and superexogeneity hypotheses in econometric models with contemporaneous conditioning variables we consider both direct parametric tests of superexogeneity as well as indirect procedures based on temporal stability tests for the parameters of interest our monte carlo analysis reveals that both types of tests may lack power in interesting classes of models an empirical illustration investigates the superexogeneity of the short term interest rate in a dynamic specification for the u s term structure

cyclic schemes for pde based image analysis we investigate a class of efficient numerical algorithms for many partial differential equations pdes in image analysis they are applicable to parabolic or elliptic pdes that have bounded coefficients and lead to space discretisations with symmetric matrices our schemes are easy to implement and well suited for parallel implementations on gpus since they are based on the explicit diffusion scheme in the parabolic case and the jacobi method in the elliptic case by supplementing these methods with cyclically varying time step sizes or relaxation parameters we achieve efficiency gains of several orders of magnitude we call the resulting algorithms fast explicit diffusion fed and fast jacobi fj methods to achieve a good compromise between efficiency and accuracy we show that one should use parameter cycles that result from factorisations of box filters for these cycles we establish stability results in the euclidean norm our schemes perform favourably in a number of applications including isotropic nonlinear diffusion filters with widely varying diffusivities as well as anisotropic diffusion methods for image filtering inpainting and regularisation in computer vision moreover they are equally suited for higher dimensional problems as well as higher order pdes and they can also be interpreted as efficient first order methods for smooth optimisation problems

improved heteroscedasticity consistent covariance matrix estimators summary the heteroscedasticity consistent covariance matrix estimator proposed by white 1980 is commonly used in practical applications and is implemented into a number of pieces of statistical software however although consistent it can display substantial bias in small to moderately large samples as shown by monte carlo simulations elsewhere this paper defines modified white estimators which are approximately bias free numerical results show that the modified estimators display much smaller bias than white s estimator in small samples we also show that the bias correction leads to some variance inflation in hypothesis testing based on heteroscedasticity consistent covariance matrix estimators numerical results suggest that tests based on the proposed bias corrected estimators typically display smaller size distortions

a note on studentizing a test for heteroscedasticity abstract breusch and pagan 1979 have recently proposed a convenient test for heteroscedasticity in general linear models this note derives the asymptotic distribution of their test under sequences of contiguous alternatives to the null hypothesis of homoscedasticity the test is shown to possess asymptotically incorrect size nominal significance level except in the case of strictly gaussian disturbances a slight modification of the test is proposed which corrects this defect

extremes of α t locally stationary gaussian processes with non constant variances abstract with motivation from 9 in this paper we derive the exact tail asymptotics of t locally stationary gaussian processes with non constant variance functions we show that some certain variance functions lead to qualitatively new results

regularized nonlinear acceleration we describe a convergence acceleration technique for generic optimization problems our scheme computes estimates of the optimum from a nonlinear average of the iterates produced by any optimization method the weights in this average are computed via a simple linear system whose solution can be updated online this acceleration scheme runs in parallel to the base algorithm providing improved estimates of the solution on the fly while the original optimization method is running numerical experiments are detailed on classical classification problems

extremes of randomly scaled gumbel risks we investigate the product y1y2 of two independent positive risks y1 and y2 if y1 has distribution in the gumbel max domain of attraction with some auxiliary function which is regularly varying at infinity and y2 is bounded then we show that y1y2 has also distribution in the gumbel max domain of attraction if both y1 y2 have log weibullian or weibullian tail behaviour we prove that y1y2 has log weibullian or weibullian asymptotic tail behaviour respectively we present here three theoretical applications concerned with a the limit of point wise maxima of randomly scaled gaussian processes b extremes of gaussian processes over random intervals and c the tail of supremum of iterated processes

resurrecting weighted least squares this paper shows how asymptotically valid inference in regression models based on the weighted least squares wls estimator can be obtained even when the model for reweighting the data is misspecified like the ordinary least squares estimator the wls estimator can be accompanied by heterokedasticty consistent hc standard errors without knowledge of the functional form of conditional heteroskedasticity first we provide rigorous proofs under reasonable assumptions second we provide numerical support in favor of this approach indeed a monte carly study demonstrates attractive finite sample properties compared to the status quo both in terms of estimation and making inference

an interior point method for semidefinite programming we propose a new interior point based method to minimize a linear function of a matrix variable subject to linear equality and inequality constraints over the set of positive semidefinite matrices

bootstrap and wild bootstrap for high dimensional linear models in this paper two bootstrap procedures are considered for the estimation of the distribution of linear contrasts and of f test statistics in high dimensional linear models an asymptotic approach will be chosen where the dimension p of the model may increase for sample size n rightarrow infty the range of validity will be compared for the normal approximation and for the bootstrap procedures furthermore it will be argued that the rates of convergence are different for the bootstrap procedures in this asymptotic framework this is in contrast to the usual asymptotic approach where p is fixed

improved algorithms for convex minimization in relative scale in this paper we propose two modifications to nesterov s algorithms for minimizing convex functions in relative scale the first is based on a bisection technique and leads to improved theoretical iteration complexity and the second is a heuristic for avoiding restarting behavior the fastest of our algorithms produces a solution within relative error o 1 k of the optimum with k being the iteration counter

some results on the glejser and koenker tests for heteroskedasticity abstract the glejser and koenker studentized lagrange multiplier tests for heteroskedasticity are considered asymptotic theory and monte carlo experiments are used to investigate the effects of nonnormality under null and alternative hypotheses and also the consequences of using an incorrect alternative a method for using test outcomes to select models of heteroskedasticity is critically examined glejser s test is not in general asymptotically valid under asymmetric disturbances koenker s test is asymptotically valid under asymmetric and other nonnormal disturbances but has estimated finite sample significance levels that are sometimes sensitive to skewness an adjusted version of koenker s statistic is discussed

on the existence of paths between points in high level excursion sets of gaussian random fields research supported in part by us israel binational science foundation 2008262 by aro grant w911nf 07 1 0078 nsf grant dms 1005903 and nsa grant h98230 11 1 0154 at cornell university by israel science foundation 853 10 by afosr fa8655 11 1 3039 by office for absorption of new scientists and erc grant rcmt152

a note on the efficiency of sandwich covariance matrix estimation the sandwich estimator also known as robust covariance matrix estimator heteroscedasticity consistent covariance matrix estimate or empirical covariance matrix estimator has achieved increasing use in the econometric literature as well as with the growing popularity of generalized estimating equations its virtue is that it provides consistent estimates of the covariance matrix for parameter estimates even when the fitted parametric model fails to hold or is not even specified surprisingly though there has been little discussion of properties of the sandwich method other than consistency we investigate the sandwich estimator in quasi likelihood models asymptotically and in the linear case analytically we show that under certain circumstances when the quasi likelihood model is correct the sandwich estimate is often far more variable than the usual parametric variance estimate the increased variance is a fixed feature of the method and the price that one pays to obtain consistency even when the p

subsampled hessian newton methods for supervised learning newton methods can be applied in many supervised learning approaches however for large scale data the use of the whole hessian matrix can be time consuming recently subsampled newton methods have been proposed to reduce the computational time by using only a subset of data for calculating an approximation of the hessian matrix unfortunately we find that in some situations the running speed is worse than the standard newton method because cheaper but less accurate search directions are used in this work we propose some novel techniques to improve the existing subsampled hessian newton method the main idea is to solve a two dimensional subproblem per iteration to adjust the search direction to better minimize the second order approximation of the function value we prove the theoretical convergence of the proposed method experiments on logistic regression linear svm maximum entropy and deep networks indicate that our techniques significantly reduce the running time of the subsampled hessian newton method the resulting algorithm becomes a compelling alternative to the standard newton method for large scale data classification

smooth strongly convex interpolation and exact worst case performance of first order methods we show that the exact worst case performance of fixed step first order methods for unconstrained optimization of smooth possibly strongly convex functions can be obtained by solving convex programs finding the worst case performance of a black box first order method is formulated as an optimization problem over a set of smooth strongly convex functions and initial conditions we develop closed form necessary and sufficient conditions for smooth strongly convex interpolation which provide a finite representation for those functions this allows us to reformulate the worst case performance estimation problem as an equivalent finite dimension independent semidefinite optimization problem whose exact solution can be recovered up to numerical precision optimal solutions to this performance estimation problem provide both worst case performance bounds and explicit functions matching them as our smooth strongly convex interpolation procedure is constructive our works build on those of drori and teboulle math program 145 1 2 451 482 2014 who introduced and solved relaxations of the performance estimation problem for smooth convex functions we apply our approach to different fixed step first order methods with several performance criteria including objective function accuracy and gradient norm we conjecture several numerically supported worst case bounds on the performance of the fixed step gradient fast gradient and optimized gradient methods both in the smooth convex and the smooth strongly convex cases and deduce tight estimates of the optimal step size for the gradient method

small sample adjustments for wald type tests using sandwich estimators the sandwich estimator of variance may be used to create robust wald type tests from estimating equations that are sums of k independent or approximately independent terms for example for repeated measures data on k individuals each term relates to a different individual these tests applied to a parameter may have greater than nominal size if k is small or more generally if the parameter to be tested is essentially estimated from a small number of terms in the estimating equation we offer some practical modifications to these robust wald type tests which asymptotically approach the usual robust wald type tests we show that one of these modifications provides exact coverage for a simple case and examine by simulation the modifications applied to the generalized estimating equations of liang and zeger 1986 conditional logistic regression and the cox proportional hazard model

heteroscedasticity robust inference in linear regression models with many covariates we consider inference in linear regression models that is robust to heteroscedasticity and the presence of many control variables when the number of control variables increases at the same rate as

the finite sample distributions of heteroskedasticity robust wald statistics the imhof procedure is used to calculate the exact finite sample distributions of alternative heteroskedasticity robust wald type tests of scalar linear hypothesis in the normal linear model the tests are distinguished by the variance estimator that they use these include the white eicker and jackknife estimators the calculations demonstrate the importance of regression design in determining the quality of conventional first order asymptotic approximations to the null distributions of the tests

quadratic matrix programming we introduce and study a special class of nonconvex quadratic problems in which the objective and constraint functions have the form f boldmath x tr boldmath x t boldmath a boldmath x 2 tr boldmath b t boldmath x c boldmath x in real r n times r the latter formulation is termed quadratic matrix programming qmp of order r we construct a specially devised semidefinite relaxation sdr and dual for the qmp problem and show that under some mild conditions strong duality holds for qmp problems with at most r constraints using a result on the equivalence of two characterizations of the nonnegativity property of quadratic functions of the above form we are able to compare the constructed sdr and dual problems to other known sdrs and dual formulations of the problem an application to robust least squares problems is discussed

analysis of biased stochastic gradient descent using sequential semidefinite programs we present a convergence rate analysis for biased stochastic gradient descent sgd where individual gradient updates are corrupted by computation errors we develop stochastic quadratic constraints to formulate a small linear matrix inequality lmi whose feasible points lead to convergence bounds of biased sgd based on this lmi condition we develop a sequential minimization approach to analyze the intricate trade offs that couple stepsize selection convergence rate optimization accuracy and robustness to gradient inaccuracy we also provide feasible points for this lmi and obtain theoretical formulas that quantify the convergence properties of biased sgd under various assumptions on the loss functions

models for longitudinal data a generalized estimating equation approach this article discusses extensions of generalized linear models for the analysis of longitudinal data two approaches are considered subject specific ss models in which heterogeneity in regression parameters is explicitly modelled and population averaged pa models in which the aggregate response for the population is the focus we use a generalized estimating equation approach to fit both classes of models for discrete and continuous outcomes when the subject specific parameters are assumed to follow a gaussian distribution simple relationships between the pa and ss parameters are available the methods are illustrated with an analysis of data on mother s smoking and children s respiratory disease

the wild bootstrap tamed at last the wild bootstrap is studied in the context of regression models with heteroskedastic disturbances we show that in one very specific case perfect bootstrap inference is possible and a substantial reduction in the error in the rejection probability of a bootstrap test is available much more generally however the version of the wild bootstrap with this desirable property is without the skewness correction afforded by the currently most popular version of the wild bootstrap simulation experiments show that this does not prevent the preferred version from having the smallest error in rejection probability in small and medium sized samples

inference in linear regression models with many covariates and heteroskedasticity the linear regression model is widely used in empirical work in economics statistics and many other disciplines researchers often include many covariates in their linear model specification in a

an optimized wild bootstrap method for evaluation of measurement uncertainties of dti derived parameters in human brain evaluation of measurement uncertainties or errors in diffusion tensor derived parameters is essential to quantify the sensitivity and specificity of these quantities as potential surrogate biomarkers for pathophysiological processes with diffusion tensor imaging dti computational methods such as the monte carlo simulation have provided insights into characterization of the measurement uncertainty in dti however due to the complexity of real brain data as well as different sources of variations during the image acquisition a robust estimator for dti measurement uncertainty in human brain is required recent studies have shown that wild bootstrap a novel nonparametric statistical method can potentially provide effective estimations of dti measurement uncertainties in human brain dti data in this study we further optimized the dti application of the wild bootstrap method for typical clinical applications we evaluated the validity of wild bootstrap utilizing numerical simulations with different combinations of dti protocol parameters and wild bootstrap experimental designs and quantitatively compared estimates of uncertainties from wild bootstrapping with those from monte carlo simulations our results demonstrate that a wild bootstrap implementation using at least 1000 wild bootstrap iterations with a type ii or type iii heteroskedasticity consistent covariance matrix estimator provides robust evaluations of most dti protocols

the moving blocks bootstrap and robust inference for linear least squares and quantile regressions abstract this paper presents the moving blocks bootstrap mbb as a new inference procedure for linear regression estimation which is robust to heteroskedasticity and autocorrelation of unknown forms the mbb covariance estimator is shown to provide heteroskedasticity and autocorrelation consistent hac standard errors for least squares ls and quantile regression qr coefficient estimators the mbb covariance estimator is shown to be asymptotically equivalent to the bartlett kernel estimator suggested by newey and west 1987 and the asymptotically optimal choice of the blocksize is discussed a monte carlo study is included showing that the mbb fares well in comparison to standard hac inference procedures considering strong mixing data generating processes the paper extends existing asymptotic results for the qr estimator the analogy to the ls case is stressed the paper analyzes both the cases of stochastic and of nonstochastic regressors and suggests two new grenander like conditions for the latter case the use of the mbb approach is illustrated for a practical example using a standard econometric package

fixed effects estimation of structural parameters and marginal effects in panel probit models fixed effects estimators of nonlinear panel models can be severely biased due to the incidental parameters problem in this paper i characterize the leading term of a large t expansion of the bias of the mle and estimators of average marginal effects in parametric fixed effects panel binary choice models for probit index coefficients the former term is proportional to the true value of the coefficients being estimated this result allows me to derive a lower bound for the bias of the mle i then show that the resulting fixed effects estimates of ratios of coefficients and average marginal effects exhibit no bias in the absence of heterogeneity and negligible bias for a wide variety of distributions of regressors and individual effects in the presence of heterogeneity i subsequently propose new bias corrected estimators of index coefficients and marginal effects with improved finite sample properties for linear and nonlinear models with predetermined regressors

longitudinal data analysis using generalized linear models summary this paper proposes an extension of generalized linear models to the analysis of longitudinal data we introduce a class of estimating equations that give consistent estimates of the regression parameters and of their variance under mild assumptions about the time dependence the estimating equations are derived without specifying the joint distribution of a subject s observations yet they reduce to the score equations for multivariate gaussian outcomes asymptotic theory is presented for the general class of estimators specific cases in which we assume independence m dependence and exchangeable correlation structures from each subject are discussed efficiency of the proposed estimators in two simple situations is considered the approach is closely related to quasi likelih ood some key ironh estimating equation generalized linear model longitudinal data quasi likelihood repeated measures