random directions stochastic approximation with deterministic perturbations we introduce deterministic perturbation dp schemes for the recently proposed random directions stochastic approximation and propose new first order and second order algorithms in the latter case these are the first second order algorithms to incorporate dps we show that the gradient and or hessian estimates in the resulting algorithms with dps are asymptotically unbiased so that the algorithms are provably convergent furthermore we derive convergence rates to establish the superiority of the first order and second order algorithms for the special case of a convex and quadratic optimization problem respectively numerical experiments are used to validate the theoretical results

a one measurement form of simultaneous perturbation stochastic approximation the simultaneous perturbation stochastic approximation spsa algorithm has proven very effective for difficult multivariate optimization problems where it is not possible to obtain direct gradient information as discussed to date spsa is based on a highly efficient gradient approximation requiring only two measurements of the loss function independent of the number of parameters being estimated this note presents a form of spsa that requires only one function measurement for any dimension theory is presented that identifies the class of problems for which this one measurement form will be asymptotically superior to the standard two measurement form

adaptive system optimization using random directions stochastic approximation we present new algorithms for simulation optimization using random directions stochastic approximation rdsa these include first order gradient as well as second order newton schemes we incorporate both continuous valued as well as discrete valued perturbations into both types of algorithms the former are chosen to be independent and identically distributed i i d symmetric uniformly distributed random variables r v while the latter are i i d asymmetric bernoulli r v s our newton algorithm with a novel hessian estimation scheme requires n dimensional perturbations and three loss measurements per iteration whereas the simultaneous perturbation newton search algorithm of 1 requires 2n dimensional perturbations and four loss measurements per iteration we prove the asymptotic unbiasedness of both gradient and hessian estimates and asymptotic strong convergence for both first order and second order schemes we also provide asymptotic normality results which in particular establish that the asymmetric bernoulli variant of newton rdsa method is better than 2spsa of 1 numerical experiments are used to validate the theoretical results

newton based stochastic optimization using q gaussian smoothed functional algorithms we present the first q gaussian smoothed functional sf estimator of the hessian and the first newton based stochastic optimization algorithm that estimates both the hessian and the gradient of the objective function using q gaussian perturbations our algorithm requires only two system simulations regardless of the parameter dimension and estimates both the gradient and the hessian at each update epoch using these we also present a proof of convergence of the proposed algorithm in a related recent work ghoshdastidar dukkipati bhatnagar 2014 we presented gradient sf algorithms based on the q gaussian perturbations our work extends prior work on sf algorithms by generalizing the class of perturbation distributions as most distributions reported in the literature for which sf algorithms are known to work turn out to be special cases of the q gaussian distribution besides studying the convergence properties of our algorithm analytically we also show the results of numerical simulations on a model of a queuing network that illustrate the significance of the proposed method in particular we observe that our algorithm performs better in most cases over a wide range of q values in comparison to newton sf algorithms with the gaussian and cauchy perturbations as well as the gradient q gaussian sf algorithms

better bootstrap confidence intervals abstract we consider the problem of setting approximate confidence intervals for a single parameter in a multiparameter family the standard approximate intervals based on maximum likelihood theory can be quite misleading in practice tricks based on transformations bias corrections and so forth are often used to improve their accuracy the bootstrap confidence intervals discussed in this article automatically incorporate such tricks without requiring the statistician to think them through for each new application at the price of a considerable increase in computational effort the new intervals incorporate an improvement over previously suggested methods which results in second order correctness in a wide variety of problems in addition to parametric families bootstrap intervals are also developed for nonparametric situations

homotopy invariants of nonorientable 4 manifolds we define a z 4 quadratic function on 2 for nonorientable 4 manifolds and show that it is a homotopy invariant we then use it to distinguish homotopy types of certain manifolds that arose from an analysis of toral action on nonorientable 4 manifolds

nonorientable 4 manifolds with fundamental group of order 2 in this paper we classify nonorientable topological closed 4 manifolds with fundamental group z 2 up to homeomorphism our results give a complete list of such manifolds and show how they can be distinguished by explicit invariants including characteristic numbers and the ti invariant associated to a normal pinc structure by the spectral asymmetry of a certain dirac operator in contrast to the oriented case there exist homotopy equivalent nonorientable topological 4 manifolds which are stably homeomorphic after connected sum with s2 x s2 but not homeomorphic

higher order infinitesimal robustness using the von mises expansion we study the higher order infinitesimal robustness of a general m functional and characterize its second order properties we show that second order robustness is equivalent to the boundedness of both the estimator s estimating function and its derivative with respect to the parameter it implies at the same time i variance robustness and ii robustness of higher order saddlepoint approximations to the estimator s finite sample density the proposed construction of second order robust m estimators is fairly general and potentially useful in a variety of relevant settings besides the theoretical contributions we discuss the main computational issues and provide an algorithm for the implementation of second order robust m estimators finally we illustrate our theory by monte carlo simulation and in a real data estimation of the maximal losses of nikkei 225 index returns our findings indicate that second order robust estimators can improve on other widely applied robus

to smooth or not to smooth bias and efficiency in fmri time series analysis this paper concerns temporal filtering in fmri time series analysis whitening serially correlated data is the most efficient approach to parameter estimation however if there is a discrepancy between the assumed and the actual correlations whitening can render the analysis exquisitely sensitive to bias when estimating the standard error of the ensuing parameter estimates this bias although not expressed in terms of the estimated responses has profound effects on any statistic used for inference the special constraints of fmri analysis ensure that there will always be a misspecification of the assumed serial correlations one resolution of this problem is to filter the data to minimize bias while maintaining a reasonable degree of efficiency in this paper we present expressions for efficiency of parameter estimation and bias in estimating standard error in terms of assumed and actual correlation structures in the context of the general linear model we show that i whitening strategies can result in profound bias and are therefore probably precluded in parametric fmri data analyses ii band pass filtering and implicitly smoothing has an important role in protecting against inferential bias

model selection in the presence of incidental parameters this paper considers model selection in panels where incidental parameters are present primary interest centers on selecting a model that best approximates the underlying structure involving parameters that are common within the panel it is well known that conventional model selection procedures are often inconsistent in panel models and this can be so even without nuisance parameters modifications are then needed to achieve consistency new model selection information criteria are developed here that use either the kullback leibler information criterion based on the profile likelihood or the bayes factor based on the integrated likelihood with a bias reducing prior these model selection criteria impose heavier penalties than those associated with standard information criteria such as aic and bic the additional penalty which is data dependent properly reflects the model complexity arising from the presence of incidental parameters a particular example is studied in detail involving lag order selection in dynamic panel models with fixed effects the new criteria are shown to control for over under selection probabilities in these models and lead to consistent order selection criteria

on the bootstrap and confidence intervals we derive an explicit formula for the first term in an unconditional edgeworth type expansion of coverage probability for the nonparametric bootstrap technique applied to a very broad class of studentized statistics the class includes sample mean k sample mean sample correlation coefficient maximum likelihood estimators expressible as functions of vector means etc we suggest that the bootstrap is really an empiric one term edgeworth inversion with the bootstrap simulations implicitly estimating the first term in an edgeworth expansion this view of the bootstrap is reinforced by our discussion of the iterated bootstrap which inverts an edgeworth expansion to arbitrary order by simulating simulations

generalised information criteria in model selection summary the problem of evaluating the goodness of statistical models is investigated from an information theoretic point of view information criteria are proposed for evaluating models constructed by various estimation procedures when the specified family of probability distributions does not contain the distribution generating the data the proposed criteria are applied to the evaluation of models estimated by maximum likelihood robust penalised likelihood bayes procedures etc we also discuss the use of the bootstrap in model evaluation problems and present a variance reduction technique in the bootstrap simulation

prepivoting to reduce level error of confidence sets summary approximate confidence sets for a parameter 0 may be obtained by referring a function of 0 and of the sample to an estimated quantile of that function s sampling distribution we call this function the root of the confidence set either asymptotic theory or bootstrap methods can be used to estimate the desired quantile when the root is not a pivot in the sense of classical statistics the actual level of the approximate confidence set may differ substantially from the intended level prepivoting is the transformation of a confidence set root by its estimated bootstrap cumulative distribution function prepivoting can be iterated bootstrap confidence sets generated from a root prepivoted one or more times have smaller error in level than do confidence sets based on the original root the first prepivoting is nearly equivalent to studentizing when that operation is appropriate further iterations of prepivoting make higher order corrections automatically

more accurate confidence intervals in exponential families summary fisher s theory of maximum likelihood estimation routinely provides approximate confidence intervals for a parameter of interest 0 the standard intervals 0 za5 s where 0 is the maximum likelihood estimator 5 is an estimate of standard error based on differentiation of the log likelihood function and za is a normal percentile point recent work has produced systems of better approximate confidence intervals which look more like exact intervals when exact intervals exist and in general have coverage probabilities an order of magnitude more accurate than the standard intervals this paper develops an efficient and dependable algorithm for calculating highly accurate approximate intervals on a routine basis for parameters 0 defined in the framework of a multiparameter exponential family the better intervals require only a few times as much computational effort as the standard intervals a variety of numerical and theoretical arguments are used to show that the algorithm works well and that the improvement over the standard intervals can be striking in realistic situations

toral actions on 4 manifolds and their classifications the existence of a cross section is proved for some nonorientable 4 manifolds with a t 2 action two 4 manifolds with a t 2 action which have the same previously known invariants are constructed by using a new homotopy invariant they are proved to be homotopy inequivalent finally a stable diffeomorphism theorem is proved

adaptive stochastic approximation by the simultaneous perturbation method stochastic approximation sa has long been applied for problems of minimizing loss functions or root finding with noisy input information as with all stochastic search algorithms there are adjustable algorithm coefficients that must be specified and that can have a profound effect on algorithm performance it is known that choosing these coefficients according to an sa analog of the deterministic newton raphson algorithm provides an optimal or near optimal form of the algorithm however directly determining the required hessian matrix or jacobian matrix for root finding to achieve this algorithm form has often been difficult or impossible in practice the paper presents a general adaptive sa algorithm that is based on a simple method for estimating the hessian matrix while concurrently estimating the primary parameters of interest the approach applies in both the gradient free optimization kiefer wolfowitz and root finding stochastic gradient based robbins monro settings and is based on the simultaneous perturbation sp idea introduced previously the algorithm requires only a small number of loss function or gradient measurements per iteration independent of the problem dimension to adaptively estimate the hessian and parameters of primary interest aside from introducing the adaptive sp approach the paper presents practical implementation guidance asymptotic theory and a nontrivial numerical evaluation also included is a discussion and numerical analysis comparing the adaptive sp approach with the iterate averaging approach to accelerated sa

multivariate stochastic approximation using a simultaneous perturbation gradient approximation the problem of finding a root of the multivariate gradient equation that arises in function minimization is considered when only noisy measurements of the function are available a stochastic approximation sa algorithm for the general kiefer wolfowitz type is appropriate for estimating the root the paper presents an sa algorithm that is based on a simultaneous perturbation gradient approximation instead of the standard finite difference approximation of keifer wolfowitz type procedures theory and numerical experience indicate that the algorithm can be significantly more efficient than the standard algorithms in large dimensional problems

feedback and weighting mechanisms for improving jacobian estimates in the adaptive simultaneous perturbation algorithm it is known that a stochastic approximation sa analogue of the deterministic newton raphson algorithm provides an asymptotically optimal or near optimal form of stochastic search however directly determining the required jacobian matrix or hessian matrix for optimization has often been difficult or impossible in practice this paper presents a general adaptive sa algorithm that is based on a simple method for estimating the jacobian matrix while concurrently estimating the primary parameters of interest relative to prior methods for adaptively estimating the jacobian matrix the paper introduces two enhancements that generally improve the quality of the estimates for underlying jacobian hessian matrices thereby improving the quality of the estimates for the primary parameters of interest the first enhancement rests on a feedback process that uses previous jacobian estimates to reduce the error in the current estimate the second enhancement is based on an optimal weighting of per iteration jacobian estimates from the use of simultaneous perturbations the algorithm requires only a small number of loss function or gradient measurements per iteration independent of the problem dimension to adaptively estimate the jacobian matrix and parameters of primary interest

the bootstrap to smooth or not to smooth summary the bootstrap and smoothed bootstrap are considered as alternative methods of estimating properties of unknown distributions such as the sampling error of parameter estimates criteria are developed for determining whether it is advantageous to use the smoothed bootstrap rather than the standard bootstrap key steps in the argument leading to these criteria include the study of the estimation of linear functionals of distributions and the approximation of general functionals by linear functionals consideration of an example the estimation of the standard error in the variance stabilized sample correlation coefficient elucidates previously published simulation results and also illustrates the use of computer algebraic manipulation as a useful technique in asymptotic statistics finally the various approximations used are vindicated by a simulation study

on the bootstrap and likelihood based confidence regions summary we describe a method for constructing likelihood based confidence regions for a vector parameter using the bootstrap and nonparametric density estimation the technique is illustrated by application to a numerical example and its theoretical properties are elucidated it is argued that likelihood based regions should not be approximated by ellipses if we are to have any hope of capturing first order departures from normality bootstrap algorithms for constructing simultaneous confidence intervals for the components of a vector parameter are also presented advantages of the percentile t method over the ordinary percentile method are demonstrated in a multivariate setting