efficient high dimensional importance sampling the paper describes a simple generic and yet highly accurate efficient importance sampling eis monte carlo mc procedure for the evaluation of high dimensional numerical integrals eis is based upon a sequence of auxiliary weighted regressions which actually are linear under appropriate conditions it can be used to evaluate likelihood functions and byproducts thereof such as ml estimators for models which depend upon unobservable variables a dynamic stochastic volatility model and a logit panel data model with unobserved heterogeneity random effects in both dimensions are used to provide illustrations of eis high numerical accuracy even under small number of mc draws mc simulations are used to characterize the finite sample numerical and statistical properties of eis based ml estimators

displacement based finite elements with nodal integration for reissner mindlin plates an assumed strain finite element technique is presented for shear deformable reissner mindlin plates the weighted residual method reminiscent of the strain displacement functional is used to enforce weakly the balance equation with the natural boundary condition and separately the kinematic equation the strain displacement relationship the a priori satisfaction of the kinematic weighted residual serves as a condition from which strain displacement operators are derived via nodal integration for linear triangles and quadrilaterals and also for quadratic triangles the degrees of freedom are only the primitive variables transverse displacements and rotations at the nodes a straightforward constraint count can partially explain the insensitivity of the resulting finite element models to locking in the thin plate limit we also construct an energy based argument for the ability of the present formulation to converge to the correct deflections in the limit of the thickness approaching zero examples are used to illustrate the performance with particular attention to the sensitivity to element shape and shear locking copyright 2009 john wiley sons ltd

using sampling and simplex derivatives in pattern search methods in this paper we introduce ways of making a pattern search more efficient by reusing previous evaluations of the objective function based on the computation of simplex derivatives e g simplex gradients at each iteration one can attempt to compute an accurate simplex gradient by identifying a sampling set of previously evaluated points with good geometrical properties this can be done using only past successful iterates or by considering all past function evaluations the simplex gradient can then be used to reorder the evaluations of the objective function associated with the directions used in the poll step or to update the mesh size parameter according to a sufficient decrease criterion neither of which requires new function evaluations we present these procedures in detail and apply them to a set of problems from the cuter collection numerical results show that these procedures can enhance significantly the practical performance of pattern search methods

benchmarking optimization software with performance profiles we propose performance profiles distribution functions for a performance metric as a tool for benchmarking and comparing optimization software we show that performance profiles combine the best features of other tools for performance evaluation

numerical analysis of mathcal v mathcal u decomposition mathcal u gradient and mathcal u hessian approximations advances in bundle methods for nonsmooth optimization have lead to the development of mathcal v mathcal u decompositions the mathcal u gradient and the mathcal u hessian these variational analysis constructs have proven extremely valuable and lead to the development of the superlinearly convergent mathcal v mathcal u algorithm for nonsmooth optimization in this paper we examine these constructs from the viewpoint of derivative free optimization we show that given a finite max function f x max i 0 1 ldots m f i x and a black box which returns function values for each f i it is possible to construct approximations of the mathcal v mathcal u decompositions mathcal u gradient and mathcal u hessian the approximations do not require excessive black box calls and the accuracy of the approximations is directly related to the accuracy of the approximate gradient and hessians for each f i

exact solution of a 1d asymmetric exclusion model using a matrix formulation several recent works have shown that the one dimensional fully asymmetric exclusion model which describes a system of particles hopping in a preferred direction with hard core interactions can be solved exactly in the case of open boundaries here the authors present a new approach based on representing the weights of each configuration in the steady state as a product of noncommuting matrices with this approach the whole solution of the problem is reduced to finding two matrices and two vectors which satisfy very simple algebraic rules they obtain several explicit forms for these non commuting matrices which are in the general case infinite dimensional their approach allows exact expressions to be derived for the current and density profiles finally they discuss briefly two possible generalizations of their results the problem of partially asymmetric exclusion and the case of a mixture of two kinds of particles

weighted average importance sampling and defensive mixture distributions importance sampling uses observations from one distribution to estimate for another distribution by weighting the observations including the target distribution as one component of a mixture distribution bounds the weights and makes importance sampling more reliable the usual importance sampling estimate is a weighted average with weights that do not sum to 1 we discuss simple normalization and other more efficient normalization methods these innovations make importance sampling useful in a wider variety of problems we demonstrate with a case study of oil inventory reliability at a large utility

phase transitions in an exactly soluble one dimensional exclusion process we consider an exclusion process with particles injected with rate at the origin and removed with rate at the right boundary of a one dimensional chain of sites the particles are allowed to hop onto unoccupied sites to the right only for the special case of 1 the model was solved previously by derridaet al here we extend the solution to general the phase diagram obtained from our exact solution differs from the one predicted by the mean field approximation

recent progress in unconstrained nonlinear optimization without derivatives we present an introduction to a new class of derivative free methods for unconstrained optimization we start by discussing the motivation for such methods and why they are in high demand by practitioners we then review the past developments in this field before introducing the features that characterize the newer algorithms in the context of a trust region framework we focus on techniques that ensure a suitable geometric quality of the considered models we then outline the class of algorithms based on these techniques as well as their respective merits we finally conclude the paper with a discussion of open questions and perspectives

derivative free optimization of expensive functions with computational error using weighted regression we propose a derivative free algorithm for optimizing computationally expensive functions with computational error the algorithm is based on the trust region regression method by conn scheinberg and vicente a r conn k scheinberg and l n vicente ima j numer anal 28 2008 pp 721 748 but uses weighted regression to obtain more accurate model functions at each trust region iteration a heuristic weighting scheme is proposed that simultaneously handles i differing levels of uncertainty in function evaluations and ii errors induced by poor model fidelity we also extend the theory of lambda poisedness and strong lambda poisedness to weighted regression we report computational results comparing interpolation regression and weighted regression methods on a collection of benchmark problems weighted regression appears to outperform interpolation and regression models on nondifferentiable functions and functions with deterministic noise

global convergence of trust region algorithms for convex constrained minimization without derivatives in this work we propose a trust region algorithm for the problem of minimizing a function within a convex closed domain we assume that the objective function is differentiable but no derivatives are available the algorithm has a very simple structure and allows a great deal of freedom in the choice of the models under reasonable assumptions for derivative free schemes we prove global convergence for the algorithm that is to say that all accumulation points of the sequence generated by the algorithm are stationary

feasible side global convergence in experimental optimization we propose a canonical form of the experimental optimization problem and review the state of the art methods to solve it as guarantees of global convergence to an optimal point via only feasible iterates are absent in these methods we present a set of sufficient conditions to enforce this behavior

totally asymmetric exclusion processes with particles of arbitrary size the steady state currents and densities of a one dimensional totally asymmetric exclusion process tasep with particles that occlude an integer number d of lattice sites are computed using various mean field approximations and monte carlo simulations taseps featuring particles of arbitrary size are relevant for modelling systems such as mrna translation vesicle locomotion along microtubules and protein sliding along dna we conjecture that the nonequilibrium steady state properties separate into low density high density and maximal current phases similar to those of the standard d 1 tasep a simple mean field approximation for steady state particle currents and densities is found to be inaccurate however we find local equilibrium particle distributions derived from a discrete tonks gas partition function yield apparently exact currents within the maximal current phase for the boundary limited phases the equilibrium tonks gas distribution cannot be used to predict currents phase boundaries or the order of the phase transitions however we employ a refined mean field approach to find apparently exact expressions for the steady state currents boundary densities and phase diagrams of the d 1 tasep extensive monte carlo simulations are performed to support our analytic mean field results

on multivariate lagrange interpolation lagrange interpolation by polynomials in several variables is studied through a finite difference approach we establish an interpolation formula analogous to that of newton and a remainder formula both of them in terms of finite differences we prove that the finite difference admits an integral representation involving simplex spline functions in particular this provides a remainder formula for lagrange interpolation of degree n of a function f which is a sum of integrals of certain n 1 st directional derivatives of f multiplied by simplex spline functions we also provide two algorithms for the computation of lagrange interpolants which use only addition scalar multiplication and point evaluation of polynomials

on trust region methods for unconstrained minimization without derivatives we consider some algorithms for unconstrained minimization without derivatives that form linear or quadratic models by interpolation to values of the objective function then a new vector of variables is calculated by minimizing the current model within a trust region techniques are described for adjusting the trust region radius and for choosing positions of the interpolation points that maintain not only nonsingularity of the interpolation equations but also the adequacy of the model particular attention is given to quadratic models with diagonal second derivative matrices because numerical experiments show that they are often more efficient than full quadratic models for general objective functions finally some recent research on the updating of full quadratic models is described briefly using fewer interpolation equations than before the resultant freedom is taken up by minimizing the frobenius norm of the change to the second derivative matrix of the model a preliminary version of this method provides some very promising numerical results

frames and grids in unconstrained and linearly constrained optimization a nonsmooth approach this paper describes a class of frame based direct search methods for unconstrained and linearly constrained optimization a template is described and analyzed using clarke s break nonsmooth calculus this provides a unified and simple approach to earlier results for grid and frame based methods and also provides partial convergence results when the objective function is not smooth undefined in some places or both the template also covers many new methods which combine elements of previous ideas using frames and grids these new methods include grid based simple descent algorithms which allow moving to points off the grid at every iteration and can automatically control the grid size provided function values are available the concept of a grid is also generalized to that of an admissible set which allows sets for example with circular symmetries the method is applied to linearly constrained problems using a simple barrier approach

adaptive importance sampling and control variates abstract we construct and investigate an adaptive variance reduction framework in which both importance sampling and control variates are employed the three lines monte carlo averaging and two variance reduction parameter search lines run in parallel on a common sequence of uniform random vectors on the unit hypercube given that these two variance reduction techniques are effective often in a complementary way their combined application is well expected to widen the applicability of adaptive variance reduction we derive convergence rates of the theoretical estimator variance towards its minimum as a fixed computing budget increases when stochastic approximation runs with optimal constant learning rates we derive sufficient conditions for the proposed algorithm to attain the minimal estimator variance in the limit by stochastic approximation with decreasing learning rates or by sample average approximation when computing budget is unlimitedly available numerical results support our theoretical findings and illustrate the effectiveness of the proposed framework

exact results for the asymmetric simple exclusion process with a blockage we present new results for the current as a function of transmission rate in the one dimensional totally asymmetric simple exclusion process tasep with a blockage that lowers the jump rate at one site from one tor 1 exact finitevolume results serve to bound the allowed values for the current in the infinite system this proves the existence of a nonequilibrium phase transition corresponding to an immiscibility gap in the allowed values of the asymptotic densities which the infinite system can have in a stationary state a series expansion inr derived from the finite systems is proven to be asymptotic for all sufficiently large systems pade approximants based on this series which make specific assumptions about the nature of the singularity atr 1 match numerical data for the infinite system to 1 part in 104

on the convergence of pattern search algorithms we introduce an abstract definition of pattern search methods for solving nonlinear unconstrained optimization problems our definition unifies an important collection of optimization methods that neither compute nor explicitly approximate derivatives we exploit our characterization of pattern search methods to establish a global convergence theory that does not enforce a notion of sufficient decrease our analysis is possible because the iterates of a pattern search method lie on a scaled translated integer lattice this allows us to relax the classical requirements on the acceptance of the step at the expense of stronger conditions on the form of the step and still guarantee global convergence

provably convergent multifidelity optimization algorithm not requiring high fidelity derivatives functions does not alter the con vergence proof of the optimization algorithm and is shown to be robust to poor low delity information the algorithm is compared with an unconstrained single delity quasi newton algorithm and two

on a likelihood approach for monte carlo integration the use of estimating equations has been a common approach for constructing monte carlo estimators recently kong et al proposed a formulation of monte carlo integration as a statistical model making explicit what information is ignored and what is retained about the baseline measure from simulated data the baseline measure is estimated by maximum likelihood and then integrals of interest are estimated by substituting the estimated measure for two different situations in which independent observations are simulated from multiple distributions we show that this likelihood approach achieves the lowest asymptotic variance possible by using estimating equations in the first situation the normalizing constants of the design distributions are estimated and meng and wong s bridge sampling estimating equation is considered in the second situation the values of the normalizing constants are known thereby imposing linear constraints on the baseline measure estimating equations including hesterberg s

non abelian symmetries of stochastic processes derivation of correlation functions for random vertex models and disordered interacting particle systems we consider systems of particles hopping stochastically on d dimensional lattices with space dependent probabilities we map the master equation onto an evolution equation in a fock space where the dynamics are given by a quantum hamiltonian continuous time or a transfer matrix discrete time using non abelian symmetries of these operators we derive duality relations expressing the time evolution of a given initial configuration in terms of correlation functions of simpler dual processes particularly simple results are obtained for the time evolution of the density profile as a special case we show that for any su 2 symmetric system the two point and three point density correlation functions in the n particle steady state can be computed from the probability distribution of a single particle moving in the same environment we apply our results to various models among them partial exclusion a simple diffusion reaction system and the two dimensional six vertex model with space dependent vertex weights for a random distribution of the vertex weights one obtains a version of the random barrier model describing diffusion of particles in disordered media we derive exact expressions for the averaged two point density correlation functions in the presence of weak correlated disorder

global convergence of general derivative free trust region algorithms to first and second order critical points in this paper we prove global convergence for first and second order stationary points of a class of derivative free trust region methods for unconstrained optimization these methods are based on the sequential minimization of quadratic or linear models built from evaluating the objective function at sample sets the derivative free models are required to satisfy taylor type bounds but apart from that the analysis is independent of the sampling techniques a number of new issues are addressed including global convergence when acceptance of iterates is based on simple decrease of the objective function trust region radius maintenance at the criticality step and global convergence for second order critical points

linked interpolation for reissner mindlin plate elements part ii a simple triangle the formulation and shape functions given in part i are extended to develop a simple plate bending triangle with good performance in both thin and thick situations indeed its performance is better than that of other nine dof elements and its computer implementation simpler when used with selective reduced integration the element produces identical results as that of xu 1 2 to save space details given in part i are not repeated and some results are also presented in the figures of part i

information theoretic shape matching in this paper we describe two related algorithms that provide both rigid and non rigid point set registration with different computational complexity and accuracy the first algorithm utilizes a nonlinear similarity measure known as correntropy the measure combines second and high order moments in its decision statistic showing improvements especially in the presence of impulsive noise the algorithm assumes that the correspondence between the point sets is known which is determined with the surprise metric the second algorithm mitigates the need to establish a correspondence by representing the point sets as probability density functions pdf the registration problem is then treated as a distribution alignment the method utilizes the cauchy schwarz divergence to measure the similarity distance between the point sets and recover the spatial transformation function needed to register them both algorithms utilize information theoretic descriptors however correntropy works at the realizations level whereas cauchy schwarz divergence works at the pdf level this allows correntropy to be less computationally expensive and for correct correspondence more accurate the two algorithms are robust against noise and outliers and perform well under varying levels of distortion they outperform several well known and state of the art methods for point set registration

patch averaged assumed strain finite elements for stress analysis summary a finite element model for linear elastic small deformation problems is presented the formulation is based on a weighted residual that requires a priori the satisfaction of the kinematic equation in this approach an averaged strain displacement matrix is constructed for each node of the mesh by defining an appropriate patch of elements yielding a smooth representation of strain and stress fields connections with traditional and similar procedure are explored linear quadrilateral four node and linear hexahedral eight node elements are derived various numerical tests show the accuracy and convergence properties of the proposed elements in comparison with extant finite elements and analytic solutions specific examples are also included to illustrate the ability to resist numerical locking in the incompressible limit and insensitive response in the presence of shape distortion furthermore the numerical inf sup test is applied to a selection of problems to show the stability of the present formulation copyright 2012 john wiley sons ltd

a method to compute statistics of large noise induced perturbations of nonlinear schrodinger solitons we describe in detail the application of importance sampling to numerical simulations of large noise induced perturbations in soliton based optical transmission systems governed by the nonlinear schrodinger equation the method allows one to concentrate the samples in monte carlo simulations around those noise realizations that are most likely to produce the large pulse deformations connected with errors and it yields computational speedups of several orders of magnitude over standard monte carlo simulations we demonstrate the method by using it to calculate the probability density functions associated with pulse amplitude frequency timing and phase fluctuations in a prototypical soliton based communication system

combining multiple surrogate models to accelerate failure probability estimation with expensive high fidelity models in failure probability estimation importance sampling constructs a biasing distribution that targets the failure event such that a small number of model evaluations is sufficient to achieve a monte carlo estimate of the failure probability with an acceptable accuracy however the construction of the biasing distribution often requires a large number of model evaluations which can become computationally expensive we present a mixed multifidelity importance sampling mmfis approach that leverages computationally cheap but erroneous surrogate models for the construction of the biasing distribution and that uses the original high fidelity model to guarantee unbiased estimates of the failure probability the key property of our mmfis estimator is that it can leverage multiple surrogate models for the construction of the biasing distribution instead of a single surrogate model alone we show that our mmfis estimator has a mean squared error that is up to a constant lower than the mean squared errors of the corresponding estimators that uses any of the given surrogate models alone even in settings where no information about the approximation qualities of the surrogate models is available in particular our mmfis approach avoids the problem of selecting the surrogate model that leads to the estimator with the lowest mean squared error which is challenging if the approximation quality of the surrogate models is unknown we demonstrate our mmfis approach on numerical examples where we achieve orders of magnitude speedups compared to using the high fidelity model only

global convergence of a class of trust region algorithms for optimization with simple bounds this paper extends the known excellent global convergence properties of trust region algorithms for unconstrained optimization to the case where bounds on the variables are present weak conditions on the accuracy of the hessian approximations are considered it is also shown that when the strict complementarily condition holds the proposed algorithms reduce to an unconstrained calculation after finitely many iterations allowing a fast asymptotic rate of convergence

two way traffic flow exactly solvable model of traffic jam we study completely asymmetric two channel exclusion processes in one dimension it describes a two way traffic flow with cars moving in opposite directions the interchannel interaction makes cars slow down in the vicinity of approaching cars in the other lane particularly we consider in detail the system with a finite density of cars on one lane and a single car on the other when the interchannel interaction reaches a critical value a traffic jam occurs which turns out to be of first order phase transition we derive exact expressions for the average velocities the current the density profile and the k point density correlation functions we also obtain the exact probability of two cars being in one lane of distance r apart provided there is a finite density of cars on the other lane and show that the two cars form a weakly bound state in the jammed phase

orbit optimization by radial basis function interpolation in trust regions we present a new derivative free algorithm orbit for unconstrained local optimization of computationally expensive functions a trust region framework using interpolating radial basis function rbf models is employed the rbf models considered often allow orbit to interpolate nonlinear functions using fewer function evaluations than the polynomial models considered by present techniques approximation guarantees are obtained by ensuring that a subset of the interpolation points is sufficiently poised for linear interpolation the rbf property of conditional positive definiteness yields a natural method for adding additional points we present numerical results on test problems to motivate the use of orbit when only a relatively small number of expensive function evaluations are available results on two very different application problems calibration of a watershed model and optimization of a pde based bioremediation plan are also encouraging and support orbit s effectiveness on blackbox functions for which no special mathematical structure is known or available

geometry of interpolation sets in derivative free optimization we consider derivative free methods based on sampling approaches for nonlinear optimization problems where derivatives of the objective function are not available and cannot be directly approximated we show how the bounds on the error between an interpolating polynomial and the true function can be used in the convergence theory of derivative free sampling methods these bounds involve a constant that reflects the quality of the interpolation set the main task of such a derivative free algorithm is to maintain an interpolation sampling set so that this constant remains small and at least uniformly bounded this constant is often described through the basis of lagrange polynomials associated with the interpolation set we provide an alternative more intuitive definition for this concept and show how this constant is related to the condition number of a certain matrix this relation enables us to provide a range of algorithms whilst maintaining the interpolation set so that this condition number or the geometry constant remain uniformly bounded we also derive bounds on the error between the model and the function and between their derivatives directly in terms of this condition number and of this geometry constant

a smoothed finite element method for plate analysis a quadrilateral element with smoothed curvatures for mindlin reissner plates is proposed the curvature at each point is obtained by a non local approximation via a smoothing function the bending stiffness matrix is calculated by a boundary integral along the boundaries of the smoothing elements smoothing cells numerical results show that the proposed element is robust computational inexpensive and simultaneously very accurate and free of locking even for very thin plates the most promising feature of our elements is their insensitivity to mesh distortion

efficient sequential monte carlo with multiple proposals and control variates sequential monte carlo is a useful simulation based method for online filtering of state space models for certain complex state space models a single proposal distribution is usually not satisfactory and using multiple proposal distributions is a general approach to address various aspects of the filtering problem this article proposes an efficient method of using multiple proposals in combination with control variates the likelihood approach of tan 2004 is used in both resampling and estimation the new algorithm is shown to be asymptotically more efficient than the direct use of multiple proposals and control variates the guidance for selecting multiple proposals and control variates is also given numerical examples are used to demonstrate that the new algorithm can significantly improve over the bootstrap filter and auxiliary particle filter

a direct search algorithm for optimization with noisy function evaluations we consider the unconstrained optimization of a function when each function evaluation is subject to a random noise we assume that there is some control over the variance of the noise term in the sense that additional computational effort will reduce the amount of noise this situation may occur when function evaluations involve simulation or the approximate solution of a numerical problem it also occurs in an experimental setting when averaging repeated observations at the same point can lead to a better estimate of the underlying function value we describe a new direct search algorithm for this type of problem we prove convergence of the new algorithm when the noise is controlled so that the standard deviation of the noise approaches zero faster than the step size we also report some numerical results on the performance of the new algorithm

global convergence of radial basis function trust region derivative free algorithms we analyze globally convergent derivative free trust region algorithms relying on radial basis function interpolation models our results extend the recent work of conn scheinberg and vicente siam j optim 20 2009 pp 387 415 to fully linear models that have a nonlinear term we characterize the types of radial basis functions that fit in our analysis and thus show global convergence to first order critical points for the orbit algorithm of wild regis and shoemaker siam j sci comput 30 2008 pp 3197 3219 using orbit we present numerical results for different types of radial basis functions on a series of test problems we also demonstrate the use of orbit in finding local minima on a computationally expensive environmental engineering problem

matrix representation of the stationary measure for the multispecies tasep in this work we construct the stationary measure of the n species totally asymmetric simple exclusion process in a matrix product formulation we make the connection between the matrix product formulation and the queueing theory picture of ferrari and martin in particular in the standard representation the matrices act on the space of queue lengths for n 2 the matrices in fact become tensor products of elements of quadratic algebras this enables us to give a purely algebraic proof of the stationary measure which we present for n 3

disordered exclusion process revisited some exact results in the low current regime we study the steady state of the totally asymmetric simple exclusion process with inhomogeneous hopping rates associated with sites site wise disorder using the fact that the non normalized steady state weights which solve the master equation are polynomials in all the hopping rates we propose a general method for calculating their first few lowest coefficients exactly in the case of binary disorder where all slow sites share the same hopping rate r 1 we apply this method to calculate the steady state current up to the quadratic term in r for some particular disorder configurations for the most general non binary disorder we show that in the low current regime the current is determined solely by the current minimizing subset of equal hopping rates provided all other slow rates are much larger our approach can be readily applied to any other driven diffusive system with unidirectional hopping if one can identify a hopping rate such that the current vanishes when this rate is set to zero

optimization by direct search new perspectives on some classical and modern methods direct search methods are best known as unconstrained optimization techniques that do not explicitly use derivatives direct search methods were formally proposed and widely applied in the 1960s but fell out of favor with the mathematical optimization community by the early 1970s because they lacked coherent mathematical analysis nonetheless users remained loyal to these methods most of which were easy to program some of which were reliable in the past fifteen years these methods have seen a revival due in part to the appearance of mathematical analysis as well as to interest in parallel and distributed com puting this review begins by briefly summarizing the history of direct search methods and considering the special properties of problems for which they are well suited our focus then turns to a broad class of methods for which we provide a unifying framework that lends itself to a variety of convergence results the underlying principles allow general ization to handle bound constraints and linear constraints we also discuss extensions to problems with nonlinear constraints

a lagrangian nodal integration method for free surface fluid flows abstract we present a lagrangian nodal integration method for the simulation of newtonian and non newtonian free surface fluid flows the proposed nodal lagrangian method uses a finite element mesh to discretize the computational domain and to define the linear shape functions for the unknown nodal variables as in the standard particle finite element method pfem in this approach however the integrals are performed over nodal patches and not over elements and strains stresses are defined at nodes and not at gauss points this allows to limit the drawbacks associated with the remeshing and leads to a more accurate stress computation than in the classical elemental pfem several numerical tests in 2d and in 3d are presented to validate the proposed nodal pfem in all cases the method has shown a very good agreement with analytical solutions and with experimental and numerical results from the literature a thorough comparison between nodal and elemental pfems is also presented focusing on crucial issues such as solution accuracy convergence mass conservation and sensitivity to mesh distortion

inference in molecular population genetics full likelihood based inference for modern population genetics data presents methodological and computational challenges the problem is of considerable practical importance and has attracted recent attention with the development of algorithms based on importance sampling is and markov chain monte carlo mcmc sampling here we introduce a new is algorithm the optimal proposal distribution for these problems can be characterized and we exploit a detailed analysis of genealogical processes to develop a practicable approximation to it we compare the new method with existing algorithms on a variety of genetic examples our approach substantially outperforms existing is algorithms with efficiency typically improved by several orders of magnitude the new method also compares favourably with existing mcmc methods in some problems and less favourably in others suggesting that both is and mcmc methods have a continuing role to play in this area we offer insights into the relative advantages of each approach and we discuss diagnostics in the is framework

on the adaptive selection of the parameter in stabilized finite element approximations a systematic approach is developed for the selection of the stabilization parameter for stabilized finite element approximation of the stokes problem whereby the parameter is chosen to minimize a computable upper bound for the error in the approximation the approach is applied in the context of both a single fixed mesh and an adaptive mesh refinement procedure the optimization is carried out by a derivative free optimization algorithm and is based on minimizing a new fully computable error estimator numerical results are presented illustrating the theory and the performance of the estimator together with the optimization algorithm

exact solution of an exclusion process with three classes of particles and vacancies we present an exact solution for an asymmetric exclusion process on a ring with three classes of particles and vacancies using a matrix product ansatz we find exact expressions for the weights of the configurations in the stationary state the solution involves tensor products of quadratic algebras

reclaiming quasimonte carlo efficiency in portfolio value at risk simulation through fourier transform quasimonte carlo methods overcome the problem of sample clustering in regular monte carlo simulation and have been shown to improve simulation efficiency in the derivatives pricing literature when the price is expressed as a multidimensional integration and the integrand is suitably smooth for portfolio value at risk var problems the distribution of portfolio value change is based on the expectation of an indicator function hence the integrand is discontinuous the purpose of this paper is to smooth the expectation estimation of an indicator function via fourier transform so that the faster convergence rate of quasimonte carlo methods can be reclaimed theoretically under fairly mild assumptions the simulation of portfolio value at risk is fast and accurate numerical examples elucidate the advantage of the proposed approach over regular monte carlo and quasimonte carlo methods

efficient importance sampling for events of moderate deviations with applications summary we propose a method for finding the alternative distribution in importance sampling the alternative distribution is optimal in the sense that the asymptotic variance is mini mised for estimating tail probabilities of asymptotically normal statistics our contribution to importance sampling is three fold to begin with we obtain an explicit expression for the mean of the optimal alternative distribution and the expression motivates a recursive approximation algorithm secondly a new multi dimensional exponential tilting formula is presented lastly a conservative estimator of the variance is given to facilitate a quick comparison among different stratified sampling schemes in conjunction with importance sampling several numerical examples illustrating the efficacy of the proposed method are also included these results indicate that the proposed method is considerably more efficient than the method based on large deviations theory and the efficiency gain is more significant in higher dimensions

the superconvergent patch recovery and a posteriori error estimates part 1 the recovery technique this is the first of two papers concerning superconvergent recovery techniques and a posteriori error estimation in this paper a general recovery technique is developed for determining the derivatives stresses of the finite element solutions at nodes the implementation of the recovery technique is simple and cost effective the technique has been tested for a group of widely used linear quadratic and cubic elements for both one and two dimensional problems numerical experiments demonstrate that the recovered nodal values of the derivatives with linear and cubic elements are superconvergent one order higher accuracy is achieved by the procedure with linear and cubic elements but two order higher accuracy is achieved for the derivatives with quadratic elements in particular an o h4 convergence of the nodal values of the derivatives for a quadratic triangular element is reported for the first time the performance of the proposed technique is compared with the widely used smoothing procedure of global l2 projection and other methods it is found that the derivatives recovered at interelement nodes by using l2 projection are also superconvergent for linear elements but not for quadratic elements numerical experiments on the convergence of the recovered solutions in the energy norm are also presented higher rates of convergence are again observed the results presented in this part of the paper indicate clearly that a new powerful and economical process is now available which should supersede the currently used post processing procedures applied in most codes

multifidelity probability estimation via fusion of estimators abstract this paper develops a multifidelity method that enables estimation of failure probabilities for expensive to evaluate models via information fusion and importance sampling the presented general fusion method combines multiple probability estimators with the goal of variance reduction we use low fidelity models to derive biasing densities for importance sampling and then fuse the importance sampling estimators such that the fused multifidelity estimator is unbiased and has mean squared error lower than or equal to that of any of the importance sampling estimators alone by fusing all available estimators the method circumvents the challenging problem of selecting the best biasing density and using only that density for sampling a rigorous analysis shows that the fused estimator is optimal in the sense that it has minimal variance amongst all possible combinations of the estimators the asymptotic behavior of the proposed method is demonstrated on a convection diffusion reaction partial differential equation model for which 105 samples can be afforded to illustrate the proposed method at scale we consider a model of a free plane jet and quantify how uncertainties at the flow inlet propagate to a quantity of interest related to turbulent mixing compared to an importance sampling estimator that uses the high fidelity model alone our multifidelity estimator reduces the required cpu time by 65 while achieving a similar coefficient of variation

bayesian monte carlo testing with one dimensional measures of evidence abstract bayesian hypothesis testing procedures are constructed by means of test statistics which are functions of the posterior distribution usually the whole sample vector is selected to form the sufficient empirical part of the posterior distribution but in certain problems one may prefer to use well established one dimensional sufficient statistics in place of the sample vector this paper introduces a bayesian monte carlo procedure specially designed for such cases it is shown that the performance of this new approach is arbitrarily close to the exact bayesian test in addition for arbitrary desired precisions we develop a theoretical rule of thumb for choosing the minimum number m 0 of monte carlo simulations surprisingly m 0 does not depend on the shape of loss cost functions when those are used to compound the test statistic the method is illustrated for testing mean vectors in high dimension and for detecting spatial clusters of diseases in aggregated maps

the tdnns method for reissner mindlin plates a new family of locking free finite elements for shear deformable reissner mindlin plates is presented the elements are based on the tangential displacement normal normal stress formulation of elasticity in this formulation the bending moments are treated as separate unknowns the degrees of freedom for the plate element are the nodal values of the deflection tangential components of the rotations and normal normal components of the bending strain contrary to other plate bending elements no special treatment for the shear term such as reduced integration is necessary the elements attain an optimal order of convergence

importance sampling for simulation of markovian physical processes transient markov chains are sometimes simulated to estimate rare event probabilities for illustration chains defined by an airborne particle dispersion model are used to estimate the probabilities that released particles reach various locations such estimated probabilities are needed for many purposes including exposure calculations for affected populations and optimization of detector placement by using experimental designs for simulation runs and embedding fitted regression models of output data in importance sampling transition kernels convergence is improved by factors of tens to hundreds

deciphering mrna sequence determinants of protein production rate one of the greatest challenges in biophysical models of translation is to identify coding sequence features that affect the rate of translation and therefore the overall protein production in the cell we propose an analytic method to solve a translation model based on the inhomogeneous totally asymmetric simple exclusion process which allows us to unveil simple design principles of nucleotide sequences determining protein production rates our solution shows an excellent agreement when compared to numerical genome wide simulations of s cerevisiae transcript sequences and predicts that the first 10 codons which is the ribosome footprint length on the mrna together with the value of the initiation rate are the main determinants of protein production rate under physiological conditions finally we interpret the obtained analytic results based on the evolutionary role of the codons choice for regulating translation rates and ribosome densities

stochastic optimization using a trust region method and random models in this paper we propose and analyze a trust region model based algorithm for solving unconstrained stochastic optimization problems our framework utilizes random models of an objective function f x obtained from stochastic observations of the function or its gradient our method also utilizes estimates of function values to gauge progress that is being made the convergence analysis relies on requirements that these models and these estimates are sufficiently accurate with high enough but fixed probability beyond these conditions no assumptions are made on how these models and estimates are generated under these general conditions we show an almost sure global convergence of the method to a first order stationary point in the second part of the paper we present examples of generating sufficiently accurate random models under biased or unbiased noise assumptions lastly we present some computational results showing the benefits of the proposed method compared to existing approaches that are based on sample averaging or stochastic gradients

a theory of statistical models for monte carlo integration summary the task of estimating an integral by monte carlo methods is formulated as a statistical model using simulated observations as data the difficulty in this exercise is that we ordinarily have at our disposal all of the information required to compute integrals exactly by calculus or numerical integration but we choose to ignore some of the information for simplicity or computational feasibility our proposal is to use a semiparametric statistical model that makes explicit what information is ignored and what information is retained the parameter space in this model is a set of measures on the sample space which is ordinarily an infinite dimensional object none the less from simulated data the base line measure can be estimated by maximum likelihood and the required integrals computed by a simple formula previously derived by vardi and by lindsay in a closely related model for biased sampling the same formula was also suggested by geyer and by meng and wong using entirely different arguments by contrast with geyer s retrospective likelihood a correct estimate of simulation error is available directly from the fisher information the principal advantage of the semiparametric model is that variance reduction techniques are associated with submodels in which the maximum likelihood estimator in the submodel may have substantially smaller variance than the traditional estimator the method is applicable to markov chain and more general monte carlo sampling schemes with multiple samplers

trust region methods without using derivatives worst case complexity and the nonsmooth case trust region methods are a broad class of methods for continuous optimization that found application in a variety of problems and contexts in particular they have been studied and applied for problems without using derivatives the analysis of trust region derivative free methods has focused on global convergence and they have been proven to generate a sequence of iterates converging to stationarity independently of the starting point most of such an analysis is carried out in the smooth case and moreover little is known about the complexity or global rate of these methods in this paper we start by analyzing the worst case complexity of general trust region derivative free methods for smooth functions for the nonsmooth case we propose a smoothing approach for which we prove global convergence and bound the worst case complexity effort for the special case of nonsmooth functions that result from the composition of smooth and nonsmooth convex components we show how to improve the existing resu

nonequilibrium steady states of matrix product form a solver s guide we consider the general problem of determining the steady state of stochastic nonequilibrium systems such as those that have been used to model among other things biological transport and traffic flow we begin with a broad overview of this class of driven diffusive systems which includes exclusion processes focusing on interesting physical properties such as shocks and phase transitions we then turn our attention specifically to those models for which the exact distribution of microstates in the steady state can be expressed in a matrix product form in addition to a gentle introduction to this matrix product approach how it works and how it relates to similar constructions that arise in other physical contexts we present a unified pedagogical account of the various means by which the statistical mechanical calculations of macroscopic physical quantities are actually performed we also review a number of more advanced topics including nonequilibrium free energy functionals the classification of exclusion processes involving multiple particle species existence proofs of a matrix product state for a given model and more complicated variants of the matrix product state that allow various types of parallel dynamics to be handled we conclude with a brief discussion of open problems for future research

a derivative free algorithm for least squares minimization we develop a framework for a class of derivative free algorithms for the least squares minimization problem these algorithms are designed to take advantage of the problem structure by building polynomial interpolation models for each function in the least squares minimization under suitable conditions global convergence of the algorithm is established within a trust region framework promising numerical results indicate the algorithm is both efficient and robust numerical comparisons are made with standard derivative free software packages that do not exploit the special structure of the least squares problem or that use finite differences to approximate the gradients

uobyqa unconstrained optimization by quadratic approximation uobyqa is a new algorithm for general unconstrained optimization calculations that takes account of the curvature of the objective function f say by forming quadratic models by interpolation therefore because no first derivatives are required each model is defined by n 1 n 2 values of f where n is the number of variables and the interpolation points must have the property that no nonzero quadratic polynomial vanishes at all of them a typical iteration of the algorithm generates a new vector of variables widetilde underline x t say either by minimizing the quadratic model subject to a trust region bound or by a procedure that should improve the accuracy of the model then usually f widetilde underline x t is obtained and one of the interpolation points is replaced by widetilde underline x t therefore the paper addresses the initial positions of the interpolation points the adjustment of trust region radii the calculation of widetilde underline x t in the two cases that have been mentioned and the selection of the point to be replaced further uobyqa works with the lagrange functions of the interpolation equations explicitly so their coefficients are updated when an interpolation point is moved the lagrange functions assist the procedure that improves the model and also they provide an estimate of the error of the quadratic approximation to f which allows the algorithm to achieve a fast rate of convergence these features are discussed and a summary of the algorithm is given finally a fortran implementation of uobyqa is applied to several choices of f in order to investigate accuracy robustness in the presence of rounding errors the effects of first derivative discontinuities and the amount of work the numerical results are very promising for n 20 but larger values are problematical because the routine work of an iteration is of fourth order in the number of variables

analysis of generalized pattern searches this paper contains a new convergence analysis for the lewis and torczon generalized pattern search gps class of methods for unconstrained and linearly constrained optimization this analysis is motivated by a desire to understand the successful behavior of the algorithm under hypotheses that are satisfied by many practical problems specifically even if the objective function is discontinuous or extended valued the methods find a limit point with some minimizing properties simple examples show that the strength of the optimality conditions at a limit point depends not only on the algorithm but also on the directions it uses and on the smoothness of the objective at the limit point in question the contribution of this paper is to provide a simple convergence analysis that supplies detail about the relation of optimality conditions to objective smoothness properties and to the defining directions for the algorithm and it gives previous results as corollaries

a posteriori error estimation based on the superconvergent recovery by compatibility in patches the present work deals with an a posteriori error estimator for linear finite element analysis based on a stress recovery procedure called recovery by compatibility in patches the key idea of this procedure is to recover improved stresses by minimizing the complementary energy over patches of elements displacements computed by the finite element analysis are prescribed on the boundary of the patch here a new form of this recovery procedure is presented adopting a different patch configuration centred upon an element instead of a node allows to drastically simplify the recovery process thus improving efficiency and making the implementation in finite element codes much easier the robustness tests demonstrate that the error estimator associated to the new form of the recovery procedure retains the very good properties of the original one such as superconvergence the numerical results on two common benchmark problems confirm the effectiveness of the proposed error estimator which appears to be competitive with those currently available copyright 2006 john wiley sons ltd

safe and effective importance sampling abstract we present two improvements on the technique of importance sampling first we show that importance sampling from a mixture of densities using those densities as control variates results in a useful upper bound on the asymptotic variance that bound is a small multiple of the asymptotic variance of importance sampling from the best single component density this allows one to benefit from the great variance reductions obtainable by importance sampling while protecting against the equally great variance increases that might take the practitioner by surprise the second improvement is to show how importance sampling from two or more densities can be used to approach a zero sampling variance even for integrands that take both positive and negative values

global convergence of radial basis function trust region algorithms for derivative free optimization we analyze globally convergent derivative free trust region algorithms relying on radial basis function interpolation models our results extend the recent work of conn scheinberg and vicente siam j optim 20 2009 pp 387 415 to fully linear models that have a nonlinear term we characterize the types of radial basis functions that fit in our analysis and thus show global convergence to first order critical points for the orbit algorithm of wild regis and shoemaker siam j sci comput 30 2008 pp 3197 3219 using orbit we present numerical results for different types of radial basis functions on a series of test problems we also demonstrate the use of orbit in finding local minima on a computationally expensive environmental engineering problem involving remediation of contaminated groundwater

smoothed functionals and constrained stochastic approximation the problem of finding an extremum for a nonconvex function under convex constraints is considered the original nonconvex function is replaced by an auxiliary one called a smoothed function which possesses some nice properties operating with the smoothed function and the given convex constraints the global extremum of the original problem is found

benchmarking derivative free optimization algorithms we propose data profiles as a tool for analyzing the performance of derivative free optimization solvers when there are constraints on the computational budget we use performance and data profiles together with a convergence test that measures the decrease in function value to analyze the performance of three solvers on sets of smooth noisy and piecewise smooth problems our results provide estimates for the performance difference between these solvers and show that on these problems the model based solver tested performs better than the two direct search solvers tested

self correcting geometry in model based algorithms for derivative free unconstrained optimization several efficient methods for derivative free optimization are based on the construction and maintenance of an interpolation model for the objective function most of these algorithms use special geometry improving iterations where the geometry poisedness of the underlying interpolation set is made better at the cost of one or more function evaluations we show that such geometry improvements cannot be completely eliminated if one wishes to ensure global convergence but we also provide an algorithm where such steps occur only in the final stage of the algorithm where criticality of a putative stationary point is verified global convergence for this method is proved by making use of a self correction mechanism inherent to the combination of trust regions and interpolation models this mechanism also throws some light on the surprisingly good numerical results reported by fasano morales and nocedal optim methods softw 24 2009 pp 145 154 for a method where no care is ever taken to guarantee poisedness of the interpolation set

an iterative stochastic method for simulating large deviations and rare events we describe an iterative method capable of determining large deviations responsible for rare events of interest in lightwave systems with additive noise the method makes use of the singular value decomposition svd to efficiently compute the most important directions in state space and a stochastic optimization scheme known as the cross entropy ce method to determine the most probable manner in which these large deviations arise information from the svd and ce steps of the method provides a basis for performing importance sampling with monte carlo simulation allowing one to determine the probabilities of the rare events associated with such large deviations we apply the combined method to investigate some of the mechanisms affecting large amplitude fluctuations in optical systems

a method to compute statistics of large noise induced perturbations of nonlinear schrodinger solitons we demonstrate in detail the application of importance sampling to the numerical simulation of large noise induced perturbations in soliton based optical transmission systems governed by the nonlinear schrodinger equation the method allows one to concentrate numerical monte carlo simulations around the noise realizations that are most likely to produce the large pulse deformations connected with errors and yields computational speedups of several orders of magnitude over standard monte carlo simulations we demonstrate the method by using it to calculate the probability density functions associated with pulse amplitude frequency and timing fluctuations in a prototypical soliton based communication system

totally asymmetric exclusion process with extended objects a model for protein synthesis the process of protein synthesis in biological systems resembles a one dimensional driven lattice gas in which the particles have spatial extent covering more than one lattice site we expand the well studied totally asymmetric exclusion process in which particles typically cover a single lattice site to include cases with extended objects exact solutions can be determined for a uniform closed system we analyze the uniform open system through two approaches first a continuum limit produces a modified diffusion equation for particle density profiles second an extremal principle based on domain wall theory accurately predicts the phase diagram and currents in each phase finally we briefly consider approximate approaches to a nonuniform open system with quenched disorder in the particle hopping rates and compare these approaches with monte carlo simulations

the effect of deterministic noise in subgradient methods in this paper we study the influence of noise on subgradient methods for convex constrained optimization the noise may be due to various sources and is manifested in inexact computation of the subgradients and function values assuming that the noise is deterministic and bounded we discuss the convergence properties for two cases the case where the constraint set is compact and the case where this set need not be compact but the objective function has a sharp set of minima for example the function is polyhedral in both cases using several different stepsize rules we prove convergence to the optimal value within some tolerance that is given explicitly in terms of the errors in the first case the tolerance is nonzero but in the second case the optimal value can be obtained exactly provided the size of the error in the subgradient computation is below some threshold we then extend these results to objective functions that are the sum of a large number of convex functions in which case an incremental subgradient method can be used

multivariate stochastic approximation using a simultaneous perturbation gradient approximation the problem of finding a root of the multivariate gradient equation that arises in function minimization is considered when only noisy measurements of the function are available a stochastic approximation sa algorithm for the general kiefer wolfowitz type is appropriate for estimating the root the paper presents an sa algorithm that is based on a simultaneous perturbation gradient approximation instead of the standard finite difference approximation of keifer wolfowitz type procedures theory and numerical experience indicate that the algorithm can be significantly more efficient than the standard algorithms in large dimensional problems

smoothed functionals in stochastic optimization we consider the problem of finding a global extremum for a nonconvex function by constructing another smoothed function which has a better behavior than the original one then operating only with the smoothed function we find the global extremum for the original function

two stage importance sampling with mixture proposals for importance sampling is multiple proposals can be combined to address different aspects of a target distribution there are various methods for is with multiple proposals including hesterberg s stratified is estimator owen and zhou s regression estimator and tan s maximum likelihood estimator for the problem of efficiently allocating samples to different proposals it is natural to use a pilot sample to select the mixture proportions before the actual sampling and estimation however most current discussions are in an empirical sense for such a two stage procedure in this article we establish a theoretical framework of applying the two stage procedure for various methods including the asymptotic properties and the choice of the pilot sample size by our simulation studies these two stage estimators can outperform estimators with naive choices of mixture proportions furthermore while owen and zhou s and tan s estimators are designed for estimating normalizing constants we extend their usa

a derivative free trust region algorithm for the optimization of functions smoothed via gaussian convolution using adaptive multiple importance sampling in this paper we consider the optimization of a functional f defined as the convolution of a function f with a gaussian kernel we propose this type of objective function for the optimization of the output of complex computational simulations which often present some form of deterministic noise and need to be smoothed for the results to be meaningful we introduce a derivative free algorithm that computes trial points from the minimization of a regression model of the noisy function f over a trust region the regression model is constructed from function values at sample points that are chosen randomly around iterates and trial points of the algorithm the weights given to the individual sample points in the regression problem are obtained according to an adaptive multiple importance sampling strategy this has two advantages first it makes it possible to reuse all noisy function values collected over the course of the optimization second the resulting regression model converges to the second o

locking free continuum displacement finite elements with nodal integration summary an assumed strain flnite element technique is presented for linear elastic small deformation models weighted residual method reminiscent of the strain displacement functional is used to enforce weakly the balance equation with the natural boundary condition and the kinematic equation the strain displacement relationship a priori satisfaction of the kinematic weighted residual serves as a condition from which strain displacement operators are derived via nodal integration a variety of element shapes is treated linear triangles quadrilaterals tetrahedra hexahedra and furthermore quadratic six node triangles and 27 node hexahedra the degrees of freedom are only the primitive variables displacements at the nodes the formulation allows for general anisotropic materials a straightforward constraint count can partially explain the insensitivity of the resulting flnite element models to locking in the incompressible limit furthermore the numerical inf sup test is applied in select problems and several variants of the proposed formulations linear triangles quadrilaterals tetrahedra hexahedra and 27 node hexahedra pass the test examples are used to illustrate the performance with with respect to sensitivity to shape distortion and ability to resist volumetric locking copyright c 2007 john wiley sons ltd

pattern search methods for user provided points application to molecular geometry problems this paper deals with the application of pattern search methods to the numerical solution of a class of molecular geometry problems with important applications in molecular physics and chemistry the goal is to find a configuration of a cluster or a molecule with minimum total energy the minimization problems in this class of molecular geometry problems have no constraints and the objective function is smooth the difficulties arise from the existence of several local minima and especially from the expensive function evaluation total energy and the possible nonavailability of first order derivatives we introduce a pattern search approach that attempts to exploit the physical nature of the problem by using energy lowering geometrical transformations and to take advantage of parallelism without the use of derivatives numerical results for a particular instance of this new class of pattern search methods are presented showing the promise of our approach the new pattern search methods can be used in any other context where there is a user provided scheme to generate points leading to a potential objective function decrease