parameter identification of nonlinear aeroelastic system with time delayed feedback control in this paper an enhanced response sensitivity approach is proposed for parameter identification of nonlinear time delay aeroelastic systems two common aeroelastic models are investigated namely

adaptive estimation of the dynamics of a discrete time stochastic volatility model abstract this paper is concerned with the discrete time stochastic volatility model y i exp x i 2 i x i 1 b x i x i i 1 where only y i is observed the model is rewritten as a particular hidden model z i x i e i x i 1 b x i x i i 1 where i and e i are independent sequences of i i d noise moreover the sequences x i and e i are independent and the distribution of e is known then our aim is to estimate the functions b and 2 when only observations z 1 z n are available we propose to estimate b f and b 2 2 f and study the integrated mean square error of projection estimators of these functions on automatically selected projection spaces by ratio strategy estimators of b and 2 are then deduced the mean square risk of the resulting estimators are studied and their rates are discussed lastly simulation experiments are provided constants in the penalty functions defining the estimators are calibrated and the quality of the estimators is checked on several examples

spatially inhomogeneous linear inverse problems with possible singularities the objective of the present paper is to introduce the concept of a spatially inhomogeneous linear inverse problem which to the best of the author s knowledge has never been considered previously in statistical framework but is emerging due to a variety of practical applications the special feature of the problem is that the degree of ill posedness depends not only on the scale but also on location in this case the rates of convergence are determined by the interaction of four parameters the smoothness and spatial homogeneity of the unknown function f and degrees of ill posedness and spatial inhomogeneity of operator q an interesting property here is that if operator q is weakly inhomogeneous then the rates of convergence are not influenced by spatial inhomogeneity of operator q and coincide with the rates which are usual for homogeneous linear inverse problems on the other hand if operator q is moderately or strongly inhomogeneous convergence rates are significantly affected by the degree of spatial inhomogeneity estimators obtained in the paper are based either on wavelet vaguelette decomposition if the norms of all vaguelettes are finite or on a hybrid of wavelet vaguelette decomposition and galerkin method if vaguelettes in the neighborhood of the singularity point have infinite norms the hybrid estimator is a combination of a linear part in the vicinity of the singularity point and the nonlinear block thresholding wavelet estimator elsewhere to attain adaptivity an optimal resolution level for the linear singularity affected portion of the estimator is obtained using lepskii 1990 1999 method subsequently this resolution level is used as the lowest resolution level for the nonlinear wavelet estimator we show that convergence rates of the hybrid estimator lie within a logarithmic factor of the optimal minimax convergence rates the theory presented in the paper is supplemented by examples of deconvolution based on irregularly spaced observations or in the presence of a heteroscedastic noise the latter problem is also examined via a limited simulation study which demonstrates advantages of the hybrid estimator when the degree of spatial inhomogeneity is high

optimal rates of convergence for deconvolving a density abstract suppose that the sum of two independent random variables x and z is observed where z denotes measurement error and has a known distribution and where the unknown density f of x is to be estimated one application is the estimation of a prior density for a sequence of location parameters a second application arises in the errors in variables problem for nonlinear and generalized linear models when one attempts to model the distribution of the true but unobservable covariates this article shows that if z is normally distributed and f has k bounded derivatives then the fastest attainable convergence rate of any nonparametric estimator of f is only log n k 2 therefore deconvolution with normal errors may not be a practical proposition other error distributions are also treated stefanski carroll 1987a estimators achieve the optimal rates the results given have versions for multiplicative errors where they imply that even optimal rates are exceptionally slow

nonlinear non negative component analysis algorithms in this paper general solutions for nonlinear non negative component analysis for data representation and recognition are proposed motivated by a combination of the non negative matrix factorization nmf algorithm and kernel theory which has lead to a recently proposed nmf algorithm in a polynomial feature space we propose a general framework where one can build a nonlinear non negative component analysis method using kernels the so called projected gradient kernel non negative matrix factorization pgknmf in the proposed approach arbitrary positive definite kernels can be adopted while at the same time it is ensured that the limit point of the procedure is a stationary point of the optimization problem moreover we propose fixed point algorithms for the special case of gaussian radial basis function rbf kernels we demonstrate the power of the proposed methods in face and facial expression recognition applications

modified subspace limited memory bfgs algorithm for large scale bound constrained optimization in this paper a subspace limited memory bfgs algorithm for solving large scale bound constrained optimization problems is developed it is modifications of the subspace limited memory quasi newton method proposed by ni and yuan q ni y x yuan a subspace limited memory quasi newton algorithm for large scale nonlinear bound constrained optimization math comput 66 1997 1509 1520 an important property of our proposed method is that more limited memory bfgs update is used under appropriate conditions the global convergence of the method is established the implementations of the method on cute test problems are presented which indicate the modifications are beneficial to the performance of the algorithm

newton methods for large scale linear inequality constrained minimization newton methods of the linesearch type for large scale minimization subject to linear inequality constraints are discussed the purpose of the paper is twofold i to give an active set type method with the ability to delete multiple constraints simultaneously and ii to give a relatively short general convergence proof for such a method it is also discussed how multiple constraints can be added simultaneously the approach is an extension of a previous work by the same authors for equality constrained problems it is shown how the search directions can be computed without the need to compute the reduced hessian of the objective function the convergence analysis states that every limit point of a sequence of iterates satisfies the second order necessary optimality conditions

an active set newton algorithm for large scale nonlinear programs with box constraints a new algorithm for large scale nonlinear programs with box constraints is introduced the algorithm is based on an efficient identification technique of the active set at the solution and on a nonmonotone stabilization technique it possesses global and superlinear convergence properties under standard assumptions a new technique for generating test problems with known characteristics is also introduced the implementation of the method is described along with computational results for large scale problems

an active set truncated newton method for large scale bound constrained optimization an active set truncated newton method for large scale bound constrained optimization is proposed the active sets are guessed by an identification technique the search direction consists of two parts some of the components are simply defined the other components are determined by the truncated newton method the method based on a nonmonotone line search technique is shown to be globally convergent numerical experiments are presented using bound constrained problems in the cuter test problem library the numerical performance reveals that our method is effective and competitive with the famous algorithm tron

newton s method for large bound constrained optimization problems we analyze a trust region version of newton s method for bound constrained problems our approach relies on the geometry of the feasible set not on the particular representation in terms of constraints the convergence theory holds for linearly constrained problems and yields global and superlinear convergence without assuming either strict complementarity or linear independence of the active constraints we also show that the convergence theory leads to an efficient implementation for large bound constrained problems

laplace deconvolution on the basis of time domain data and its application to dynamic contrast enhanced imaging in the present paper we consider the problem of laplace deconvolution with noisy discrete non equally spaced observations on a finite time interval which appears in many different contexts we propose a new method for laplace deconvolution which is based on expansions of the convolution kernel the unknown function and the observed signal over laguerre functions basis which acts as a surrogate eigenfunction basis using regression setting the expansion results in a small system of linear equations with the matrix of the system being triangular and toeplitz the number m of the terms in the expansion of the estimator is controlled via complexity penalty the advantage of this methodology is that it leads to very fast computations produces no boundary effects due to extension at zero and cut off at t and provides an estimator with the risk within a logarithmic factor of m of the oracle risk under no assumptions on the model and within a constant factor of the oracle risk under mild assumptions we emphasize that in the present paper we consider the true observational model with possibly non equispaced observations which are available on a finite interval of length t and account for the bias associated with this model the study is motivated by perfusion imaging using a short injection of contrast agent a procedure which is widely applied for medical assessment of blood flows within tissues such as cancerous tumors presence of a tuning parameter a allows to choose the most advantageous time units so that both the kernel and the unknown right hand side of the equation are well represented for the further deconvolution in addition absence of boundary effects allows for an accurate estimator of the tissue blood flow parameter which is of great interest to medical doctors the methodology is illustrated by an extensive simulation study and a real data example which confirms that the proposed technique is fast efficient accurate usable from a practical point of view and competitive

shearlet based deconvolution in this paper a new type of deconvolution algorithm is proposed that is based on estimating the image from a shearlet decomposition shearlets provide a multidirectional and multiscale decomposition that has been mathematically shown to represent distributed discontinuities such as edges better than traditional wavelets constructions such as curvelets and contourlets share similar properties yet their implementations are significantly different from that of shearlets taking advantage of unique properties of a new m channel implementation of the shearlet transform we develop an algorithm that allows for the approximation inversion operator to be controlled on a multiscale and multidirectional basis a key improvement over closely related approaches such as forward is the automatic determination of the threshold values for the noise shrinkage for each scale and direction without explicit knowledge of the noise variance using a generalized cross validation gcv various tests show that this method can perform significantly better than many competitive deconvolution algorithms

optimality measures for performance profiles we examine the importance of optimality measures when benchmarking a set of solvers and show that the scale invariance requirements we impose lead to a convergence test for nonlinearly constrained optimization solvers that uses a mixture of absolute and relative error measures we demonstrate that this convergence test is well behaved at any point where the constraints satisfy the mangasarian fromovitz constraint qualification and also avoids the explicit use of a complementarity measure computational experiments explore the impact of this convergence test on the benchmarking process with performance profiles

on the identification of active constraints nondegeneracy conditions that guarantee that the optimal active constraints are identified in a finite number of iterations are studied results of this type have only been established for a few algorithms and then under restrictive hypothesis the main result is a characterization of those algorithms that identify the optimal constraints in a finite number of iterations this result is obtained with a nondegeneracy assumption which is equivalent in the standard nonlinear programming problem to the assumption that there is a set of strictly complementary lagrange multipliers as an important consequence of the authors results the way that this characterization applies to gradient projection and sequential quadratic programming algorithms is shown

on the optimal rates of convergence for nonparametric deconvolution problems deconvolution problems arise in a variety of situations in statistics an interesting problem is to estimate the density f of a random variable x based on n i i d observations from y x varepsilon where varepsilon is a measurement error with a known distribution in this paper the effect of errors in variables of nonparametric deconvolution is examined insights are gained by showing that the difficulty of deconvolution depends on the smoothness of error distributions the smoother the harder in fact there are two types of optimal rates of convergence according to whether the error distribution is ordinary smooth or supersmooth it is shown that optimal rates of convergence can be achieved by deconvolution kernel density estimators

density estimation for the case of supersmooth measurement error abstract the problem is to estimate the probability density of a random variable contaminated by an independent measurement error i explore one of the worst case scenario when the characteristic function of this measurement error decreases exponentially and thus optimal estimators converge only with logarithmic rate the particular example of such measurement error is any random variable contaminated by normal cauchy or another stable random variable for this setting and circular data i suggest an asymptotically efficient data driven estimator that is adaptive to both smoothness of estimated density and distribution of measurement error moreover this estimator is universal in sense that its derivatives and integral are sharp estimators of the corresponding derivatives and the cumulative distribution function and these estimators are sharp both globally and pointwise for the case of small sample sizes i suggest a modified estimator that mimics an optimal linear pseudoestimator i explore this est

image processing using newton based algorithm of nonnegative matrix factorization in this paper we propose a newton based algorithm for nonnegative matrix factorization in image processing we employ the new algorithm to three real world databases extensive numerical results show the feasibility and validity of the proposed algorithm

a limited memory quasi newton trust region method for box constrained optimization by means of wolfe conditions strategy we propose a quasi newton trust region method to solve box constrained optimization problems this method is an adequate combination of the compact limited memory bfgs and the trust region direction while the generated point satisfies the wolfe conditions and therefore maintains a positive definite approximation to the hessian of the objective function the global convergence and the quadratic convergence of this method are established under suitable conditions finally we compare our algorithms iwtral and ibwtral with an active set trust region algorithm astral xu and burke 2007 on the cutest box constrained test problems presented by gould et al 2015 numerical results show that the presented method is competitive and totally interesting for solving box constrained optimization

projected gradient methods for nonnegative matrix factorization nonnegative matrix factorization nmf can be formulated as a minimization problem with bound constraints although bound constrained optimization has been studied extensively in both theory and practice so far no study has formally applied its techniques to nmf in this letter we propose two projected gradient methods for nmf both of which exhibit strong optimization properties we discuss efficient implementations and demonstrate that one of the proposed methods converges faster than the popular multiplicative update approach a simple matlab code is also provided

two metric projection methods for constrained optimization this paper is concerned with the problem min f x mid x in x where x is a convex subset of a linear space h and f is a smooth real valued function on h we propose the class of methods x k 1 p x k alpha k g k where p denotes projection on x with respect to a hilbert space norm cdot g k denotes the frechet derivative of f at x k with respect to another hilbert space norm cdot k on h and alpha k is a positive scalar stepsize we thus remove an important restriction in the original proposal of goldstein 1 and levitin and poljak 2 where the norms cdot and cdot k must be the same it is therefore possible to match the norm cdot with the structure of x so that the projection operation is simplified while at the same time reserving the option to choose cdot k on the basis of approximations to the hessian of f so as to attain a typically superlinear rate of convergence the resulting methods are particularly attrac

incomplete cholesky factorizations with limited memory we propose an incomplete cholesky factorization for the solution of large scale trust region subproblems and positive definite systems of linear equations this factorization depends on a parameter p that specifies the amount of additional memory in multiples of n the dimension of the problem that is available there is no need to specify a drop tolerance our numerical results show that the number of conjugate gradient iterations and the computing time are reduced dramatically for small values of p we also show that in contrast with drop tolerance strategies the new approach is more stable in terms of number of iterations and memory requirements

wavelet deconvolution in a periodic setting in this paper we present an inverse estimation procedure which combines fourier analysis with wavelet expansion in the periodic setting our method can recover a blurred function observed in white noise the blurring process is achieved through a convolution operator which can either be smooth polynomial decay of the fourier transform or irregular such as the convolution with a box car the proposal is non linear and does not require any prior knowledge of the smoothness class it enjoys fast computation and is spatially adaptive this contrasts with more traditional ltering methods which demand a certain amount of regularisation and often fail to recover non homogeneous functions a ne tuning of our method is derived via asymptotic minimax theory which reveals some key dierences with the direct case of donoho et al 1995 a band limited wavelet families have nice theoretical and computing features b the high frequency cut o depends on the spectral characteristics of the convolution kernel c thresholds are level dependent in a geometric fashion we tested our method using simulated lidar data for underwater remote sensing both visual and numerical results show an improvement over existing methods finally the theory behind our estimation paradigm gives a complete characterisation of the maxiset of the method i e the set of functions where the method attains a near optimal rate of convergence for a variety of l p loss functions

adaptive wavelet estimator for nonparametric density deconvolution 1999 institute of mathematical statistics the electronic version of this article is the complete one and can be found online at http projecteuclid org euclid aos 1017939249

wavelet deconvolution with noisy eigenvalues over the last decade there has been a lot of interest in wavelet vaguelette methods for the recovery of noisy signals or images in motion blur nonlinear wavelet estimators are known to have good adaptive properties and to outperform linear approximations over a wide range of signals and images see e g the recent waved method of johnstone kerkyacharian picard and raimondo 2004 or the forward method of neelamani choi and baraniuk 2004 and also fan and koo 2002 in the density setting in the deblurring setting wavelet vaguelette methods rely on the complete knowledge of a convolution operator s eigenvalues this is an unlikely situation in practice however a more realistic scenario such as would arise when passing the fourier basis as an input signal through a linear time invariant system is to imagine that one also observes a set of noisy eigenvalues in this paper we define a version of the waved estimator which is near optimal when used with noisy eigenvalues a key feature of our method includes a data driven method for choosing the fine resolution level in waved estimation asymptotic theory is illustrated with a wide range of finite sample examples

global convergence of a class of trust region algorithms for optimization with simple bounds this paper extends the known excellent global convergence properties of trust region algorithms for unconstrained optimization to the case where bounds on the variables are present weak conditions on the accuracy of the hessian approximations are considered it is also shown that when the strict complementarily condition holds the proposed algorithms reduce to an unconstrained calculation after finitely many iterations allowing a fast asymptotic rate of convergence

a study on reduced support vector machines recently the reduced support vector machine rsvm was proposed as an alternate of the standard svm motivated by resolving the difficulty on handling large data sets using svm with nonlinear kernels it preselects a subset of data as support vectors and solves a smaller optimization problem however several issues of its practical use have not been fully discussed yet for example we do not know if it possesses comparable generalization ability as the standard svm in addition we would like to see for how large problems rsvm outperforms svm on training time in this paper we show that the rsvm formulation is already in a form of linear svm and discuss four rsvm implementations experiments indicate that in general the test accuracy of rsvm are a little lower than that of the standard svm in addition for problems with up to tens of thousands of data if the percentage of support vectors is not high existing implementations for svm is quite competitive on the training time thus from this empirical study rsvm will be mainly useful for either larger problems or those with many support vectors experiments in this paper also serve as comparisons of 1 different implementations for linear svm and 2 standard svm using linear and quadratic cost functions

multiscale methods for shape constraints in deconvolution confidence statements for qualitative features we derive multiscale statistics for deconvolution in order to detect qualitative features of the unknown density an important example covered within this framework is to test for local monotonicity on all scales simultaneously we investigate the moderately ill posed setting where the fourier transform of the error density in the deconvolution model is of polynomial decay for multiscale testing we consider a calibration motivated by the modulus of continuity of brownian motion we investigate the performance of our results from both the theoretical and simulation based point of view a major consequence of our work is that the detection of qualitative features of a density in a deconvolution problem is a doable task although the minimax rates for pointwise estimation are very slow

a sparse counterpart of reichel and gragg s package qrup we describe how to maintain the triangular factor of a sparse qr factorization when columns are added and deleted and q cannot be stored for sparsity reasons the updating procedures could be thought of as a sparse counterpart of reichel and gragg s package qrup they allow us to solve a sequence of sparse linear least squares subproblems in which each matrix b k is an independent subset of the columns of a fixed matrix a and b k 1 is obtained by adding or deleting one column like coleman and hulbert t coleman l hulbert a direct active set algorithm for large sparse quadratic programs with simple bounds math program 45 1989 373 406 we adapt the sparse direct methodology of bjorck and oreborn of the late 1980s but without forming a ta which may be only positive semidefinite our matlab 5 implementation works with a suitable row and column numbering within a static triangular sparsity pattern that is computed in advance by symbolic factorization of a ta and preserved with placeholders

convergence properties of trust region methods for linear and convex constraints we develop a convergence theory for convex and linearly constrained trust region methods which only requires that the step between iterates produce a sufficient reduction in the trust region subproblem global convergence is established for general convex constraints while the local analysis is for linearly constrained problems the main local result establishes that if the sequence converges to a nondegenerate stationary point then the active constraints at the solution are identified in a finite number of iterations as a consequence of the identification properties we develop rate of convergence results by assuming that the step is a truncated newton method our development is mainly geometrical this approach allows the development of a convergence theory without any linear independence assumptions

on deconvolution of distribution functions the subject of this paper is the problem of nonparametric estimation of a continuous distribution function from observations with measurement errors we study minimax complexity of this problem when unknown distribution has a density belonging to the sobolev class and the error density is ordinary smooth we develop rate optimal estimators based on direct inversion of empirical characteristic function we also derive minimax affine estimators of the distribution function which are given by an explicit convex optimization problem adaptive versions of these estimators are proposed and some numerical results demonstrating good practical behavior of the developed procedures are presented

h control with transients in h infty or uniformly optimal control problems it is usually assumed that the system initial conditions are zero in this paper an h infty like control problem that incorporates uncertainty in initial conditions is formulated this is done by defining a worst case performance measure both finite and infinite horizon problems are considered necessary and sufficient conditions are derived for the existence of controllers that yield a closed loop system for which the above mentioned performance measure is less than a prespecified value state space formulae for the controllers are also presented

discrete transform approach to deconvolution problems summary if fourier series are used as the basis for inference in deconvolution problems the effects of the errors factorise out in a way that is easily exploited empirically this property is the consequence of elementary addition formulae for sine and cosine functions and is not readily available when one is using methods based on other orthogonal series or on continuous fourier transforms it allows relatively simple estimators to be constructed founded on the addition of finite series rather than on integration the performance of these methods can be particularly effective when edge effects are involved since cosine series estimators are quite resistant to boundary problems in this context we point to the advantages of trigonometric series methods for density deconvolution they have better mean squared error performance when edge effects are involved they are particularly easy to code and they admit a simple approach to empirical choice of smoothing parameter in which a version of thresholding familiar in wavelet based inference is used in place of conventional smoothing applications to other deconvolution problems are briefly discussed

data driven density estimation in the presence of additive noise with unknown distribution we study the following model of deconvolution y x varepsilon with i i d observations y 1 dots y n and varepsilon 1 dots varepsilon m the x j 1 leq j leq n are i i d with density f independent of the varepsilon j the aim of the paper is to estimate f without knowing the density f varepsilon of the varepsilon j we first define a projection estimator for which we provide bounds for the pointwise and the integrated l 2 risk we consider ordinary smooth and supersmooth noise varepsilon with regard to ordinary smooth and supersmooth densities f then we present an adaptive estimator of the density of f this estimator is obtained by penalization of the projection contrast which provides model selection lastly we present simulation experiments to illustrate the good performances of our estimator and study from the empirical point of view the importance of theoretical constraints

automatic alignment of genus zero surfaces a new algorithm is presented that provides a constructive way to conformally warp a triangular mesh of genus zero to a destination surface with minimal metric deformation as well as a means to compute automatically a measure of the geometric difference between two surfaces of genus zero the algorithm takes as input a pair of surfaces that are topological 2 spheres each surface given by a distinct triangulation the algorithm then constructs a map f between the two surfaces first each of the two triangular meshes is mapped to the unit sphere using a discrete conformal mapping algorithm the two mappings are then composed with a mobius transformation to generate the function f the mobius transformation is chosen by minimizing an energy that measures the distance of f from an isometry we illustrate our approach using several real life data sets we show first that the algorithm allows for accurate automatic and landmark free nonrigid registration of brain surfaces we then validate our approach by comparing shapes of proteins we provide numerical experiments to demonstrate that the distances computed with our algorithm between low resolution surface based representations of proteins are highly correlated with the corresponding distances computed between high resolution atomistic models for the same proteins

an infeasible active set method for quadratic problems with simple bounds a primal dual active set method for quadratic problems with bound constraints is presented based on a guess on the active set a primal dual pair x s is computed that satisfies the first order optimality condition and the complementarity condition if x s is not feasible a new active set is determined and the process is iterated sufficient conditions for the iterations to stop in a finite number of steps with an optimal solution are provided computational experience indicates that this approach often requires only a few less than 10 iterations to find the optimal solution

uniform confidence bands for nonparametric errors in variables regression abstract this paper develops a method to construct uniform confidence bands for a nonparametric regression function where a predictor variable is subject to a measurement error we allow for the distribution of the measurement error to be unknown but assume the availability of validation data or repeated measurements on the latent predictor variable the proposed confidence band builds on the deconvolution kernel estimation and a novel application of the multiplier bootstrap method we establish asymptotic validity of the proposed confidence band to our knowledge this is the first paper to derive asymptotically valid uniform confidence bands for nonparametric errors in variables regression

a connection between h control and the absolute stabilizability of uncertain systems abstract this paper considers a stabilization problem for a class of uncertain systems in which the uncertainty satisfies a certain integral quadratic constraint such uncertain systems have been previously studied in the theory of absolute stability with most publications in this area being in the russian literature the paper introduces a notion of absolute stabilizability and shows that an uncertain system is absolutely stabilizable if and only if there exists a solution to a corresponding h control problem the paper also shows that if an uncertain system can be absolutely stabilized via nonlinear output feedback control then it can be absolutely stabilized via linear output feedback control

wavelet deconvolution this paper studies the issue of optimal deconvolution density estimation using wavelets the approach taken here can be considered as orthogonal series estimation in the more general context of the density estimation we explore the asymptotic properties of estimators based on thresholding of estimated wavelet coefficients minimax rates of convergence under the integrated square loss are studied over besov classes b sub spl sigma pq of functions for both ordinary smooth and supersmooth convolution kernels the minimax rates of convergence depend on the smoothness of functions to be deconvolved and the decay rate of the characteristic function of convolution kernels it is shown that no linear deconvolution estimators can achieve the optimal rates of convergence in the besov spaces with p 2 when the convolution kernel is ordinary smooth and super smooth if the convolution kernel is ordinary smooth then linear estimators can be improved by using thresholding wavelet deconvolution estimators which are asymptotically minimax within logarithmic terms adaptive minimax properties of thresholding wavelet deconvolution estimators are also discussed

a trust region and affine scaling interior point method for nonconvex minimization with linear inequality constraints a trust region and affine scaling interior point method tram is proposed for a general nonlinear minimization with linear inequality constraints in 8 in the proposed approach a newton step is derived from the complementarity conditions based on this newton step a trust region subproblem is formed and the original objective function is monotonically decreased explicit sufficient decrease conditions are proposed for satisfying complementarity dual feasibility and second order optimality the objective of this paper is to establish global and local convergence properties of the proposed trust region and affine scaling interior point method it is shown that the proposed decrease conditions are sufficient for achieving complementarity dual feasibility and second order optimality respectively it is also established that a trust region solution is asymptotically in the interior of the proposed trust region subproblem and a damped trust region step can achieve quadratic convergence

global convergence of a class of trust region algorithms for optimization using inexact projections on convex constraints a class of trust region based algorithms is presented for the solution of nonlinear optimization problems with a convex feasible set at variance with previously published analyses of this type the theory presented allows for the use of general norms furthermore the proposed algorithms do not require the explicit computation of the projected gradient and can therefore be adapted to cases where the projection onto the feasible domain may be expensive to calculate strong global convergence results are derived for the class it is also shown that the set of linear and nonlinear constraints that are binding at the solution are identified by the algorithms of the class in a finite number of iterations

robust control with rejection of harmonic disturbances this paper considers a finite time output feedback control problem for a class of uncertain systems subject to harmonic disturbances of known frequencies the controller is required to minimize a quadratic cost functional subject to the given uncertainties and harmonic disturbances the main result gives a necessary and sufficient condition for the existence of a suitable controller the condition is given terms of a pair of riccati differential equations which arise in a corresponding h sup spl infin control problem

a deconvolution approach to estimation of a common shape in a shifted curves model this paper considers the problem of adaptive estimation of a mean pattern in a randomly shifted curve model we show that this problem can be transformed into a linear inverse problem where the density of the random shifts plays the role of a convolution operator an adaptive estimator of the mean pattern based on wavelet thresholding is proposed we study its consistency for the quadratic risk as the number of observed curves tends to infinity and this estimator is shown to achieve a near minimax rate of convergence over a large class of besov balls this rate depends both on the smoothness of the common shape of the curves and on the decay of the fourier coefficients of the density of the random shifts hence this paper makes a connection between mean pattern estimation and the statistical analysis of linear inverse problems which is a new point of view on curve registration and image warping problems we also provide a new method to estimate the unknown random shifts between curves some numerical experiments are given to illustrate the performances of our approach and to compare them with another algorithm existing in the literature

thresholding estimators for linear inverse problems and deconvolutions thresholding algorithms in an orthonormal basis are studied to estimate noisy discrete signals degraded by a linear operator whose inverse is not bounded for signals in a set sufficient conditions are established on the basis to obtain a maximum risk with minimax rates of convergence deconvolutions with kernels having a fourier transform which vanishes at high frequencies are examples of unstable inverse problems where a thresholding in a wavelet basis is a suboptimal estimator a new mirror wavelet basis is constructed to obtain a deconvolution risk which is proved to be asymptotically equivalent to the minimax risk over bounded variation signals this thresholding estimator is used to restore blurred satellite images

preconditioned conjugate gradient algorithms for nonconvex problems with box constraints the paper describes new conjugate gradient algorithms for large scale nonconvex problems with box constraints in order to speed up convergence the algorithms employ scaling matrices which transform the space of original variables into the space in which hessian matrices of the problem s functionals have more clustered eigenvalues this is done by applying limited memory bfgs updating matrices once the scaling matrix is calculated the next few conjugate gradient iterations are performed in the transformed space the box constraints are treated efficiently by the projection we also present a limited memory quasi newton method which is a special version of our general algorithm the presented algorithms have strong global convergence properties in particular they identify constraints active at a solution in a finite number of iterations we believe that they are competitive to the l bfgs b method and present some numerical results which support our claim

deconvolution of microfluorometric histograms with b splines abstract we consider the problem of estimating a probability density from observations from that density which are further contaminated by random errors we propose a method of estimation using spline functions discuss the numerical implementation of the method and prove its consistency the problem is motivated by the analysis of dna content obtained by microfluorometry and an example of such an analysis is included

a new active set algorithm for box constrained optimization an active set algorithm asa for box constrained optimization is developed the algorithm consists of a nonmonotone gradient projection step an unconstrained optimization step and a set of rules for branching between the two steps global convergence to a stationary point is established for a nondegenerate stationary point the algorithm eventually reduces to unconstrained optimization without restarts similarly for a degenerate stationary point where the strong second order sufficient optimality condition holds the algorithm eventually reduces to unconstrained optimization without restarts a specific implementation of the asa is given which exploits the recently developed cyclic barzilai borwein cbb algorithm for the gradient projection step and the recently developed conjugate gradient algorithm cg descent for unconstrained optimization numerical experiments are presented using box constrained problems in the cuter and minpack 2 test problem libraries

alternative gradient algorithms with applications to nonnegative matrix factorizations three nonnegative matrix factorization nmf algorithms are discussed and employed to three real world applications based on the alternative gradient algorithm with the iteration steps being determined columnwisely without projection and columnwisely and elementwisely with projections three algorithms are developed respectively also the computational costs and the convergence properties of the new algorithms are given the numerical examples show the advantage of our algorithms over the multiplicative update algorithm proposed by lee and seung 11

low order approximations in deconvolution and regression with errors in variables summary we suggest two new methods which are applicable to both deconvolution and regression with errors in explanatory variables for nonparametric inference the two approaches involve kernel or orthogonal series methods they are based on defining a low order approximation to the problem at hand and proceed by constructing relatively accurate estimators of that quantity rather than attempting to estimate the true target functions consistently of course both techniques could be employed to construct consistent estimators but in many contexts of importance e g those where the errors are gaussian consistency is from a practical viewpoint an unattainable goal we rephrase the problem in a form where an explicit interpretable low order approximation is available the information that we require about the error distribution the error in variables distribution in the case of regression is only in the form of low order moments and so is readily obtainable by a rudimentary analysis of indirect measurements of errors e g through repeated measurements in particular we do not need to estimate a function such as a characteristic function which expresses detailed properties of the error distribution this feature of our methods coupled with the fact that all our estimators are explicitly defined in terms of readily computable averages means that the methods are particularly economical in computing time

uniform confidence bands in deconvolution with unknown error distribution abstract this paper develops a method to construct uniform confidence bands in deconvolution when the error distribution is unknown simulation studies demonstrate the performance of the multiplier bootstrap confidence band in the finite sample we apply our method to the outer continental shelf ocs auction data and draw confidence bands for the density of common values of mineral rights on oil and gas tracts we also present an application of our main theoretical result specifically to additive fixed effect panel data models and we draw confidence bands for the density of the total factor productivity in a manufacturing industry in chile

a convergent least squares regularized blind deconvolution approach the aim of this work is to present a new and efficient optimization method for the solution of blind deconvolution problems with data corrupted by gaussian noise which can be reformulated as a constrained minimization problem whose unknowns are the point spread function psf of the acquisition system and the true image the objective function we consider is the weighted sum of the least squares fit to data discrepancy and possible regularization terms accounting for specific features to be preserved in both the image and the psf the solution of the corresponding minimization problem is addressed by means of a proximal alternating linearized minimization palm algorithm in which the updating procedure is made up of one step of a gradient projection method along the arc and the choice of the parameter identifying the steplength in the descent direction is performed automatically by exploiting the optimality conditions of the problem the resulting approach is a particular case of a general scheme whose convergence to stationary points of the constrained minimization problem has been recently proved the effectiveness of the iterative method is validated in several numerical simulations in image reconstruction problems

new projection type methods for monotone lcp with finite termination in this paper we establish two new projection type methods for the solution of monotone linear complementarity problem lcp the methods are a combination of the extragradient method and the newton method in which the active set strategy is used and only one linear system of equations with lower dimension is solved at each iteration it is shown that under the assumption of monotonicity these two methods are globally and linearly convergent furthermore under a nondegeneracy condition they have a finite termination property at last the methods are extended to solving monotone affine variational inequality problem

brief damping of harmonic disturbances in sampled data systems parameterization of all optimal controllers optimal damping of harmonic disturbances of known frequencies is studied for sampled data systems a sampled data output feedback controller is designed to minimize the intersample variations of the controlled variable the set of all stabilizing optimal controllers is obtained in terms of the youla parameterization and a set of interpolation conditions at the disturbance frequencies which ensure that the stationary cost is minimized

sample size selection in optimization methods for machine learning this paper presents a methodology for using varying sample sizes in batch type optimization methods for large scale machine learning problems the first part of the paper deals with the delicate issue of dynamic sample selection in the evaluation of the function and gradient we propose a criterion for increasing the sample size based on variance estimates obtained during the computation of a batch gradient we establish an o 1 epsilon complexity bound on the total cost of a gradient method the second part of the paper describes a practical newton method that uses a smaller sample to compute hessian vector products than to evaluate the function and the gradient and that also employs a dynamic sampling technique the focus of the paper shifts in the third part of the paper to l 1 regularized problems designed to produce sparse solutions we propose a newton like method that consists of two phases a minimalistic gradient projection phase that identifies zero variables and subspace phase that applies a subsampled hessian newton iteration in the free variables numerical tests on speech recognition problems illustrate the performance of the algorithms

least squares problems with inequality constraints as quadratic constraints abstract linear least squares problems with box constraints are commonly solved to find model parameters within bounds based on physical considerations common algorithms include bounded variable least squares bvls and the matlab function lsqlin here the goal is to find solutions to ill posed inverse problems that lie within box constraints to do this we formulate the box constraints as quadratic constraints and solve the corresponding unconstrained regularized least squares problem using box constraints as quadratic constraints is an efficient approach because the optimization problem has a closed form solution the effectiveness of the proposed algorithm is investigated through solving three benchmark problems and one from a hydrological application results are compared with solutions found by lsqlin and the quadratically constrained formulation is solved using the l curve maximum a posteriori estimation map and the 2 regularization method the 2 regularization method with quadratic constraints is the most effective method for solving least squares problems with box constraints

universal regulators for optimal tracking in discrete time systems affected by harmonic disturbances considers the problem of controlling a discrete time linear system by output feedback so as to have a second output z sub t track an observed reference signal r sub t the authors turn to an approach to the tracking problem which allows for damping of other system and control variables and this is our main result the measure of performance is given by a natural quadratic cost function the object is to design an optimal regulator which is universal in the sense that it does not depend on the unknown amplitudes and phases of r sub t and w sub t and is optimal for all choices of r sub t and w sub t the authors prove that an optimal universal regulator exists in a wide class of stabilizing and possibly nonlinear regulators under natural technical conditions and that this regulator is in fact linear provided that the second dimensionality condition above is satisfied on the other hand if it is not satisfied the existence of an optimal universal regulator is not a generic property so as a rule no optimal universal regulator exists

optimal damping of forced oscillations in discrete time systems in this paper we consider a linear discrete time control system affected by an additive sinusoidal disturbance with known frequencies but unknown amplitudes and phases the problem is to damp this forced oscillation in an optimal fashion we show that the natural solution from the point of view of optimal control is neither robust with respect to errors in the frequencies and thus not optimal in practice nor independent of the unknown amplitudes and phases the main result of this paper concerns the existence and design of a realizable robust optimal regulator which is universal in the sense that it does not depend on the unknown amplitudes and phases and is optimal for all choices of such parameters the regulator allows for a considerable degree of design freedom to satisfy other design specifications finally it is shown that this regulator is optimal also for a wide class of stochastic control problems

global and finite termination of a two phase augmented lagrangian filter method for general quadratic programs we present a two phase algorithm for solving large scale quadratic programs qps in the first phase gradient projection iterations approximately minimize a bound constrained augmented lagrangian function and provide an estimate of the optimal active set in the second phase an equality constrained qp defined by the current active set is approximately minimized in order to generate a second order search direction a filter determines the required accuracy of the subproblem solutions and provides an acceptance criterion for the search directions the resulting algorithm is globally and finitely convergent the algorithm is suitable for large scale problems with many degrees of freedom and provides an alternative to interior point methods when iterative methods must be used to solve the underlying linear systems numerical experiments on a subset of the cuter qp test problems demonstrate the effectiveness of the approach

error estimates for regularization methods in hilbert scales in this paper we study regularization methods to reconstruct the solution x of the linear ill posed problem ax y a x to y from noisy data y delta in y the regularization methods are of the general form x alpha delta bar x g alpha b s a a b s a y delta a bar x where b denotes an unbounded self adjoint strictly positive definite operator in the hilbert space x assuming ax sim x a and x bar x p le e for some bar x in x a ge 0 and p ge 0 x r b r 2 x is the norm in a hilbert scale x r r in rnus we derive error estimates which show that the accuracy of the above regularization methods is order optimal provided that the function g alpha lambda the regularization parameter alpha and the parameter s are chosen properly

a fourier approach to nonparametric deconvolution of a density estimate we consider the problem of constructing a nonparametric estimate of a probability density function h from independent random samples of observations from densities a and f when a represents the convolution of h and f our approach is based on truncated fourier inversion in which the truncation point plays the role of a smoothing parameter we derive the asymptotic mean integrated squared error of the estimate and use this formula to suggest a simple practical method for choosing the truncation point from the data strikingly when the smoothing parameter is chosen in this way then in many circumstances the estimator behaves to first order as though the true f were known

inference in components of variance models with low replication in components of variance models the data are viewed as arising through a sum of two random variables representing between and within group variation respectively the former is generally interpreted as a group effect and the latter as error it is assumed that these variables are stochastically independent and that the distributions of the group effect and the error do not vary from one instance to another if each group effect can be replicated a large number of times then standard methods can be used to estimate the distributions of both the group effect and the error this cannot be achieved without replication however how feasible is distribution estimation if it is not possible to replicate prolifically can the distributions of random effects and errors be estimated consistently from a small number of replications of each of a large number of noisy group effects for example in a nonparametric setting often extensive replication is practically infeasible in particular if inherently small numbers of individuals exhibit any given group effect yet it is quite unclear how to conduct inference in this case we show that inference is possible even if the number of replications is as small as 2 two methods are proposed both based on fourier inversion one which is substantially more computer intensive than the other exhibits better performance in numerical experiments

solving karush kuhn tucker systems via the trust region and the conjugate gradient methods a popular approach to solving the karush kuhn tucker kkt system mainly arising from the variational inequality problem is to reformulate it as a constrained minimization problem with simple bounds in this paper we propose a trust region method for solving the reformulation problem with the trust region subproblems being solved by the truncated conjugate gradient cg method which is cost effective other advantages of the proposed method over existing ones include the fact that a good approximated solution to the trust region subproblem can be found by the truncated cg method and is judged in a simple way also the working matrix in each iteration is h instead of the condensed hth where h is a matrix element of the generalized jacobian of the function used in the reformulation as a matter of fact the matrix used is of reduced dimension we pay extra attention to ensure the success of the truncated cg method as well as the feasibility of the iterates with respect to the simple constraints another feature of the proposed method is that we allow the merit function value to be increased at some iterations to speed up the convergence global and superlinear quadratic convergence is shown under standard assumptions numerical results are reported on a subset of problems from the mcplib collection s p dirkse and m c ferris optim methods softw 5 1995 pp 319 345

a matrix free approach to build band preconditioners for large scale bound constrained optimization abstract we propose a procedure for building symmetric positive definite band preconditioners for large scale symmetric possibly indefinite linear systems when the coefficient matrix is not explicitly available but matrix vector products involving it can be computed we focus on linear systems arising in newton type iterations within matrix free versions of projected methods for bound constrained nonlinear optimization in this case the structure and the size of the matrix may significantly change in subsequent iterations and preconditioner updating algorithms that exploit information from previous steps cannot be easily applied our procedure is based on a recursive approach that incrementally improves the quality of the preconditioner while requiring a modest number of matrix vector products a strategy for dynamically choosing the bandwidth of the preconditioners is also presented numerical results are provided showing the performance of our preconditioning technique within a trust region newton method for bound constrained optimization

statistical inverse estimation in hilbert scales the recovery of signals from indirect measurements blurred by random noise is considered under the assumption that prior knowledge regarding the smoothness of the signal is avialable for greater flexibility the general problem is embedded in an abstract hilbert scale in the applications sobolev scales are used for the construction of estimators we employ preconditioning along with regularized operator inversion in the appropriate inner product where the operator is bounded but not necessarily compact a lower bound to certain minimax rates is included and it is shown that in generic examples the proposed estimators attain the asymptotic minimax rate examples include errors in variables deconvolution and indirect nonparametric regression special instances of the latter are estimation of the source term in a differential equation and the estimation of the initial state in the heat equation

variational inequality approach to enforcing the non negative constraint for advection diffusion equations abstract predictive simulations are crucial for the success of many subsurface applications and it is highly desirable to obtain accurate non negative solutions for transport equations in these numerical simulations to this end optimization based methodologies based on quadratic programming qp have been shown to be a viable approach to ensuring discrete maximum principles and the non negative constraint for anisotropic diffusion equations in this paper we propose a computational framework based on the variational inequality vi which can also be used to enforce important mathematical properties e g maximum principles and physical constraints e g the non negative constraint we demonstrate that this framework is not only applicable to diffusion equations but also to non symmetric advection diffusion equations an attractive feature of the proposed framework is that it works with any weak formulation for the advection diffusion equations including single field formulations which are computationally attractive a particular emphasis is placed on the parallel and algorithmic performance of the vi approach across large scale and heterogeneous problems it is also shown that qp and vi are equivalent under certain conditions state of the art qp and vi solvers available from the petsc library are used on a variety of steady state 2d and 3d benchmarks and a comparative study on the scalability between the qp and vi solvers is presented we then extend the proposed framework to transient problems by simulating the miscible displacement of fluids in a heterogeneous porous medium and illustrate the importance of enforcing maximum principles for these types of coupled problems our numerical experiments indicate that vis are indeed a viable approach for enforcing the maximum principles and the non negative constraint in a large scale computing environment also provided are firedrake project files as well as a discussion on the computer implementation to help facilitate readers in understanding the proposed framework

deconvolution with unknown error distribution we consider the problem of estimating a density fx using a sample y1 yn from fy fx fe where fe is an unknown density we assume that an additional sample e1 em from fe is observed estimators of fx and its derivatives are constructed by using nonparametric estimators of fy and fe and by applying a spectral cut off in the fourier domain we derive the rate of convergence of the estimators in case of a known and unknown error density fe where it is assumed that fx satisfies a polynomial logarithmic or general source condition it is shown that the proposed estimators are asymptotically optimal in a minimax sense in the models with known or unknown error density if the density fx belongs to a sobolev space h mathcal p and fe is ordinary smooth or supersmooth

trust region newton method for logistic regression large scale logistic regression arises in many applications such as document classification and natural language processing in this paper we apply a trust region newton method to maximize the log likelihood of the logistic regression model the proposed method uses only approximate newton steps in the beginning but achieves fast convergence in the end experiments show that it is faster than the commonly used quasi newton approach for logistic regression we also extend the proposed method to large scale l2 loss linear support vector machines svm

scalable nonlinear programming via exact differentiable penalty functions and trust region newton methods we present an approach for nonlinear programming based on the direct minimization of an exact differentiable penalty function using trust region newton techniques the approach provides desirable features required for scalability it can detect and exploit directions of negative curvature it is superlinearly convergent and it enables the scalable computation of the newton step through iterative linear algebra moreover it presents features that are desirable for parametric optimization problems that must be solved in a latency limited environment as is the case for model predictive control and mixed integer nonlinear programming these features are fast detection of activity efficient warm starting and progress on a primal dual merit function at every iteration we note that other algorithmic approaches fail to satisfy at least one of these features we derive general convergence results for our approach and demonstrate its behavior through numerical studies

an alternating projected gradient algorithm for nonnegative matrix factorization abstract due to the extensive applications of nonnegative matrix factorizations nmfs of nonnegative matrices such as in image processing text mining spectral data analysis speech processing etc algorithms for nmf have been studied for years in this paper we propose a new algorithm for nmf which is based on an alternating projected gradient apg approach in particular no zero entries appear in denominators in our algorithm which implies no breakdown occurs and even if some zero entries appear in numerators new updates can always be improved in our algorithm it is shown that the effect of our algorithm is better than that of lee and seung s algorithm when we do numerical experiments on two known facial databases and one iris database

an active set quasi newton method with projected search for bound constrained minimization we analyze an active set quasi newton method for large scale bound constrained problems our approach combines the accurate active set identification function and the projected search both of these strategies permit fast change in the working set the limited memory method is employed to update the inactive variables while the active variables are updated by simple rules a further division of the active set enables the global convergence of the new algorithm numerical tests demonstrate the efficiency and performance of the present strategy and its comparison with some existing active set strategies

a note on the usefulness of superkernels in density estimation we consider the akaike parzen rosenblatt density estimate f nh based upon any superkernel l i e an absolutely integrable function with int l 1 whose characteristic function is 1 on lbrack 1 1 rbrack and compare it with a kernel estimate g nh based upon an arbitrary kernel k we show that for a given subclass of analytic densities inf l sup k lim sup n rightarrow infty frac inf h mathbb e int f nh f inf h mathbb e int g nh f 1 where h 0 is the smoothing factor thus asymptotically the class of superkernels is as good as any other class of kernels when certain analytic densities are estimated we also obtain exact asymptotic expressions for the expected l 1 error of the kernel estimate when superkernels are used

analysis of generalized pattern searches this paper contains a new convergence analysis for the lewis and torczon generalized pattern search gps class of methods for unconstrained and linearly constrained optimization this analysis is motivated by a desire to understand the successful behavior of the algorithm under hypotheses that are satisfied by many practical problems specifically even if the objective function is discontinuous or extended valued the methods find a limit point with some minimizing properties simple examples show that the strength of the optimality conditions at a limit point depends not only on the algorithm but also on the directions it uses and on the smoothness of the objective at the limit point in question the contribution of this paper is to provide a simple convergence analysis that supplies detail about the relation of optimality conditions to objective smoothness properties and to the defining directions for the algorithm and it gives previous results as corollaries

a new subspace limited memory bfgs algorithm for large scale bound constrained optimization an active set limited memory bfgs algorithm for large scale bound constrained optimization is introduced the active sets are based on guessing technique to be identified at each iteration the search direction in free subspace is determined by limited memory bfgs l bfgs algorithm which provides an efficient means for attacking large scale optimization problems the implementations of the method on cute test problems are described

a framework for the upscaling of the electrical conductivity in the quasi static maxwells equations electromagnetic simulations of complex geologic settings are computationally expensive one reason for this is the fact that a fine mesh is required to accurately discretize the electrical conductivity model of a given setting this conductivity model may vary over several orders of magnitude and these variations can occur over a large range of length scales using a very fine mesh for the discretization of this setting leads to the necessity to solve a large system of equations that is often difficult to deal with to keep the simulations computationally tractable coarse meshes are often employed for the discretization of the model such coarse meshes typically fail to capture the fine scale variations in the conductivity model resulting in inaccuracies in the predicted data in this work we introduce a framework for constructing a coarse mesh or upscaled conductivity model based on a prescribed fine mesh model rather than using analytical expressions we opt to pose upscaling as a parameter estimation problem by solving an optimization problem we obtain a coarse mesh conductivity model the optimization criterion can be tailored to the survey setting in order to produce coarse models that accurately reproduce the predicted data generated on the fine mesh this allows us to upscale arbitrary conductivity structures as well as to better understand the meaning of the upscaled quantity we use 1d and 3d examples to demonstrate that the proposed framework is able to emulate the behavior of the heterogeneity in the fine mesh conductivity model and to produce an accurate description of the desired predicted data obtained by using a coarse mesh in the simulation process

projected newton methods for optimization problems with simple constraints we consider the problem min f x x geqq 0 and propose algorithms of the form x k 1 x k alpha k d k nabla f x k where cdot denotes projection on the positive orthant alpha k is a stepsize chosen by an armijo like rule and d k is a positive definite symmetric matrix which is partly diagonal we show that d k can be calculated simply on the basis of second derivatives of f so that the resulting newton like algorithm has a typically superlinear rate of convergence with other choices of d k convergence at a typically linear rate is obtained the algorithms are almost as simple as their unconstrained counterparts they are well suited for problems of large dimension such as those arising in optimal control while being competitive with existing methods for low dimensional problems the effectiveness of the newton like algorithm is demonstrated via computational examples involving as many as 10 000 variables extensions to general linearly constrained pr

robust stabilization of uncertain linear systems quadratic stabilizability and h sup infinity control theory the problem of robustly stabilizing a linear uncertain system is considered with emphasis on the interplay between the time domain results on the quadratic stabilization of uncertain systems and the frequency domain results on h sup infinity optimization a complete solution to a certain quadratic stabilization problem in which uncertainty enters both the state and the input matrices of the system is given relations between these robust stabilization problems and h sup infinity control theory are explored it is also shown that in a number of cases if a robust stabilization problem can be solved via lyapunov methods then it can be also be solved via h sup infinity control theory based methods

globally and superlinearly convergent algorithms for the solution of box constrained optimization abstract based on the identification technique of active constraints we propose a newton like algorithm and a quasi newton algorithm for solving the box constrained optimization problem the two algorithms require only the solution of a lower dimensional system of linear equations at each iteration in the proposed quasi newton algorithm we make use of an approximate direction derivative of the multiplier functions so that only first order derivatives of the objective function are needed to evaluate under mild assumptions global convergence of the two algorithms is established in particular locally quadratic convergence for the newton like algorithm and locally superlinear convergence for the quasi newton algorithm are obtained without assuming that the strict complementarity condition holds at the solution

superlinear and quadratic convergence of affine scaling interior point newton methods for problems with simple bounds without strict complementarity assumption a class of affine scaling interior point methods for bound constrained optimization problems is introduced which are locally q superlinear or q quadratic convergent it is assumed that the strong second order sufficient optimality conditions at the solution are satisfied but strict complementarity is not required the methods are modifications of the affine scaling interior point newton methods introduced by t f coleman and y li math programming 67 189 224 1994 there are two modifications one is a modification of the scaling matrix the other one is the use of a projection of the step to maintain strict feasibility rather than a simple scaling of the step a comprehensive local convergence analysis is given a simple example is presented to illustrate the pitfalls of the original approach by coleman and li in the degenerate case and to demonstrate the performance of the fast converging modifications developed in this paper

nonparametric density estimation from covariate information an increasing number of statistical problems arise in connection with functional calibration in each case inexpensive indirect data in a particular context are combined with direct expensive to acquire data from different but related settings to estimate quantities in the former case sometimes e g in chemometrics problems where spectroscopic calibration is used the indirect data are functional but more commonly they are scalar or vector valued and the functional component is the quantity that we wish to estimate the problem treated here is of the latter type we observe data that give us access to the distribution of u given v and from these and data on u we wish to estimate the density of v the motivating real datasets are of age and covariate information in fish populations we suggest two methodologies each of which is based on transforming the problem to one involving inversion of a symmetric linear operator our techniques have connections to methods for functional data analysis and fo