new algorithms for singly linearly constrained quadratic programs subject to lower and upper bounds there are many applications related to singly linearly constrained quadratic programs subjected to upper and lower bounds in this paper a new algorithm based on secant approximation is provided for the case in which the hessian matrix is diagonal and positive definite to deal with the general case where the hessian is not diagonal a new efficient projected gradient algorithm is proposed the basic features of the projected gradient algorithm are 1 a new formula is used for the stepsize 2 a recently established adaptive non monotone line search is incorporated and 3 the optimal stepsize is determined by quadratic interpolation if the non monotone line search criterion fails to be satisfied numerical experiments on large scale random test problems and some medium scale quadratic programs arising in the training of support vector machines demonstrate the usefulness of these algorithms

benchmarking optimization software with performance profiles we propose performance profiles distribution functions for a performance metric as a tool for benchmarking and comparing optimization software we show that performance profiles combine the best features of other tools for performance evaluation

box constrained quadratic programming with proportioning and projections two new closely related concepts are introduced that depend on a positive constant gamma an iteration is proportional if the norm of violation of the kuhn tucker conditions at active variables does not excessively exceed the norm of the part of the gradient that corresponds to free variables while a progressive direction determines a descent direction that enables the released variables to move far enough from the boundary in a step called proportioning an algorithm that uses the conjugate gradient method to explore the face of the region defined by the current iterate until a disproportional iteration is generated is proposed it then changes the face by means of the progressive direction it is proved that for strictly convex problems the proportioning is a spacer iteration so that the algorithm converges to the solution if the solution is nondegenerate then the algorithm finds the solution in a finite number of steps moreover a simple lower bound on gamma is given to ensure finite termination even for problems with degenerate solutions the theory covers a class of algorithms allowing many constraints to be added or dropped at a time and accepting approximate solutions of auxiliary problems preliminary numerical results are promising

a note on n estimators for the binomial distribution abstract consider k success counts from a binomial distribution with unknown n and success probability p we examine the problem of estimating n by integrating the likelihood for n and p over a beta density for p we obtain the beta binomial distribution resulting in stable and reasonably efficient estimators of n which compare favorably with and are often better than the estimates introduced by olkin et al 1981

a unified treatment of integer parameter models abstract it is shown that there is an analog to the score function the derivative of the log likelihood with respect to the parameter here called the difference score which plays an important theoretical role in integer parameter models a unified treatment of integer parameter models can be obtained by recognizing that many commonly used models have a linear difference score lds it is established that many of the exact results available for exponential family models also hold for lds models in particular we find an information lower bound show the uniqueness of the maximum likelihood estimator establish monotone likelihood ratio and establish optimality properties in two sided tests it is also shown that if one has a multiparameter likelihood in which there is a conditional likelihood depending on n alone then these exact results apply there as well it is also established when and why the usual likelihood methodology will apply to a class of nonstandard models in which the natural approxima

stopping rules and backward error analysis for bound constrained optimization termination criteria for the iterative solution of bound constrained optimization problems are examined in the light of backward error analysis it is shown that the problem of determining a suitable perturbation on the problem s data corresponding to the definition of the backward error is analytically solvable under mild assumptions moreover a link between existing termination criteria and this solution is clarified indicating that some standard measures of criticality may be interpreted in the sense of backward error analysis the backward error problem is finally considered from the multicriteria optimization point of view and some numerical illustration is provided

stability criteria of linear neutral systems with multiple delays in the present note stability of linear neutral systems with multiple delays is investigated by principle of the argument stability criteria are derived which are necessary and sufficient for asymptotic stability of the neutral systems the stability criteria need only to evaluate the characteristic function and the argument on the boundary of a certain rectangular region several numerical examples are given to illustrate the results

incomplete cholesky factorizations with limited memory we propose an incomplete cholesky factorization for the solution of large scale trust region subproblems and positive definite systems of linear equations this factorization depends on a parameter p that specifies the amount of additional memory in multiples of n the dimension of the problem that is available there is no need to specify a drop tolerance our numerical results show that the number of conjugate gradient iterations and the computing time are reduced dramatically for small values of p we also show that in contrast with drop tolerance strategies the new approach is more stable in terms of number of iterations and memory requirements

a practical relative error criterion for augmented lagrangians this paper develops a new error criterion for the approximate minimization of augmented lagrangian subproblems this criterion is practical since it is readily testable given only a gradient or subgradient of the augmented lagrangian it is also relative in the sense of relative error criteria for proximal point algorithms in particular it uses a single relative tolerance parameter rather than a summable parameter sequence our analysis first describes an abstract version of the criterion within rockafellar s general parametric convex duality framework and proves a global convergence result for the resulting algorithm specializing this algorithm to a standard formulation of convex programming produces a version of the classical augmented lagrangian method with a novel inexact solution condition for the subproblems finally we present computational results drawn from the cute test set including many nonconvex problems indicating that the approach works well in practice

novel delay dependent kalman luenberger type filter design for neutral systems this article presents a new delay dependent condition to design a kalman luenberger type filter for neutral systems where the resulting filter is such that a prescribed level is guaranteed the proposed approach is based on linear matrix inequalities and on lyapunov krasovskii theory also the gu s discretisation method and a slack matrix technique that decouples the system matrices from the lyapunov ones are applied in order to obtain less conservative results than the similar ones two numerical examples are performed to illustrate the advantage of the method proposed

modeling and optimization of an ammonia reactor using a penalty like method abstract ammonia is one of the most important chemicals produced in the world due to its broad applicability modeling and simulation of process of ammonia has received considerable attention among the industrial processes ammonia is produced from the reaction of hydrogen and nitrogen at high temperature and high pressure along with a catalyst its production depends on temperature of feed gas at the top of the reactor the partial pressures of the reactants and the reactor length the optimal design problem requires obtaining the optimal reactor length with maximum economic return subject to a number of equality constraints involving solution of coupled nonlinear differential equations a more efficient algorithm for the solution of optimization constrained differential equations is proposed by coupling numerical techniques barrier function with a direct search method with runge kutta method the global convergence of the proposed algorithm is established under some usual conditions numerical results showed that the resulting algorithm based only on function values is highly competitive with other global optimization methods

augmented lagrangians with adaptive precision control for quadratic programming with simple bounds and equality constraints in this paper we discuss a specialization of the augmented lagrangian type algorithm of conn gould and toint to the solution of strictly convex quadratic programming problems with simple bounds and equality constraints the new feature of the presented algorithm is the adaptive precision control of the solution of auxiliary problems in the inner loop of the basic algorithm which yields a rate of convergence that does not have any term that accounts for inexact solution of auxiliary problems moreover boundedness of the penalty parameter is achieved for the precision control used numerical experiments illustrate the efficiency of the presented algorithm and encourage its usage

a decomposition method for large scale box constrained optimization a decomposition method for solving large scale box constrained optimization is proposed the algorithm is motivated by the successful use of the decomposition method presented by joachims for training support vector machines in particular a new technique based on the new definition kkt violating index is introduced for working set identification finally the numerical experiments and implementation details show that this method is practical for large scale problems

non monotone trust region algorithms for nonlinear optimization subject to convex constraints this paper presents two new trust region methods for solving nonlinear optimization problems over convex feasible domains these methods are distinguished by the fact that they do not enforce strict monotonicity of the objective function values at successive iterates the algorithms are proved to be convergent to critical points of the problem from any starting point extensive numerical experiments show that this approach is competitive with the lancelot package

modified subspace limited memory bfgs algorithm for large scale bound constrained optimization in this paper a subspace limited memory bfgs algorithm for solving large scale bound constrained optimization problems is developed it is modifications of the subspace limited memory quasi newton method proposed by ni and yuan q ni y x yuan a subspace limited memory quasi newton algorithm for large scale nonlinear bound constrained optimization math comput 66 1997 1509 1520 an important property of our proposed method is that more limited memory bfgs update is used under appropriate conditions the global convergence of the method is established the implementations of the method on cute test problems are presented which indicate the modifications are beneficial to the performance of the algorithm

an affine scaling interior point cbb method for box constrained optimization we develop an affine scaling algorithm for box constrained optimization which has the property that each iterate is a scaled cyclic barzilai borwein cbb gradient iterate that lies in the interior of the feasible set global convergence is established for a nonmonotone line search while there is local r linear convergence at a nondegenerate local minimizer where the second order sufficient optimality conditions are satisfied numerical experiments show that the convergence speed is insensitive to problem conditioning the algorithm is particularly well suited for image restoration problems which arise in positron emission tomography where the cost function can be infinite on the boundary of the feasible set

inference for exponential order statistic models based on an integrated likelihood function abstract methods of statistical inference are developed for the exponential order statistic eos model where only a subset of order statistics from a collection of n iid exponential random detection times is observable when the rate parameter for detections is unknown the maximum likelihood estimator mle of the unknown integer parameter n can be infinite with substantial probability inference for n is developed using a pseudolikelihood function obtained by integrating out the rate parameter the estimator that maximizes this function called the integrated likelihood estimator ile is shown to be finite and to have better sampling properties than the mle parameter based asymptotics are developed for the case where n is large application of the methodology is illustrated using two datasets

bayesian analyses for a multiple capture recapture model summary in multiple capture recapture surveys the probability of capture can vary between sampling occasions the model accounting for this variation is known as at bayes empirical bayes and bayes empirical bayes solutions are given to the problems of interval estimation decision making and point estimation of the population size n when the number of sampling occasions is small to moderate and the number of recaptured units observed on each sampling occasion is moderate estimates obtained from empirical bayes and bayes empirical bayes methods compare closely to bayesian methods using a reference prior distribution for the capture probabilities however when the number of sampling occasions is large and the number of recaptured units observed on each sampling occasion is small inferences obtained using different reference priors can differ considerably

a new conjugate gradient method with guaranteed descent and an efficient line search a new nonlinear conjugate gradient method and an associated implementation based on an inexact line search are proposed and analyzed with exact line search our method reduces to a nonlinear version of the hestenes stiefel conjugate gradient scheme for any inexact line search our scheme satisfies the descent condition bf g k sf t bf d k le frac 7 8 bf g k 2 moreover a global convergence result is established when the line search fulfills the wolfe conditions a new line search scheme is developed that is efficient and highly accurate efficiency is achieved by exploiting properties of linear interpolants in a neighborhood of a local minimizer high accuracy is achieved by using a convergence criterion which we call the approximate wolfe conditions obtained by replacing the sufficient decrease criterion in the wolfe conditions with an approximation that can be evaluated with greater precision in a neighborhood of a local minimum than the usual sufficient decrease criterion numerical comparisons are given with both l bfgs and conjugate gradient methods using the unconstrained optimization problems in the cute library

on delay dependent stability under model transformations of some neutral linear systems this paper focuses on the stability analysis of a class of neutral linear systems some sufficient delay dependent stability conditions are derived in terms of an appropriate model transformation of the original system transformation similar to the one used in the retarded case the analysis makes use of an appropriate construction of a degenerate lyapunov krasovskii functional several computational schemes and frequency domain interpretations as well as an example are also presented

global convergence of a class of trust region algorithms for optimization with simple bounds this paper extends the known excellent global convergence properties of trust region algorithms for unconstrained optimization to the case where bounds on the variables are present weak conditions on the accuracy of the hessian approximations are considered it is also shown that when the strict complementarily condition holds the proposed algorithms reduce to an unconstrained calculation after finitely many iterations allowing a fast asymptotic rate of convergence

on the convergence of interior reflective newton methods for nonlinear minimization subject to bounds we consider a new algorithm an interior reflective newton approach for the problem of minimizing a smooth nonlinear function of many variables subject to upper and or lower bounds on some of the variables this approach generatesstrictly feasible iterates by using a new affine scaling transformation and following piecewise linear paths reflection paths the interior reflective approach does not require identification of an activity set in this paper we establish that the interior reflective newton approach is globally and quadratically convergent moreover we develop a specific example of interior reflective newton methods which can be used for large scale and sparse problems

a new active set algorithm for box constrained optimization an active set algorithm asa for box constrained optimization is developed the algorithm consists of a nonmonotone gradient projection step an unconstrained optimization step and a set of rules for branching between the two steps global convergence to a stationary point is established for a nondegenerate stationary point the algorithm eventually reduces to unconstrained optimization without restarts similarly for a degenerate stationary point where the strong second order sufficient optimality condition holds the algorithm eventually reduces to unconstrained optimization without restarts a specific implementation of the asa is given which exploits the recently developed cyclic barzilai borwein cbb algorithm for the gradient projection step and the recently developed conjugate gradient algorithm cg descent for unconstrained optimization numerical experiments are presented using box constrained problems in the cuter and minpack 2 test problem libraries

inadmissibility of the maximum likelihood estimator in the sequential estimation of the size of a population summary for the sequential estimation of the size of a finite population the maximum likelihood estimator is a commonly used estimator but has no explicit expression in this paper it is shown that the maximum likelihood estimator is inadmissible an estimator which is simple in form and dominates the maximum likelihood estimator asymptotically is proposed to study the optimality property of the estimator for the stopping rule sb from a decision theoretical approach we investigate the admissibility of the estimator under a general weighted squared error loss function although the maximum likelihood estimator and its risk have no closed form expression the estimator will be shown to be inadmissible by means of a general theorem that we establish later as theorem 2 1 in this paper we only consider the simple capture capture model discussed by samuel 1968 more complex and realistic capture recapture models have been developed subsequently pollack otto 1983 pickands raghavachari 1987 bonett woodward bentler 1987 cormack 1989 huggins 1989 we conjecture that many of the

a morley finite element method for the displacement obstacle problem of clamped kirchhoff plates abstract we study a morley finite element method for the displacement obstacle problem of clamped kirchhoff plates on polygonal domains error estimates are derived in the energy norm and the l norm the performance of the method is illustrated by numerical experiments

inference for the binomial n parameter a hierarchical bayes approach summary a hierarchical bayes approach to the problem of estimating n in the binomial distribution is presented this provides a simple and flexible way of specifying prior information and also allows a convenient representation of vague prior knowledge it yields solutions to the problems of interval estimation prediction and decision making as well as that of point estimation the bayes estimator compares favourably with the best previously proposed point estimators in the literature unknown parameters n and 0 most of the literature about statistical analysis of this model has focused on point estimation of n while interval estimation prediction and decision making have been little considered see 2 i adopt a hierarchical bayes approach this provides a simple way of specifying prior information and also allows a convenient representation of vague prior knowledge using limiting improper prior forms it leads to solutions of the problems of interval estimation prediction and decision making as well as that of point estimation a difficulty with bayesian analysis of this problem has been the absence of a sufficiently flexible and tractable family of prior distributions mainly due to the fact that n is an integer the present approach gets around this by first assuming that n has a poisson distribution the resulting hyperparameters are then continuous valued and one may use existing results about conjugate and vague priors in better understood settings

the barzilai and borwein gradient method for the large scale unconstrained minimization problem the barzilai and borwein gradient method for the solution of large scale unconstrained minimization problems is considered this method requires few storage locations and very inexpensive computations furthermore it does not guarantee descent in the objective function and no line search is required recently the global convergence for the convex quadratic case has been established however for the nonquadratic case the method needs to be incorporated in a globalization scheme in this work a nonmonotone line search strategy that guarantees global convergence is combined with the barzilai and borwein method this strategy is based on the nonmonotone line search technique proposed by grippo lampariello and lucidi siam j numer anal 23 1986 pp 707 716 numerical results to compare the behavior of this method with recent implementations of the conjugate gradient method are presented these results indicate that the global barzilai and borwein method may allow some significant reduction in the number of line searches and also in the number of gradient evaluations

an assessment of nonmonotone linesearch techniques for unconstrained optimization the purpose of this paper is to discuss the potential of nonmonotone techniques for enforcing convergence of unconstrained minimization algorithms from starting points distant from the solution linesearch based algorithms are considered for both small and large problems and extensive numerical experiments show that this potential is sometimes considerable a new variant is introduced in order to limit some of the identified drawbacks of the existing techniques this variant is again numerically tested and appears to be competitive finally the impact of preconditioning on the considered methods is examined

improved robust ℋ control for neutral systems via discretised lyapunov krasovskii functional this paper deals with the robust stability analysis and robust control synthesis with guaranteed performance index for time invariant uncertain neutral systems based on linear matrix inequalities lmis the novelty is to extend the discretisation technique introduced by k gu for an appropriate parameter dependent lyapunov krasovskii functional then the approach proposed improves the previous criteria from the literature that treats robust stability and control for uncertain neutral systems the main strategy used to derive the obtained conditions is to introduce slack variables allowing one to decouple the system matrices from the lyapunov functional matrices without including additional dynamics two numerical examples are performed to support the theoretical predictions the first example deals with stability analysis and the other illustrates how the proposed approach may provide a better solution for the guaranteed cost control problem when compared with others available in the literature

estimation of the optical constants and the thickness of thin films using unconstrained optimization the problem of estimating the thickness and the optical constants of thin films using transmission data only is very challenging from the mathematical point of view and has a technological and an economic importance in many cases it represents a very ill conditioned inverse problem with many local nonglobal solutions in a recent publication we proposed nonlinear programming models for solving this problem well known software for linearly constrained optimization was used with success for this purpose in this paper we introduce an unconstrained formulation of the nonlinear programming model and we solve the estimation problem using a method based on repeated calls to a recently introduced unconstrained minimization algorithm numerical experiments on computer generated films show that the new procedure is reliable

improved convergence rates for lasserre type hierarchies of upper bounds for box constrained polynomial optimization we consider the problem of minimizing a given n variate polynomial f over the hypercube 1 1 n an idea introduced by lasserre is to find a probability distribution on 1 1 n with polynomial density function h of given degree r that minimizes the expectation int 1 1 n f x h x d mu x where d mu x is a fixed finite borel measure supported on 1 1 n it is known that for the lebesgue measure d mu x dx one may show an error bound o 1 sqrt r if h is a sum of squares density and an o 1 r error bound if h is the density of a beta distribution in this paper we show an error bound of o 1 r 2 if d mu x left prod i 1 n sqrt 1 x i 2 right 1 the well known measure in the study of orthogonal polynomials and h has a schmudgen type representation with respect to 1 1 n which is a more general condition than a sum of squares the convergence rate analysis relies on the theory of polynomial kernels and in particular on jackson kerne

an active set newton algorithm for large scale nonlinear programs with box constraints a new algorithm for large scale nonlinear programs with box constraints is introduced the algorithm is based on an efficient identification technique of the active set at the solution and on a nonmonotone stabilization technique it possesses global and superlinear convergence properties under standard assumptions a new technique for generating test problems with known characteristics is also introduced the implementation of the method is described along with computational results for large scale problems

an active set truncated newton method for large scale bound constrained optimization an active set truncated newton method for large scale bound constrained optimization is proposed the active sets are guessed by an identification technique the search direction consists of two parts some of the components are simply defined the other components are determined by the truncated newton method the method based on a nonmonotone line search technique is shown to be globally convergent numerical experiments are presented using bound constrained problems in the cuter test problem library the numerical performance reveals that our method is effective and competitive with the famous algorithm tron

convergence properties of trust region methods for linear and convex constraints we develop a convergence theory for convex and linearly constrained trust region methods which only requires that the step between iterates produce a sufficient reduction in the trust region subproblem global convergence is established for general convex constraints while the local analysis is for linearly constrained problems the main local result establishes that if the sequence converges to a nondegenerate stationary point then the active constraints at the solution are identified in a finite number of iterations as a consequence of the identification properties we develop rate of convergence results by assuming that the step is a truncated newton method our development is mainly geometrical this approach allows the development of a convergence theory without any linear independence assumptions

newton s method for large bound constrained optimization problems we analyze a trust region version of newton s method for bound constrained problems our approach relies on the geometry of the feasible set not on the particular representation in terms of constraints the convergence theory holds for linearly constrained problems and yields global and superlinear convergence without assuming either strict complementarity or linear independence of the active constraints we also show that the convergence theory leads to an efficient implementation for large bound constrained problems

augmented lagrangian method within l shaped method for stochastic linear programs stochastic programming is an optimization technique used in the presence of uncertainty and it typically leads to very large problem sizes in this paper a modified version of the l shaped method was used to solve two stage stochastic linear programs with recourse based on the projection method and the augmented lagrangian method using this modified version of the l shaped method allows us to reduce the number of iterations and the time of solving a two stage stochastic linear program with fixed recourse in comparison with traditional methods

on the erratic behavior of estimators of n in the binomial n p distribution abstract the problem of estimating the number of trials in a binomial distribution is known to be particularly difficult in certain circumstances both the maximum likelihood and method of moments estimators can produce very erratic results being extremely sensitive to minor fluctuations in the data but existing asymptotic theory for these estimators does not provide any insight into this peculiar behavior as the theory typically produces asymptotic normal distributions with respectable means and variances likewise the limiting distributions of other estimators do not convey much information about why those estimators might perform better than their more classical counterparts in this article we develop an alternative asymptotic theory that clearly and vividly depicts not only the erratic performance of the maximum likelihood and method of moments estimators but also the more regular robust properties of alternative estimators proposed by carroll and lombard we note that the parameter configurati

a class of methods for solving large convex quadratic programs subject to box constraints in this paper we analyze conjugate gradient type algorithms for solving convex quadratic programs subject only to box constraints i e lower and upper bounds on the variables programs of this type which we denote by bqp play an important role in many optimization models and algorithms we propose a new class of finite algorithms based on a nonfinite heuristic for solving a large sparse bqp the numerical results suggest that these algorithms are competitive with dembo and tulowitzski s 1983 crgp algorithm in general and perform better than crgp for problems that have a low percentage of free variables at optimality and for problems with only nonnegativity constraints

a new solver for the elastic normal contact problem using conjugate gradients deflation and an fft based preconditioner this paper presents our new solver bccg fai for solving elastic normal contact problems this is a comprehensible approach that is based on the conjugate gradients cg algorithm and that uses ffts a first novel aspect is the definition of the fft based approximate inverse preconditioner the underlying idea is that the inverse matrix can be approximated well using a toeplitz or block toeplitz form which can be computed using the fft of the original matrix elements this preconditioner makes the total number of cg iterations effectively constant in 2d and very slowly increasing in 3d problems a second novelty is how we deal with a prescribed total force this uses a deflation technique in such a way that cgs convergence and finite termination properties are maintained numerical results show that this solver is more effective than existing cg based strategies such that it can compete with multi grid strategies over a much larger problem range in our opinion it could be the new method of choice because of its simple structure and elegant theory and because robust performance is achieved independently of any problem specific parameters we present a new cg and fft based solver for the elastic normal contact problem using ffts we construct an effective preconditioner with block toeplitz form o 1 iterations are needed in 2d problems as predicted by theory a prescribed total force is dealt with by ignoring the average using deflation the performance of the resulting algorithm is competitive with multi grid strategies

bayesian methods for prevalence estimation from incomplete administrative lists many studies aimed at estimating prevalence use several administrative lists from different sources in an attempt to enumerate all persons affected with the health condition of interest each list is incomplete in the sense that none of them enumerates all persons affected with the health condition further because the lists are drawn from different administrative sources the probability of enumeration varies from list to list the goal is to use information from the lists to estimate the total number of affected persons in the population but with some accounting for the different but unknown probabilities of enumeration on each list this paper presents a bayesian method to estimate prevalence when the probability of enumeration varies from list to list data from a survey of children with spina bifida illustrate the methodology

a fast algorithm for sparse reconstruction based on shrinkage subspace optimization and continuation we propose a fast algorithm for solving the ell 1 regularized minimization problem min x in mathbb r n mu x 1 ax b 2 2 for recovering sparse solutions to an undetermined system of linear equations ax b the algorithm is divided into two stages that are performed repeatedly in the first stage a first order iterative shrinkage method yields an estimate of the subset of components of x likely to be nonzero in an optimal solution restricting the decision variables x to this subset and fixing their signs at their current values reduces the ell 1 norm x 1 to a linear function of x the resulting subspace problem which involves the minimization of a smaller and smooth quadratic function is solved in the second phase our code fpc as embeds this basic two stage algorithm in a continuation homotopy approach by assigning a decreasing sequence of values to mu this code exhibits state of the art performance in terms of both its speed and its ability to recover sparse signals

an active set quasi newton method with projected search for bound constrained minimization we analyze an active set quasi newton method for large scale bound constrained problems our approach combines the accurate active set identification function and the projected search both of these strategies permit fast change in the working set the limited memory method is employed to update the inactive variables while the active variables are updated by simple rules a further division of the active set enables the global convergence of the new algorithm numerical tests demonstrate the efficiency and performance of the present strategy and its comparison with some existing active set strategies

technical communique robust stability of uncertain delay differential systems of neutral type this paper focuses on the stability problem for neutral delay differential systems with norm bounded and possibly time varying uncertainty the time delay is assumed constant and known a stability criterion that is formulated in a linear matrix inequality lmi form is derived numerical examples show that the results obtained by this new criterion significantly improve the estimate of stability limit over some existing results in the literature

a new active set method for nonnegative matrix factorization nonnegative matrix factorization nmf is powerful in data analysis by recovering hidden features from redundant data recently various extensions of nmf have been proposed to further enhance the classification results in this paper we present a novel proximal alternating nonnegative least squares panls framework for nmf we show the globally convergent property that any accumulation point of panls is a stationary point of nmf a new active set method based on the framework is proposed to solve each subproblem which reduces to unconstrained optimization after finite steps and converges to the unique minimizer of the subproblem the panls framework and the associated new active set method can be easily extended to variants of nmf extensive computational experiments using both synthetic data and real data show that our proposed method is attractive in terms of computational speed the solution quality and the flexibility to extensions of nmf

on computing the maximum time delay bound for stability of linear neutral systems this note is concerned with the stability problem of linear delay differential systems of neutral type a discretized lyapunov functional approach is developed the resulting stability criterion is formulated in the form of a linear matrix inequality for nominal systems the analytical results can be approached with fine discretization numerical examples show significant improvement over approaches in the literature

some simple criteria for stability of neutral delay differential systems this paper deals with the stability of linear neutral delay differential systems some simple criteria for delay independent stability of this class of problems which are described by evaluating the measure and norm of corresponding matrices are presented the main results proposed in this study are sharper than those reported in the literature

a nonmonotone line search technique and its application to unconstrained optimization a new nonmonotone line search algorithm is proposed and analyzed in our scheme we require that an average of the successive function values decreases while the traditional nonmonotone approach of grippo lampariello and lucidi siam j numer anal 23 1986 pp 707 716 requires that a maximum of recent function values decreases we prove global convergence for nonconvex smooth functions and r linear convergence for strongly convex functions for the l bfgs method and the unconstrained optimization problems in the cute library the new nonmonotone line search algorithm used fewer function and gradient evaluations on average than either the monotone or the traditional nonmonotone scheme

technical communique new stability criteria for linear systems with interval time varying delay this paper investigates robust stability of uncertain linear systems with interval time varying delay the time varying delay is assumed to belong to an interval and is a fast time varying function the uncertainty under consideration includes polytopic type uncertainty and linear fractional norm bounded uncertainty a new lyapunov krasovskii functional which makes use of the information of both the lower and upper bounds of the interval time varying delay is proposed to drive some new delay dependent stability criteria in order to obtain much less conservative results a tighter bounding for some term is estimated moreover no redundant matrix variable is introduced finally three numerical examples are given to show the effectiveness of the proposed stability criteria

projected newton methods for optimization problems with simple constraints we consider the problem min f x x geqq 0 and propose algorithms of the form x k 1 x k alpha k d k nabla f x k where cdot denotes projection on the positive orthant alpha k is a stepsize chosen by an armijo like rule and d k is a positive definite symmetric matrix which is partly diagonal we show that d k can be calculated simply on the basis of second derivatives of f so that the resulting newton like algorithm has a typically superlinear rate of convergence with other choices of d k convergence at a typically linear rate is obtained the algorithms are almost as simple as their unconstrained counterparts they are well suited for problems of large dimension such as those arising in optimal control while being competitive with existing methods for low dimensional problems the effectiveness of the newton like algorithm is demonstrated via computational examples involving as many as 10 000 variables extensions to general linearly constrained pr

on the identification of active constraints nondegeneracy conditions that guarantee that the optimal active constraints are identified in a finite number of iterations are studied results of this type have only been established for a few algorithms and then under restrictive hypothesis the main result is a characterization of those algorithms that identify the optimal constraints in a finite number of iterations this result is obtained with a nondegeneracy assumption which is equivalent in the standard nonlinear programming problem to the assumption that there is a set of strictly complementary lagrange multipliers as an important consequence of the authors results the way that this characterization applies to gradient projection and sequential quadratic programming algorithms is shown

stabilizing binomial n estimators abstract the maximum likelihood estimator of the binomial parameter n can be highly unstable resulting in huge fluctuations in the estimate when the data are only slightly perturbed a method of assessing this sensitivity based on perturbing the log likelihood function is proposed and examined

a further refinement of discretized lyapunov functional method for the stability of time delay systems the discretized lyapunov functional method for the stability problem of time delay systems is further refined using a combination of integral inequality and variable elimination technique as a result the computational requirement is further reduced for the same discretization mesh for systems without uncertainty the convergence to the analytical solution is greatly accelerated for uncertain systems the new method is much less conservative numerical examples are presented to illustrate the effectiveness of the method

w methods in optimal control this paper addresses consistency and stability of w methods up to order three for nonlinear ode constrained control problems with possible restrictions on the control the analysis is based on the transformed adjoint system and the control uniqueness property these methods can also be applied to large scale pde constrained optimization since they offer an efficient way to compute gradients of the discrete objective function

an affine scaling interior point method for continuous knapsack constraints with application to support vector machines an affine scaling algorithm asl for optimization problems with a single linear equality constraint and box restrictions is developed the algorithm has the property that each iterate lies in the relative interior of the feasible set the search direction is obtained by approximating the hessian of the objective function in newton s method by a multiple of the identity matrix the algorithm is particularly well suited for optimization problems where the hessian of the objective function is a large dense and possibly ill conditioned matrix global convergence to a stationary point is established for a nonmonotone line search when the objective function is strongly convex asl converges r linearly to the global optimum provided the constraint multiplier is unique and a nondegeneracy condition holds a specific implementation of the algorithm is developed in which the hessian approximation is given by the cyclic barzilai borwein cbb formula the algorithm is evaluated numerically using support vector machine test problems

on the accurate identification of active constraints we consider nonlinear programs with inequality constraints and we focus on the problem of identifying those constraints which will be active at an isolated local solution the correct identification of active constraints is important from both a theoretical and a practical point of view such an identification removes the combinatorial aspect of the problem and locally reduces the inequality constrained minimization problem to an equality constrained problem which can be more easily dealt with we present a new technique which identifies active constraints in a neighborhood of a solution and which requires neither complementary slackness nor uniqueness of the multipliers we also present extensions to variational inequalities and numerical examples illustrating the identification technique

a comparison of n estimators for the binomial distribution abstract estimating the number n of trials given a sequence of independent success counts obtained by replicating the n trial experiment is less studied and a considerably harder problem than estimating p for the binomial distribution both the method of moments and the maximum likelihood estimators of n are considered when p is unknown and small while n is large both estimators become highly unstable stabilized versions of these estimators are proposed and their performance is described in terms of results obtained by simulation

a subspace interior and conjugate gradient method for large scale bound constrained minimization problems a subspace adaptation of the coleman li trust region and interior method is proposed for solving large scale bound constrained minimization problems this method can be implemented with either sparse cholesky factorization or conjugate gradient computation under reasonable conditions the convergence properties of this subspace trust region method are as strong as those of its full space version computational performance on various large test problems is reported advantages of our approach are demonstrated our experience indicates that our proposed method represents an efficient way to solve large bound constrained minimization problems

projected barzilai borwein methods for large scale box constrained quadratic programming this paper studies projected barzilai borwein pbb methods for large scale box constrained quadratic programming recent work on this method has modified the pbb method by incorporating the grippo lampariello lucidi gll nonmonotone line search so as to enable global convergence to be proved we show by many numerical experiments that the performance of the pbb method deteriorates if the gll line search is used we have therefore considered the question of whether the unmodified method is globally convergent which we show not to be the case by exhibiting a counter example in which the method cycles a new projected gradient method pabb is then considered that alternately uses the two barzilai borwein steplengths we also give an example in which this method may cycle although its practical performance is seen to be superior to the pbb method with the aim of both ensuring global convergence and preserving the good numerical performance of the unmodified methods we examine other recent work on nonmonotone line searches and propose a new adaptive variant with some attractive features further numerical experiments show that the pabb method with the adaptive line search is the best bb like method in the positive definite case and it compares reasonably well against the gpcg algorithm of more and toraldo in the indefinite case the pbb method with the adaptive line search is shown on some examples to find local minima with better solution values and hence may be preferred for this reason

superlinear and quadratic convergence of affine scaling interior point newton methods for problems with simple bounds without strict complementarity assumption a class of affine scaling interior point methods for bound constrained optimization problems is introduced which are locally q superlinear or q quadratic convergent it is assumed that the strong second order sufficient optimality conditions at the solution are satisfied but strict complementarity is not required the methods are modifications of the affine scaling interior point newton methods introduced by t f coleman and y li math programming 67 189 224 1994 there are two modifications one is a modification of the scaling matrix the other one is the use of a projection of the step to maintain strict feasibility rather than a simple scaling of the step a comprehensive local convergence analysis is given a simple example is presented to illustrate the pitfalls of the original approach by coleman and li in the degenerate case and to demonstrate the performance of the fast converging modifications developed in this paper

global convergence of trust region interior point algorithms for infinite dimensional nonconvex minimization subject to pointwise bounds a class of interior point trust region algorithms for infinite dimensional nonlinear optimization subject to pointwise bounds in l p banach spaces 2 le p le infty is formulated and analyzed the problem formulation is motivated by optimal control problems with l p controls and pointwise control constraints the interior point trust region algorithms are generalizations of those recently introduced by coleman and li siam j optim 6 1996 pp 418 445 for finite dimensional problems many of the generalizations derived in this paper are also important in the finite dimensional context all first and second order global convergence results known for trust region methods in the finite dimensional setting are extended to the infinite dimensional framework of this paper

a trust region and affine scaling interior point method for nonconvex minimization with linear inequality constraints a trust region and affine scaling interior point method tram is proposed for a general nonlinear minimization with linear inequality constraints in 8 in the proposed approach a newton step is derived from the complementarity conditions based on this newton step a trust region subproblem is formed and the original objective function is monotonically decreased explicit sufficient decrease conditions are proposed for satisfying complementarity dual feasibility and second order optimality the objective of this paper is to establish global and local convergence properties of the proposed trust region and affine scaling interior point method it is shown that the proposed decrease conditions are sufficient for achieving complementarity dual feasibility and second order optimality respectively it is also established that a trust region solution is asymptotically in the interior of the proposed trust region subproblem and a damped trust region step can achieve quadratic convergence

absolute stability criteria for a class of nonlinear singular systems with time delay abstract this paper deals with the problem of absolute stability for a class of time delay singular systems with sector bounded nonlinearity both delay independent and delay dependent criteria are presented and formulated in the form of linear matrix inequalities lmis neither model transformation nor a bounding technique for cross terms nor a slack variable method is involved in obtaining the stability criteria numerical examples are given to show the effectiveness and improvements over some existing results

on bayesian estimation of group size summary in estimating the size of a group of animals it is commonly the case that the sighting probability for individuals is less than one if each individual in the group is independently sighted with equal probability the problem is one of estimating the number of trials from the number of successes in one or more binomial experiments in many cases some individuals in the group have unit sighting probability whereas others do not this note describes the bayesian estimation of group size for this case

stochastic numerical treatment for solving troesch s problem abstract in this article the numerical techniques are presented for the solution of troesch s problem based on neural networks optimized with three different methods including particle swarm optimization pso active set as and pso hybridized with as pso as algorithms the variable transformation is applied in order to convert the original problem to a transformed problem which is relatively less stiff to solve feed forward artificial neural networks are used to model the transformed problem learning of adjustable parameters is made with pso as and pso as algorithms the proposed methodologies are applied to a number of cases for stiff and non stiff boundary value problems the comparative analyses are carried out with other standard numerical solutions as well as approximate analytical solver