generalized substring compression in substring compression one is given a text to preprocess so that upon request a compressed substring is returned generalized substring compression is the same with the following twist the queries contain an additional context substring or a collection of context substrings and the answers are the substring in compressed format where the context substring is used to make the compression more efficient we focus our attention on generalized substring compression and present the first non trivial correct algorithm for this problem inherent to our algorithm is a new method for finding the bounded longest common prefix of substrings which may be of independent interest in addition we propose an efficient algorithm for substring compression which makes use of range successor queries we present several tradeoffs for both problems for compressing the substring s i j possibly with the substring s email protected as a context the best query times we achieve are o c and o clog j ic for substring compression query and generalized substring compression query respectively where c is the number of phrases encoded a preliminary version of this paper has been presented in 21

a conjugate prior for discrete hierarchical log linear models in the bayesian analysis of contingency table data the selection of a prior distribution for either the log linear parameters or the cell probabilities parameter is a major challenge though the conjugate prior on cell probabilities has been defined by dawid and lauritzen 1993 for decomposable graphical models it has not been identified for the larger class of graphical models markov with respect to an arbitrary undirected graph or for the even wider class of hierarchical log linear models in this paper working with the log linear parameters used by glim we first define the conjugate prior for these parameters and then derive the induced prior for the cell probabilities this is done for the general class of hierarchical log linear models we show that the conjugate prior has all the properties that one expects from a prior notational simplicity ability to reflect either no prior knowledge or a priori expert knowledge a moderate number of hyperparameters and mathematical convenience it also has the strong hyper markov property which allows for local updates within prime components for graphical models

projected newton methods and optimization of multicommodity flows a superlinearly convergent newton like method for linearly constrained optimization problems is adapted for solution of multicommodity network flow problems of the type arising in communication and transportation networks we show that the method can be implemented approximately by making use of conjugate gradient iterations without the need to compute explicitly the hessian matrix preliminary computational results suggest that this type of method is capable of yielding highly accurate solutions of nonlinear multicommodity flow problems far more efficiently than any of the methods available at present

the active set method for nonnegative regularization of linear ill posed problems abstract in this work we analyze the behavior of the active set method for the nonnegative regularization of discrete ill posed problems in many applications the solution of a linear ill posed problem is known to be nonnegative standard tikhonov regularization often provides an approximated solution with negative entries we apply the active set method to find a nonnegative approximate solution of the linear system starting from the tikhonov regularized one our numerical experiments show that the active set method is effective in reducing the oscillations in the tikhonov regularized solution and in providing a nonnegative regularized solution of the original linear system

binary models for marginal independence summary log linear models are a classical tool for the analysis of contingency tables in particular the subclass of graphical log linear models provides a general framework for modelling conditional independences however with the exception of special structures marginal independence hypotheses cannot be accommodated by these traditional models focusing on binary variables we present a model class that provides a framework for modelling marginal independences in contingency tables the approach that is taken is graphical and draws on analogies with multivariate gaussian models for marginal independence for the graphical model representation we use bidirected graphs which are in the tradition of path diagrams we show how the models can be parameterized in a simple fashion and how maximum likelihood estimation can be performed by using a version of the iterated conditional fitting algorithm finally we consider combining these models with symmetry restrictions

constrained 3d rotation smoothing via global manifold regression for video stabilization we present a novel motion smoothing algorithm for hand held cameras with application to video stabilization video stabilization seeks to remove unwanted frame to frame jitter due to camera shake for video stabilization we use a pure 3d rotation motion model with known camera projection parameters the 3d camera rotation can be reliably tracked by a gyroscope as commonly found on a smart phone or tablet in this paper we directly smooth the sequence of camera rotation matrices for the video frames by exploiting the riemannian geometry on a manifold our contributions are 1 formulation of motion smoothing as a geodesic convex constrained regression problem on a nonlinear manifold based on geodesic distance 2 computation of gradient and hessian of the objective function using riemannian geometry for gradient related manifold optimization and 3 generalization of the two metric projection algorithm in euclidean space to manifolds to solve the proposed manifold optimization problem efficiently the geodesic distance based smoothness metric better exploits the manifold structure of sequences of rotation matrices the geodesic convex constraints effectively guarantee that no black borders intrude into the stabilized frames the proposed manifold optimization algorithm can find the global optimal solution in only a few iterations experimental results show that video stabilization based on our motion smoothing algorithm outperforms state of the art methods by generating videos with less jitter and without black borders

additivity and interaction in three component experiments with mixtures summary additivity and interaction are defined for a three component mixture experiment and studied both for polynomial models and factorial models the notion of the blandness of a component is introduced linear blending is equivalent to the combination of no interaction and the blandness of all three components

dependency reduction with divisive normalization justification and effectiveness efficient coding transforms that reduce or remove statistical dependencies in natural sensory signals are important for both biology and engineering in recent years divisive normalization dn has been advocated as a simple and effective nonlinear efficient coding transform in this work we first elaborate on the theoretical justification for dn as an efficient coding transform specifically we use the multivariate t model to represent several important statistical properties of natural sensory signals and show that dn approximates the optimal transforms that eliminate statistical dependencies in the multivariate t model second we show that several forms of dn used in the literature are equivalent in their effects as efficient coding transforms third we provide a quantitative evaluation of the overall dependency reduction performance of dn for both the multivariate t models and natural sensory signals finally we find that statistical dependencies in the multivariate t model and natural sensory signals are increased by the dn transform with low input dimensions this implies that for dn to be an effective efficient coding transform it has to pool over a sufficiently large number of inputs

a unified approach for parameter identification of inelastic material models in the frame of the finite element method this work is concerned with identification of parameters for inelastic material models in order to account for possible non uniformness of stress and strain distributions the identification is performed in the frame of the finite element method in particular linearization procedures are described in a systematic manner for the case of complex material models within a geometric linear theory this unified approach allows one to apply the newton method for solving the associated direct problem and to apply gradient based methods for solving the associated inverse problem which is considered as an optimization problem two numerical examples demonstrate the versatility of our approach firstly we consider cooks membrane problem based on simulated data for re identification of material parameters for a viscoplastic power law furthermore material data for j2 flow theory are determined based on experimental data obtained by a grating method for a compact specimen and we will investigate the results by using different starting values and stochastic perturbation of the experimental data

mixture models based on neural network averaging a modified version of the single hidden layer perceptron architecture is proposed for modeling mixtures a particular flexible mixture model is obtained by implementing the box cox transformation as transfer function in this case the network response can be expressed in closed form as a weighted power mean the quadratic scheffe k polynomial and the exponential wilson equation turn out to be special forms of this general mixture model advantages of the proposed network architecture are that binary data sets suffice for training and that it is readily extended to incorporate additional mixture components while retaining all previously determined weights

modified subspace limited memory bfgs algorithm for large scale bound constrained optimization in this paper a subspace limited memory bfgs algorithm for solving large scale bound constrained optimization problems is developed it is modifications of the subspace limited memory quasi newton method proposed by ni and yuan q ni y x yuan a subspace limited memory quasi newton algorithm for large scale nonlinear bound constrained optimization math comput 66 1997 1509 1520 an important property of our proposed method is that more limited memory bfgs update is used under appropriate conditions the global convergence of the method is established the implementations of the method on cute test problems are presented which indicate the modifications are beneficial to the performance of the algorithm

construction of ϕp optimal exact designs with minimum experimental run size for a linear log contrast model in mixture experiments we propose a new method with minimum experimental run size using the properties of hadamard matrices through which some p optimal exact designs including a d and e optimal designs are constructed for a linear log contrast model in mixture experiments copyright 2011 oxford university press

on the identification property of a projected gradient method the authors study the convergence properties of a projected gradient algorithm for the general problem min f x x in omega where f r n to r is a mapping continuously differentiable on a closed convex set omega subseteq r n the algorithm which requires only one projection per iteration is a special version of the method of projection of the gradient by demyanov and rubinov approximate methods in optimization problems elsevier new york 1970 where the step choice is made according to a scheme similar to the one used by calamai and more math programming 39 1987 pp 93 116 the authors are mainly interested in analysing the identification property of the algorithm for the case where the set omega is a polyhedron that is the ability to identify in a finite number of steps the face in which the final solution lies the convergence results that are shown are very similar to those shown in 6 for the standard projected gradient method

an ode traffic network model we are interested in models for vehicular traffic flow based on partial differential equations and their extensions to networks of roads in this paper we simplify a fluidodynamic traffic model and derive a new traffic flow model based on ordinary differential equations odes this is obtained by spatial discretization of an averaged density evolution and a suitable approximation of the coupling conditions at junctions of the network we show that the new model inherits similar features of the full model e g traffic jam propagation we consider optimal control problems controlled by the ode model and derive the optimality system we present numerical results on the simulation and optimization of traffic flow in sample networks

the method of successive affine reduction for nonlinear minimization the traditional development of conjugate gradient cg methods emphasizes notions of conjugacy and the minimization of quadratic functions the associated theory of conjugate direction methods strictly a branch of numerical linear algebra is both elegant and useful for obtaining insight into algorithms for nonlinear minimization nevertheless it is preferable that favorable behavior on a quadratic be a consquence of a more general approach one which fits in more naturally with newton and variable metric methods we give new cg algorithms along these lines and discuss some of their properties along with some numerical supporting evidence

efficient and modular algorithms in modeling finite inelastic deformations objective integration parameter identification and sub stepping techniques this paper presents a general algorithmic formulation for constitutive equations describing inelastic material deformation the algorithmic representation focuses on the generality and modularity which is a key aspect within the development phase of constitutive equations it is shown explicitly how to solve the incremental constitutive equations by use of objective algorithms and an implicit integration scheme applying numerical as well as analytical derivatives although it is shown that using numerical gradients is significantly slower their application is easier especially in the development phase it is also demonstrated that having the constitutive equations modeled in the general context introduced the issue of parameter identification can be tackled by the same implementation using a driver algorithm besides parameter identification the general structure is also applied to two different fe simulations first the objectivity of different integration methods will be presented considering a simple shear loading superposed by a rigid rotation using a strain rate independent material model as example in the second example it is shown for a strain rate dependent material model how the simulation of the wire drawing process can be accelerated crucially by using sub stepping techniques

a general class of distributions on the simplex summary the dirichlet class of distributions on the simplex is overstructured for practical work because of its many strong independence properties a persistent problem of distribution theory has thus been to extend this class to include members free of these independence properties an alternative approach has been the recent introduction of the logistic normal class which contains members with and without the independence properties unfortunately this class is separate from the dirichlet class and so cannot sustain discussion of some strong forms of independence these difficulties are overcome in this paper by the introduction of a more general class which includes as special cases the dirichlet and logistic normal classes the role of this new class in relation to the analysis of compositional data in particular the investigation of independence hypotheses is discussed and illustrated

methods for assessing curvature and interaction in mixture experiments the terms curvature and interaction traditionally are not defined or used in the context of mixture experiments because curvature and interaction effects are partially confounded due to the mixture constraint that the component proportions sum to 1 instead the term nonlinear blending traditionally is defined and used however just as the concept of a component effect is defined specifically for mixture experiments the concepts of curvature and interaction can be also these special definitions lead to special sets of points for assessing the curvature and interaction of mixture components an interaction plot for mixture components is obtained by plotting measured or predicted response values at the special points for assessing interaction together with response trace component effects plots the new mixture component interaction plots provide for graphically assessing the linear curvature and interaction effects of mixture components these methods provide a more complete picture of how mixture

restricted simplicial decomposition for convex constrained problems the strategy of restricted simplicial decomposition is extended to convex programs with convex constraints the resulting algorithm can also be viewed as an extension of the scaled topkis veinott method of feasible directions in which the master problem involves optimization over a simplex rather than the usual line search global convergence of the method is proven and conditions are given under which the master problem will be solved a finite number of times computational testing with dense quadratic problems confirms that the method dramatically improves the topkis veinott algorithm and that it is competitive with the generalized reduced gradient method

efficient nonnegative matrix factorization via projected newton method nonnegative matrix factorization nmf is a popular decomposition technique in pattern analysis document clustering image processing and related fields in this paper we propose a fast nmf algorithm via projected newton method pnm first we propose pnm to efficiently solve a nonnegative least squares problem which achieves a quadratic convergence rate under appropriate assumptions second in the framework of an alternating optimization method we adopt pnm as an essential subroutine to efficiently solve the nmf problem moreover by exploiting the low rank assumption of nmf we make pnm very suitable for solving nmf efficiently empirical studies on both synthetic and real world text and image data demonstrate that pnm is quite efficient to solve nmf compared with several state of the art algorithms

a primal dual active set method for non negativity constrained total variation deblurring problems this paper studies image deblurring problems using a total variation based model with a non negativity constraint the addition of the non negativity constraint improves the quality of the solutions but makes the solution process a difficult one the contribution of our work is a fast and robust numerical algorithm to solve the non negatively constrained problem to overcome the nondifferentiability of the total variation norm we formulate the constrained deblurring problem as a primal dual program which is a variant of the formulation proposed by chan golub and mulet for unconstrained problems here dual refers to a combination of the lagrangian and fenchel duals to solve the constrained primal dual program we use a semi smooth newton s method we exploit the relationship between the semi smooth newton s method and the primal dual active set method to achieve considerable simplification of the computations the main advantages of our proposed scheme are no parameters need significant adjustment a standard inverse preconditioner works very well quadratic rate of local convergence theoretical and numerical numerical evidence of global convergence and high accuracy of solving the optimality system the scheme shows robustness of performance over a wide range of parameters a comprehensive set of numerical comparisons are provided against other methods to solve the same problem which show the speed and accuracy advantages of our scheme

a new active set algorithm for box constrained optimization an active set algorithm asa for box constrained optimization is developed the algorithm consists of a nonmonotone gradient projection step an unconstrained optimization step and a set of rules for branching between the two steps global convergence to a stationary point is established for a nondegenerate stationary point the algorithm eventually reduces to unconstrained optimization without restarts similarly for a degenerate stationary point where the strong second order sufficient optimality condition holds the algorithm eventually reduces to unconstrained optimization without restarts a specific implementation of the asa is given which exploits the recently developed cyclic barzilai borwein cbb algorithm for the gradient projection step and the recently developed conjugate gradient algorithm cg descent for unconstrained optimization numerical experiments are presented using box constrained problems in the cuter and minpack 2 test problem libraries

projected affine scaling interior point newton s method with line search filter for box constrained optimization abstract we present a new projected affine scaling interior point newton s method with a line search filter algorithm for nonlinear optimization problems with box constraints the global convergence of the proposed algorithm is established under some reasonable conditions furthermore the method is shown to be locally quadratically convergent under the strong second order sufficiency condition without assuming strict complementarity of the solution numerical experiments are presented that confirm the robustness and efficiency of the approach

a note on exploiting structure when using slack variables we show how to exploit the structure inherent in the linear algebra for constrained nonlinear optimization problems when inequality constraints have been converted to equations by adding slack variables and the problem is solved using an augmented lagrangian method

comparison of numerical methods for identification of viscoelastic line spectra from static test data viscoelastic line spectra are identified from creep or relaxation data of static experiments with different numerical methods which may or may not depend on additional informations to be provided by the user about the unknown parameters if the least square method is applied a non linear optimization problem with non negative constraints on the parameters has to be solved its solution can be achieved directly by using a gradient based optimization algorithm like the projected newton method of bertsekas however appropriate starting values for the unknown parameters must be chosen the problem can be alleviated by dividing the identification task into three successive steps based on the tschebyscheff approximation and the quadratic optimization method by wolfe alternatively the identification task can be reduced to a quadratic optimization problem if the user provides additional informations about the distribution of the respondance times of the spectra the windowing method of emri and tschoegl is based on this assumption if the line spectrum is assumed to have equally distributed spectrum lines on the logarithmic axis the identification problem can also be solved by standard regularization techniques like the truncated singular value decomposition or the tikhonov regularization the choice of qualified respondance times as additional information requires some experience with the identification task at hand its results may be improved after several reruns of the algorithms various applications of the methods to test and experimental data are given and a comparison of their performance is discussed copyright 2005 john wiley sons ltd

on the toric algebra of graphical models we formulate necessary and sufficient conditions for an arbitrary discrete probability distribution to factor according to an undirected graphical model or a log linear model or other more general exponential models for decomposable graphical models these conditions are equivalent to a set of conditional independence statements similar to the hammersley clifford theorem however we show that for nondecomposable graphical models they are not we also show that nondecomposable models can have nonrational maximum likelihood estimates these results are used to give several novel characterizations of decomposable graphical models

superlinear convergence of affine scaling interior point newton methods for infinite dimensional nonlinear problems with pointwise bounds we develop and analyze a superlinearly convergent affine scaling interior point newton method for infinite dimensional problems with pointwise bounds in lp space the problem formulation is motivated by optimal control problems with lp controls and pointwise control constraints the finite dimensional convergence theory by coleman and li siam j optim 6 1996 pp 418 445 makes essential use of the equivalence of norms and the exact identifiability of the active constraints close to an optimizer with strict complementarity since these features are not available in our infinite dimensional framework algorithmic changes are necessary to ensure fast local convergence the main building block is a newton like iteration for an affine scaling formulation of the kkt condition we demonstrate in an example that a stepsize rule to obtain an interior iterate may require very small stepsizes even arbitrarily close to a nondegenerate solution using a pointwise projection instead we prove superlinear convergence under a weak strict complementarity condition and convergence with mbox q rate 1 under a slightly stronger condition if a smoothing step is available we discuss how the algorithm can be embedded in the class of globally convergent trust region interior point methods recently developed by m heinkenschloss and the authors numerical results for the control of a heating process confirm our theoretical findings

irrigation control in the presence of salinity extended linear quadratic approach an intraseasonal irrigation scheduling problem is dealt with via extended linear quadratic elq control the elq control is well suited for constrained multidimensional problems and provides openloop feedback control rules over the control horizon a conceptual model is developed to describe the dynamics of water allocation and salt movement in the root zone of a crop moisture stress and osmotic stress are combined to obtain the integrated inhibitory effect of salinity on transpiration for the intraseasonal model to be effective against perennial salt accumulation in the root zone it should be able to yield control laws which will lead to favorable root zone conditions at the end of an irrigation season thus avoiding any significant leaching prior to the next growing season this long term aspect of salinity control is handled via probabilistic state constraints which impose desired salinity and moisture levels with desired confidence level the elq control is employed in a case study of expected net benefit maximization over an irrigation season of corn in fort morgan colorado the results in general correspond well with expected irrigation schedules under different conditions and provide valuable information on both short and long term aspects of irrigation control under saline conditions the elq control being an analytic iterative solution scheme with theoretically guaranteed fast convergence has a distinct computational advantage over state of the art procedures

conditional gradient algorithms for norm regularized smooth convex optimization motivated by some applications in signal processing and machine learning we consider two convex optimization problems where given a cone k k a norm vert cdot vert and a smooth convex function f f we want either 1 to minimize the norm over the intersection of the cone and a level set of f f or 2 to minimize over the cone the sum of f f and a multiple of the norm we focus on the case where a the dimension of the problem is too large to allow for interior point algorithms b vert cdot vert is too complicated to allow for computationally cheap bregman projections required in the first order proximal gradient algorithms on the other hand we assume that it is relatively easy to minimize linear forms over the intersection of k k and the unit vert cdot vert ball motivating examples are given by the nuclear norm with k k being the entire space of matrices or the positive semidefinite cone in the space of symmetric matrices and the total variation norm on the space of 2d images we discuss versions of the conditional gradient algorithm capable to handle our problems of interest provide the related theoretical efficiency estimates and outline some applications

a unifying polyhedral approximation framework for convex optimization we propose a unifying framework for polyhedral approximation in convex optimization it subsumes classical methods such as cutting plane and simplicial decomposition but also includes new methods and new versions extensions of old methods such as a simplicial decomposition method for nondifferentiable optimization and a new piecewise linear approximation method for convex single commodity network flow problems our framework is based on an extended form of monotropic programming a broadly applicable model which includes as special cases fenchel duality and rockafellar s monotropic programming and is characterized by an elegant and symmetric duality theory our algorithm combines flexibly outer and inner linearization of the cost function the linearization is progressively refined by using primal and dual differentiation and the roles of outer and inner linearization are reversed in a mathematically equivalent dual algorithm we provide convergence results for the general case where outer and inner linearization are combined in the same algorithm

on projected newton krylov solvers for instationary laminar reacting gas flows numerical aspects of computational modeling of chemical vapor deposition are discussed large sparse strongly nonlinear algebraic systems are to be solved per time step for this inexact newton methods and preconditioned krylov subspace methods are suitable to ensure positivity of concentrations we propose a novel approach namely a projected inexact newton method unlike the commonly used method of clipping this conserves mass efficiency of several preconditionings is compared our numerical tests culminate in an unusually large computation namely a three dimensional case with 17 species and 26 reactions

an active constraints newton algorithm for the spatial price equilibrium problem abstract we consider the general spatial price equilibrium problem in order to solve such problems we develop a newton type algorithm that is combined with an active constraints strategy to handle the nonnegativety constraints or upper bounds on the variables at each iteration of the algorithm the choice of the initial active set is based on the value of the gradient and the dual variables two variants of the algorithm are presented one in which the newton equations are solved by the gauss method and one in which these equations are solved by the conjugate gradient method computational results demonstrate the efficiency of the method

a two stage successive overrelaxation algorithm for solving the symmetric linear complementarity problem we propose a two stage successive overrelaxation method tsor algorithm for solving the symmetric linear complementarity problem after the first sor preprocessing stage this algorithm concentrates on updating a certain prescribed subset of variables which is determined by exploiting the complementarity property we demonstrate that this algorithm successfully solves problems with up to ten thousand variables

algorithms for bound constrained quadratic programming problems we present an algorithm which combines standard active set strategies with the gradient projection method for the solution of quadratic programming problems subject to bounds we show in particular that if the quadratic is bounded below on the feasible set then termination occurs at a stationary point in a finite number of iterations moreover if all stationary points are nondegenerate termination occurs at a local minimizer a numerical comparison of the algorithm based on the gradient projection algorithm with a standard active set strategy shows that on mildly degenerate problems the gradient projection algorithm requires considerable less iterations and time than the active set strategy on nondegenerate problems the number of iterations typically decreases by at least a factor of 10 for strongly degenerate problems the performance of the gradient projection algorithm deteriorates but it still performs better than the active set method

an active set truncated newton method for large scale bound constrained optimization an active set truncated newton method for large scale bound constrained optimization is proposed the active sets are guessed by an identification technique the search direction consists of two parts some of the components are simply defined the other components are determined by the truncated newton method the method based on a nonmonotone line search technique is shown to be globally convergent numerical experiments are presented using bound constrained problems in the cuter test problem library the numerical performance reveals that our method is effective and competitive with the famous algorithm tron

projection free parallel quadratic programming for linear model predictive control a key component in enabling the application of model predictive control mpc in fields such as automotive aerospace and factory automation is the availability of low complexity fast optimisation algorithms to solve the mpc finite horizon optimal control problem in architectures with reduced computational capabilities in this paper we introduce a projection free iterative optimisation algorithm and discuss its application to linear mpc the algorithm originally developed by brand for non negative quadratic programs is based on a multiplicative update rule and it is shown to converge to a fixed point which is the optimum an acceleration technique based on a projection free line search is also introduced to speed up the convergence to the optimum the algorithm is applied to mpc through the dual of the quadratic program qp formulated from the mpc finite time optimal control problem we discuss how termination conditions with guaranteed degree of suboptimality can be enforced and how the algorithm

primal dual active set strategy for a general class of constrained optimal control problems an algorithm for efficient solution of optimal control problems with pointwise constraints is introduced it is based on an active set strategy using primal as well as dual variables under certain assumptions a decrease of an appropriately chosen merit function and convergence of the algorithm are proved numerical examples for a parabolic optimal control problem demonstrate the efficiency of the proposed method for control constrained problems

log mean linear models for binary data this paper is devoted to the theory and application of a novel class of models for binary data which we call log mean linear lml models the characterizing feature of these models is that they are specied by linear constraints on the

newton s method for large bound constrained optimization problems we analyze a trust region version of newton s method for bound constrained problems our approach relies on the geometry of the feasible set not on the particular representation in terms of constraints the convergence theory holds for linearly constrained problems and yields global and superlinear convergence without assuming either strict complementarity or linear independence of the active constraints we also show that the convergence theory leads to an efficient implementation for large bound constrained problems

convergence properties of trust region methods for linear and convex constraints we develop a convergence theory for convex and linearly constrained trust region methods which only requires that the step between iterates produce a sufficient reduction in the trust region subproblem global convergence is established for general convex constraints while the local analysis is for linearly constrained problems the main local result establishes that if the sequence converges to a nondegenerate stationary point then the active constraints at the solution are identified in a finite number of iterations as a consequence of the identification properties we develop rate of convergence results by assuming that the step is a truncated newton method our development is mainly geometrical this approach allows the development of a convergence theory without any linear independence assumptions

mixture experiments ill conditioning and quadratic model specification well conditioned models are important particularly for practitioners who work with regression models for mixture experiments where parameter estimates are individually meaningful in this article we investigate conditioning in second order mixture models using variance inflation factors maximum and minimum eigenvalues of the information matrix and condition numbers to assess conditioning a range of equivalent mixture models that lie between the scheffe model s model and the kronecker model k model is examined and pseudocomponent transformations for lower bounds l pseudocomponents and upper bounds u pseudocomponents are also discussed we prove that the maximum eigenvalue for the information matrix for the k model is always smaller than that for any other model in the above range we recommend in practice the use of the k model to reduce ill conditioning and the appropriate use of pseudocomponents

a solver for nonconvex bound constrained quadratic optimization we present a new algorithm for nonconvex bound constrained quadratic optimization in the strictly convex case our method is equivalent to the state of the art algorithm by dostal and schoberl comput optim appl 30 2005 pp 23 43 unlike their method however we establish a convergence theory for our algorithm that holds even when the problems are nonconvex this is achieved by carefully addressing the challenges associated with directions of negative curvature in particular those that may naturally arise when applying the conjugate gradient algorithm to an indefinite system of equations our presentation and analysis deal explicitly with both lower and upper bounds on the optimization variables whereas the work by dostal and schoberl considers only strictly convex problems with lower bounds to handle this generality we introduce the reduced chopped gradient that is analogous to the reduced free gradient previously used the reduced chopped gradient leads to a new condition that is used to

log contrast models for experiments with mixtures summary a new form of expected response function involving log contrasts of the proportions is introduced for experiments with mixtures the advantages and disadvantages of log contrast models are discussed and illustrated in applications in particular the parameters of the model are related to certain useful hypotheses about mixtures and some relevant contrasts are identified

generalized linear models with linear constraints for microbiome compositional data motivated by regression analysis for microbiome compositional data this article considers generalized linear regression analysis with compositional covariates where a group of linear constraints on regression coefficients are imposed to account for the compositional nature of the data and to achieve subcompositional coherence a penalized likelihood estimation procedure using a generalized accelerated proximal gradient method is developed to efficiently estimate the regression coefficients a de biased procedure is developed to obtain asymptotically unbiased and normally distributed estimates which leads to valid confidence intervals of the regression coefficients simulations results show the correctness of the coverage probability of the confidence intervals and smaller variances of the estimates when the appropriate linear constraints are imposed the methods are illustrated by a microbiome study in order to identify bacterial species that are associated with inflammatory bowel disease ibd and to predict ibd using fecal microbiome

structure estimation for discrete graphical models generalized covariance matrices and their inverses we investigate the relationship between the structure of a discrete graphical model and the support of the inverse of a generalized covariance matrix we show that for certain graph structures the support of the inverse covariance matrix of indicator variables on the vertices of a graph reflects the conditional independence structure of the graph our work extends results that have previously been established only in the context of multivariate gaussian graphical models thereby addressing an open question about the significance of the inverse covariance matrix of a non gaussian distribution the proof exploits a combination of ideas from the geometry of exponential families junction tree theory and convex analysis these population level results have various consequences for graph selection methods both known and novel including a novel method for structure estimation for missing or corrupted observations we provide nonasymptotic guarantees for such methods and illustrate the sharpness of these predictions via simulations

measuring component effects in constrained mixture experiments in a mixture experiment the response to a mixture of q components is a function of the proportions x 1 x 2 x q of components in the mixture the proporitons satisfy the constraint x i 1 and the nature of a particular situation may impose other restrictions the problem considered is the measurement of the effect each component has on the response a new mixture component effect measure is presented and compared to previously suggested measures this new measure incorporates more information concerning the size shape and location of the constraint region than do the previous suggestions a distinction between partial and total effects is made and the uses of these effects in modifying and interpreting mixture response prediction equations are considered the methods of the article are illustrated in an example from a glass development study in a waste vitrification program

additive heredity model for the analysis of mixture of mixtures experiments the mixture of mixtures mom experiment is different from the classical mixture experiment in that the mixture component in mom experiments known as the major component is made up of subcomponen

a feasible active set method with reoptimization for convex quadratic mixed integer programming we propose a feasible active set method for convex quadratic programming problems with nonnegativity constraints this method is specifically designed to be embedded into a branch and bound algorithm for convex quadratic mixed integer programming problems the branch and bound algorithm generalizes the approach for unconstrained convex quadratic integer programming proposed by buchheim caprara and lodi math program 135 2012 pp 369 395 to the presence of linear constraints the main feature of the latter approach consists of a sophisticated preprocessing phase leading to a fast enumeration of the branch and bound nodes moreover the feasible active set method takes advantage of this preprocessing phase and is well suited for reoptimization experimental results for randomly generated instances show that the new approach significantly outperforms the miqp solver of tt cplex 12 6 for instances with a small number of constraints

maximum likelihood estimation in log linear models we study maximum likelihood estimation in log linear models under conditional poisson sampling schemes we derive necessary and sufficient conditions for existence of the maximum likelihood estimator mle of the model parameters and investigate estimability of the natural and mean value parameters under a nonexistent mle our conditions focus on the role of sampling zeros in the observed table we situate our results within the framework of extended exponential families and we exploit the geometric properties of log linear models we propose algorithms for extended maximum likelihood estimation that improve and correct the existing algorithms for log linear model analysis

tackling box constrained optimization via a new projected quasi newton approach numerous scientific applications across a variety of fields depend on box constrained convex optimization box constrained problems therefore continue to attract research interest we address box constrained strictly convex problems by deriving two new quasi newton algorithms our algorithms are positioned between the projected gradient j b rosen j siam 8 1960 pp 181 217 and projected newton d p bertsekas siam j control optim 20 1982 pp 221 246 methods we also prove their convergence under a simple armijo step size rule we provide experimental results for two particular box constrained problems nonnegative least squares nnls and nonnegative kullback leibler nnkl minimization for both nnls and nnkl our algorithms perform competitively as compared to well established methods on medium sized problems for larger problems our approach frequently outperforms the competition

a feasible trust region sequential quadratic programming algorithm an algorithm for smooth nonlinear constrained optimization problems is described in which a sequence of feasible iterates is generated by solving a trust region sequential quadratic programming sqp subproblem at each iteration and by perturbing the resulting step to retain feasibility of each iterate by retaining feasibility the algorithm avoids several complications of other trust region sqp approaches the objective function can be used as a merit function and the sqp subproblems are feasible for all choices of the trust region radius global convergence properties are analyzed under various assumptions on the approximate hessian under additional assumptions superlinear convergence to points satisfying second order sufficient conditions is proved

quasi newton projection methods and the discrepancy principle in image restoration abstract in this work the problem of the restoration of images corrupted by space invariant blur and noise is considered this problem is ill posed and regularization is required the image restoration problem is formulated as a nonnegatively constrained minimization problem whose objective function depends on the statistical properties of the noise corrupting the observed image the cases of gaussian and poisson noise are both considered a newton like projection method with early stopping of the iterates is proposed as an iterative regularization method in order to determine a nonnegative approximation to the original image a suitable approximation of the hessian of the objective function is proposed for a fast solution of the newton system the results of the numerical experiments show the effectiveness of the method in computing a good solution in few iterations when compared with some methods recently proposed as best performing

models for mixture experiments when the response depends on the total amount the usual definition of a mixture experiment requires that the response depend only on the proportions of the mixture components and not on the total amount of the mixture we consider mixture experiments in which the response also depends on the total amount and we develop mixture amount models appropriate for such situations models in the component amounts are also considered and are shown to be reduced forms of the mixture amount models examples are used to illustrate the development interpretation and comparison of the models

variable selection in regression with compositional covariates motivated by research problems arising in the analysis of gut microbiome and metagenomic data we consider variable selection and estimation in high dimensional regression with compositional covariates we propose an l1 regularization method for the linear log contrast model that respects the unique features of compositional data we formulate the proposed procedure as a constrained convex optimization problem and introduce a coordinate descent method of multipliers for efficient computation in the high dimensional setting where the dimensionality grows at most exponentially with the sample size model selection consistency and ell infty bounds for the resulting estimator are established under conditions that are mild and interpretable for compositional data the numerical performance of our method is evaluated via simulation studies and its usefulness is illustrated by an application to a microbiome study relating human body mass index to gut microbiome composition

descent algorithms on oblique manifold for source adaptive ica contrast a riemannian manifold optimization strategy is proposed to facilitate the relaxation of the orthonormality constraint in a more natural way in the course of performing independent component analysis ica that employs a mutual information based source adaptive contrast function despite the extensive development of manifold techniques catering to the orthonormality constraint only a limited number of works have been dedicated to oblique manifold ob algorithms to intrinsically handle the normality constraint which has been empirically shown to be superior to other riemannian and euclidean approaches imposing the normality constraint implicitly in line with the ica definition essentially guarantees a substantial improvement in the solution accuracy by way of increased degrees of freedom while searching for an optimal unmixing ica matrix in contrast with the orthonormality constraint designs of the steepest descent conjugate gradient with hager zhang or a hybrid update parameter quasi newton and cost effective quasi newton methods intended for ob are presented in this paper their performance is validated using natural images and systematically compared with the popular state of the art approaches in order to assess the performance effects of the choice of algorithm and the use of a riemannian rather than euclidean framework we surmount the computational challenge associated with the direct estimation of the source densities using the improved fast gauss transform in the evaluation of the contrast function and its gradient the proposed ob schemes may find applications in the offline image signal analysis wherein on one hand the computational overhead can be tolerated and on the other the solution quality holds paramount interest

on the convergence of nonconvex minimization methods for image recovery nonconvex nonsmooth regularization method has been shown to be effective for restoring images with neat edges fast alternating minimization schemes have also been proposed and developed to solve the nonconvex nonsmooth minimization problem the main contribution of this paper is to show the convergence of these alternating minimization schemes based on the kurdyka ojasiewicz property in particular we show that the iterates generated by the alternating minimization scheme converges to a critical point of this nonconvex nonsmooth objective function we also extend the analysis to nonconvex nonsmooth regularization model with box constraints and obtain similar convergence results of the related minimization algorithm numerical examples are given to illustrate our convergence analysis

fast algorithms for finding nearest common ancestors we consider the following problem given a collection of rooted trees answer on line queries of the form what is the nearest common ancester of vertices x and y we show that any pointer machine that solves this problem requires omega log log n time per query in the worst case where n is the total number of vertices in the trees on the other hand we present an algorithm for a random access machine with uniform cost measure and a bound of omega log n on the number of bits per word that requires o 1 time per query and o n preprocessing time assuming that the collection of trees is static for a version of the problem in which the trees can change between queries we obtain an almost linear time and linear space algorithm

parameter identification for finite deformation elasto plasticity in principal directions this work is concerned with identification of material parameters based on experimental data which represent nonuniform distributions of stresses and deformations within the volume of the specimen both elastic and inelastic material non linearities in the frame of a finite deformation theory are taken into account gradient based descent methods e g gauss newton method quasi newton method are used for minimization of a least squares function to this end a sensitivity analysis is performed and the resulting expressions are presented in a spatial and a material setting in particular the cases of an isotropic hyperelastic model and a multiplicative plasticity model with an exponential type integration scheme both formulated in principal directions are considered two numerical examples based on simulated data and experimental data obtained by a grating method demonstrate the versatility of our approach

a sequential homotopy method for mathematical programming problems we propose a sequential homotopy method for the solution of mathematical programming problems formulated in abstract hilbert spaces under the guignard constraint qualification the method is equivalent to performing projected backward euler timestepping on a projected gradient antigradient flow of the augmented lagrangian the projected backward euler equations can be interpreted as the necessary optimality conditions of a primal dual proximal regularization of the original problem the regularized problems are always feasible satisfy a strong constraint qualification guaranteeing uniqueness of lagrange multipliers yield unique primal solutions provided that the stepsize is sufficiently small and can be solved by a continuation in the stepsize we show that equilibria of the projected gradient antigradient flow and critical points of the optimization problem are identical provide sufficient conditions for the existence of global flow solutions and show that critical points with emanating descent curves cannot be asymptotically stable equilibria of the projected gradient antigradient flow practically eradicating convergence to saddle points and maxima the sequential homotopy method can be used to globalize any locally convergent optimization method that can be used in a homotopy framework we demonstrate its efficiency for a class of highly nonlinear and badly conditioned control constrained elliptic optimal control problems with a semismooth newton approach for the regularized subproblems

general blending models for data from mixture experiments we propose a new class of models providing a powerful unification and extension of existing statistical methodology for analysis of data obtained in mixture experiments these models which integrate models proposed by scheffe and becker extend considerably the range of mixture component effects that may be described they become complex when the studied phenomenon requires it but remain simple whenever possible this article has supplementary material online

projected newton methods for optimization problems with simple constraints we consider the problem min f x x geqq 0 and propose algorithms of the form x k 1 x k alpha k d k nabla f x k where cdot denotes projection on the positive orthant alpha k is a stepsize chosen by an armijo like rule and d k is a positive definite symmetric matrix which is partly diagonal we show that d k can be calculated simply on the basis of second derivatives of f so that the resulting newton like algorithm has a typically superlinear rate of convergence with other choices of d k convergence at a typically linear rate is obtained the algorithms are almost as simple as their unconstrained counterparts they are well suited for problems of large dimension such as those arising in optimal control while being competitive with existing methods for low dimensional problems the effectiveness of the newton like algorithm is demonstrated via computational examples involving as many as 10 000 variables extensions to general linearly constrained pr

a family of second order methods for convex ell _1 l1 regularized optimization this paper is concerned with the minimization of an objective that is the sum of a convex function f and an ell 1 l1 regularization term our interest is in active set methods that incorporate second order information about the function f to accelerate convergence we describe a semismooth newton framework that can be used to generate a variety of second order methods including block active set methods orthant based methods and a second order iterative soft thresholding method the paper proposes a new active set method that performs multiple changes in the active manifold estimate at every iteration and employs a mechanism for correcting these estimates when needed this corrective mechanism is also evaluated in an orthant based method numerical tests comparing the performance of three active set methods are presented

two metric projection methods for constrained optimization this paper is concerned with the problem min f x mid x in x where x is a convex subset of a linear space h and f is a smooth real valued function on h we propose the class of methods x k 1 p x k alpha k g k where p denotes projection on x with respect to a hilbert space norm cdot g k denotes the frechet derivative of f at x k with respect to another hilbert space norm cdot k on h and alpha k is a positive scalar stepsize we thus remove an important restriction in the original proposal of goldstein 1 and levitin and poljak 2 where the norms cdot and cdot k must be the same it is therefore possible to match the norm cdot with the structure of x so that the projection operation is simplified while at the same time reserving the option to choose cdot k on the basis of approximations to the hessian of f so as to attain a typically superlinear rate of convergence the resulting methods are particularly attrac

partitioned dynamic programming for optimal control parallel algorithms for the solution of linear quadratic optimal control problems are described the algorithms are based on a straightforward decomposition of the domain of the problem and are related to multiple shooting methods for two point boundary value problems their arithmetic cost is approximately twice that of the serial dynamic programming approach however they have the advantage that they can be efficiently implemented on a wide variety of parallel architectures extension to the case in which there are box constraints on the controls is simple the algorithms can be used to solve linear quadratic subproblems arising from the application of newton s method or two metric gradient projection methods to nonlinear problems

a direct active set algorithm for large sparse quadratic programs with simple bounds we show how a direct active set method for solving definite and indefinite quadratic programs with simple bounds can be efficiently implemented for large sparse problems all of the necessary factorizations can be carried out in a static data structure that is set up before the numeric computation begins the space required for these factorizations is no larger than that required for a single sparse cholesky factorization of a matrix with the same sparsity structure as the hessian of the quadratic we propose several improvements to this basic algorithm a new way to find a search direction in the indefinite case that allows us to free more than one variable at a time and a new heuristic method for finding a starting point these ideas are motivated by the two norm trust region problem additionally we also show how projection techniques can be used to add several constraints to the active set at each iteration our experimental results show that an algorithm with these improvements runs much faster than the basic algorithm for positive definite problems and finds local minima with lower function values for indefinite problems

resolution and uncertainty in 1d csem inversion a bayesian approach and open source implementation we show that resolution and uncertainty in csem inversion is most naturally approached using a bayesian framework resolution can be inferred by either hierarchical models with free parameters for effective correlation lengths bayesian smoothing or model choice frameworks applied to variable resolution spatial models bayesian splitting merging we find that typical 1d csem data can be modelled with quite parsimonious models typically o 10 parameters per common midpoint efficient opt imizations for the csem problem must address the challenges of poor scaling strong nonlinearity multimodality and the necessity of bound constraints the posterior parameter uncertainties are frequently controlled by the nonlinearity and linearised approaches to uncertainty are usually very poor in markov chain monte carlo mcmc approaches the nonlinearity and poor scaling make good mixing hard to achieve a novel approximate frequentist method we call the bayesianized parametric bootstrap sometimes called randomized maximum likelihood is much more efficient than mcmc in this problem is considerably better than linearized analysis but tends to modestly overstate uncertainties the software that implements these ideas for the 1d csem problem is made available under an open source license agreement

superlinear and quadratic convergence of affine scaling interior point newton methods for problems with simple bounds without strict complementarity assumption a class of affine scaling interior point methods for bound constrained optimization problems is introduced which are locally q superlinear or q quadratic convergent it is assumed that the strong second order sufficient optimality conditions at the solution are satisfied but strict complementarity is not required the methods are modifications of the affine scaling interior point newton methods introduced by t f coleman and y li math programming 67 189 224 1994 there are two modifications one is a modification of the scaling matrix the other one is the use of a projection of the step to maintain strict feasibility rather than a simple scaling of the step a comprehensive local convergence analysis is given a simple example is presented to illustrate the pitfalls of the original approach by coleman and li in the degenerate case and to demonstrate the performance of the fast converging modifications developed in this paper

on the rate of convergence of a partially asynchronous gradient projection algorithm recently bertsekas and tsitsiklis proposed a partially asynchronous implementation of the gradient projection algorithm of goldstein and levitin and polyak for the problem of minimizing a differentiable function over a closed convex set in this paper the rate of convergence of this algorithm is analyzed it is shown that if the standard assumptions hold that is the solution set is nonempty and the gradient of the function is lipschitz continuous and i the isocost surfaces of the objective function restricted to the solution set are properly separated and ii a certain multifunction associated with the problem is locally upper lipschitzian then this algorithm attains a linear rate of convergence

computing minimal and maximal suffixes of a substring we consider the problems of computing the maximal and the minimal non empty suffixes of substrings of a longer text of length n for the minimal suffix problem we show that for every 1 log n there exists a linear space data structure with o query time and o n log n preprocessing time as a sample application we show that this data structure can be used to compute the lyndon decomposition of any substring of the text in o k time where k is the number of distinct factors in the decomposition for the maximal suffix problem we give a linear space structure with o 1 query time and o n preprocessing time in other words we simultaneously achieve both the optimal query time and the optimal construction time

global sensitivity analysis for mixture experiments a mixture experiment is characterized by having two or more inputs that are specified as a percentage contribution to a total amount of material in such situations the input variables are correlated because they must sum to one consequently additional care must be taken when fitting statistical models or visualizing the effect of one or more inputs on the response in this article we consider the use of a gaussian process to model the output from a computer simulator taking a mixture input we introduce a procedure to perform global sensitivity analysis of the code output providing main effects and revealing interactions the resulting methodology is illustrated using a function with analytically tractable results for comparison a chemical compositional simulator and a physical experiment supplementary materials providing assistance with implementing this methodology are available online

a new technique for the design of two dimensional fir and iir filters the design problem of two dimensional digital filters is the subject of this paper the problem is formulated for both fir and iir filters in such a way that the performance index is a convex function of the optimization vector the design technique is based on newton s method each iteration consists of n easily computed steps where n is the number of parameters to be optimized the coefficients can be obtained in the quantization levels according to the desired number representation illustrative examples and comparisons with existing techniques are also included