benchmarking optimization software with performance profiles we propose performance profiles distribution functions for a performance metric as a tool for benchmarking and comparing optimization software we show that performance profiles combine the best features of other tools for performance evaluation

optimal approximation of sparse hessians and its equivalence to a graph coloring problem we consider the problem of approximating the hessian matrix of a smooth non linear function using a minimum number of gradient evaluations particularly in the case that the hessian has a known fixed sparsity pattern we study the class of direct methods for this problem and propose two new ways of classifying direct methods examples are given that show the relationships among optimal methods from each class the problem of finding a non overlapping direct cover is shown to be equivalent to a generalized graph coloring problem the distance 2 graph coloring problem a theorem is proved showing that the general distance k graph coloring problem is np complete for all fixedk 2 and hence that the optimal non overlapping direct cover problem is also np complete some worst case bounds on the performance of a simple coloring heuristic are given an appendix proves a well known folklore result which gives lower bounds on the number of gradient evaluations needed in any possible approximation method

krein type theorems and ordered structure for cauchy de branges spaces abstract we extend some results of m g krein to the class of entire functions which can be represented as ratios of discrete cauchy transforms in the plane as an application we obtain new versions of de branges ordering theorem for nearly invariant subspaces in a class of hilbert spaces of entire functions examples illustrating sharpness of the obtained results are given we also discuss applications to spectral theory of rank one perturbations of normal operators

optimal estimation of jacobian and hessian matrices that arise in finite difference calculations in this paper the problem of estimating jacobian and hessian matrices arising in the finite difference approximation of partial differential equations is considered using the notion of computational molecule or stencil schemes are developed that require the minimum number of differences to estimate these matrices a procedure applicable to more complicated structures is also given 1 introduction in the past few years there has been a growing interest in studying efficient ways to obtain good estimates of sparse jacobian or hessian matrices two basic approaches have been followed in the first approach it is assumed that one has an estimate of the jacobian hessian which one wishes to revise given arbi trary changes in the variables and the corresponding changes in the functions gradients this has led to the development of several sparse quasi newton methods see 11 14 12 for example in the second approach it is assumed that one can specify the changes in the variables one then forms an estimate of the jacobian hessian matrix from the observed changes in the functions gradients when the dimension of the matrix is large and the evaluation of each function gradient vector is expensive it is necessary to take the sparsity structure of the matrix into account in order to make this approach efficient several methods for doing this for arbitrary sparsity structures have already appeared in the literature 2 5 7 10 13 in this paper we consider the second approach for cases where the sparsity structure arises from finite difference approximations to partial differential equa tions both symmetric and unsymmetric matrices are considered although the algorithms presented do not generalize to every pattern generated by finite dif ferences they cover the most useful structures and in some cases extend existing methods to more complex situations in the next section we describe the method of curtis powell and reid 5 henceforth referred to as the cpr method for estimating sparse jacobian matrices we then show how the so called computational molecule or stencil 1 of the finite difference operator associated with the jacobian matrix under study can be used to determine the groupings i e function differencing directions used by the

incomplete cholesky factorizations with limited memory we propose an incomplete cholesky factorization for the solution of large scale trust region subproblems and positive definite systems of linear equations this factorization depends on a parameter p that specifies the amount of additional memory in multiples of n the dimension of the problem that is available there is no need to specify a drop tolerance our numerical results show that the number of conjugate gradient iterations and the computing time are reduced dramatically for small values of p we also show that in contrast with drop tolerance strategies the new approach is more stable in terms of number of iterations and memory requirements

efficient preconditioner updates for shifted linear systems we present a technique for building effective and low cost preconditioners for sequences of shifted linear systems a alpha i x alpha b where a is symmetric positive definite and alpha 0 this technique updates a preconditioner for a available in the form of an ldl t factorization by modifying only the nonzero entries of the l factor in such a way that the resulting preconditioner mimics the diagonal of the shifted matrix and reproduces its overall behavior this approach is supported by a theoretical analysis as well as by numerical experiments showing that it works efficiently for a broad range of values of alpha

a scalable parallel algorithm for incomplete factor preconditioning we describe a parallel algorithm for computing incomplete factor ilu preconditioners the algorithm attains a high degree of parallelism through graph partitioning and a two level ordering strategy both the subdomains and the nodes within each subdomain are ordered to preserve concurrency we show through an algorithmic analysis and through computational results that this algorithm is scalable experimental results include timings on three parallel platforms for problems with up to 20 million unknowns running on up to 216 processors the resulting preconditioned krylov solvers have the desirable property that the number of iterations required for convergence is insensitive to the number of processors

newton type minimization via the lanczos method this paper discusses the use of the linear conjugate gradient method developed via the lanczos method in the solution of large scale unconstrained minimization problems at each iteration of a newton type method the direction of search is defined as the solution of a quadratic subproblem when the number of variables is very large this subproblem may be solved using the linear conjugate gradient method of hestenes and stiefel we show how the equivalent lanczos characterization of the linear conjugate gradient method may be exploited to define a modified newton method which can be applied to problems that do not necessarily have positive definite hessian matrices at all points of the region of interest this derivation also makes it possible to compute a negative curvature direction at a stationary point the idea of a truncated newton method is to perform only a limited number of iterations of the quadratic subproblem this effectively gives a search direction that interpolates between the steepest de

schrodinger operators and de branges spaces abstract we present an approach to de branges s theory of hilbert spaces of entire functions that emphasizes the connections to the spectral theory of differential operators the theory is used to discuss the spectral representation of one dimensional schrodinger operators and to solve the inverse spectral problem

completeness and spectral synthesis of nonselfadjoint one dimensional perturbations of selfadjoint operators abstract we study spectral properties of nonselfadjoint rank one perturbations of compact selfadjoint operators the problems under consideration include completeness of eigenvectors relations between completeness of the perturbed operator and its adjoint and the spectral synthesis problem we obtain new criteria for completeness and spectral synthesis in this class as well as a series of counterexamples which show that the spectral structure of rank one perturbations is in general unexpectedly rich and complicated a parallel spectral theory is developed for one dimensional singular perturbations of unbounded selfadjoint operators our approach is based on a functional model for this class which translates the properties of operators to completeness problems for systems of reproducing kernels and their biorthogonals in some spaces of analytic entire functions

a matrix free approach to build band preconditioners for large scale bound constrained optimization abstract we propose a procedure for building symmetric positive definite band preconditioners for large scale symmetric possibly indefinite linear systems when the coefficient matrix is not explicitly available but matrix vector products involving it can be computed we focus on linear systems arising in newton type iterations within matrix free versions of projected methods for bound constrained nonlinear optimization in this case the structure and the size of the matrix may significantly change in subsequent iterations and preconditioner updating algorithms that exploit information from previous steps cannot be easily applied our procedure is based on a recursive approach that incrementally improves the quality of the preconditioner while requiring a modest number of matrix vector products a strategy for dynamically choosing the bandwidth of the preconditioners is also presented numerical results are provided showing the performance of our preconditioning technique within a trust region newton method for bound constrained optimization

global convergence of a class of trust region algorithms for optimization with simple bounds this paper extends the known excellent global convergence properties of trust region algorithms for unconstrained optimization to the case where bounds on the variables are present weak conditions on the accuracy of the hessian approximations are considered it is also shown that when the strict complementarily condition holds the proposed algorithms reduce to an unconstrained calculation after finitely many iterations allowing a fast asymptotic rate of convergence

on the order of indeterminate moment problems abstract for an indeterminate moment problem we denote the orthonormal polynomials by p n we study the relation between the growth of the function p z n 0 p n z 2 1 2 and summability properties of the sequence p n z under certain assumptions on the recurrence coefficients from the three term recurrence relation z p n z b n p n 1 z a n p n z b n 1 p n 1 z we show that the function p is of order with 0 1 if and only if the sequence p n z is absolutely summable to any power greater than 2 furthermore the order is equal to the exponent of convergence of the sequence b n similar results are obtained for logarithmic order and for more general types of slow growth to prove these results we introduce a concept of an order function and its dual we also relate the order of p with the order of certain entire functions defined in terms of the moments or the leading coefficient of p n

ilum a multi elimination ilu preconditioner for general sparse matrices standard preconditioning techniques based on incomplete lu ilu factorizations offer a limited degree of parallelism in general a few of the alternatives advocated so far consist of either using some form of polynomial preconditioning or applying the usual ilu factorization to a matrix obtained from a multicolor ordering in this paper we present an incomplete factorization technique based on independent set orderings and multicoloring we note that in order to improve robustness it is necessary to allow the preconditioner to have an arbitrarily high accuracy as is done with ilus based on threshold techniques the ilum factorization described in this paper is in this category it can be viewed as a multifrontal version of a gaussian elimination procedure with threshold dropping which has a high degree of potential parallelism the emphasis is on methods that deal specifically with general unstructured sparse matrices such as those arising from finite element methods on unstructured meshes

recursive blocked algorithms and hybrid data structures for dense matrix library software matrix computations are both fundamental and ubiquitous in computational science and its vast application areas along with the development of more advanced computer systems with complex memory hierarchies there is a continuing demand for new algorithms and library software that efficiently utilize and adapt to new architecture features this article reviews and details some of the recent advances made by applying the paradigm of recursion to dense matrix computations on today s memory tiered computer systems recursion allows for efficient utilization of a memory hierarchy and generalizes existing fixed blocking by introducing automatic variable blocking that has the potential of matching every level of a deep memory hierarchy novel recursive blocked algorithms offer new ways to compute factorizations such as cholesky and qr and to solve matrix equations in fact the whole gamut of existing dense linear algebra factorization is beginning to be reexamined in view of the recursive paradigm use of recursion has led to using new hybrid data structures and optimized superscalar kernels the results we survey include new algorithms and library software implementations for level 3 kernels matrix factorizations and the solution of general systems of linear equations and several common matrix equations the software implementations we survey are robust and show impressive performance on today s high performance computing systems

estimation of sparse jacobian matrices and graph coloring blems given a mapping with a sparse jacobian matrix the problem of minimizing the number of function evaluations needed to estimate the jacobian matrix by differences is investigated this problem can be attacked as a graph coloring problem and this approach leads to very efficient algorithms the behavior of these algorithms is studied and in particular it is proved that two of the algorithms are optimal for band graphs numerical evidence is presented which indicates that these two algorithms are nearly optimal on practical problems

cheaper jacobians by simulated annealing jacobian matrices can be accumulated by applying the chain rule to vector functions given as computer programs in different orders resulting in varying operations counts while yielding identical results up to round off the minimization of the number of operations performed leads to a computationally hard combinatorial optimization problem based on vertex elimination in computational graphs this paper discusses simulated annealing as a method for generating nearly optimal jacobian code

on the limited memory bfgs method for large scale optimization we study the numerical performance of a limited memory quasi newton method for large scale optimization which we call the l bfgs method we compare its performance with that of the method developed by buckley and lenir 1985 which combines cycles of bfgs steps and conjugate direction steps our numerical tests indicate that the l bfgs method is faster than the method of buckley and lenir and is better able to use additional storage to accelerate convergence we show that the l bfgs method can be greatly accelerated by means of a simple scaling we then compare the l bfgs method with the partitioned quasi newton method of griewank and toint 1982a the results show that for some problems the partitioned quasi newton method is clearly superior to the l bfgs method however we find that for other problems the l bfgs method is very competitive due to its low iteration cost we also study the convergence properties of the l bfgs method and prove global convergence on uniformly convex problems

higher order reverse automatic differentiation with emphasis on the third order it is commonly assumed that calculating third order information is too expensive for most applications but we show that the directional derivative of the hessian d 3 f x cdot d d3f x d can be calculated at a cost proportional to that of a state of the art method for calculating the hessian matrix we do this by first presenting a simple procedure for designing high order reverse methods and applying it to deduce several methods including a reverse method that calculates d 3f x cdot d d3f x d we have implemented this method taking into account symmetry and sparsity and successfully calculated this derivative for functions with a million variables these results indicate that the use of third order information in a general nonlinear solver such as halley chebyshev methods could be a practical alternative to newton s method furthermore high order sensitivity information is used in methods for robust aerodynamic design an efficient high order differentiation tool could facilitate the use of similar methods in the design of other mechanical structures

a matrix free preconditioner for sparse symmetric positive definite systems and least squares problems we analyze and discuss matrix free and limited memory preconditioners for sparse symmetric positive definite systems and normal equations of large and sparse least squares problems the preconditioners are based on a partial cholesky factorization and can be coupled with a deflation strategy the construction of the preconditioners requires only matrix vector products is breakdown free and does not need to form the matrix the memory requirements of the preconditioners are defined in advance and they do not depend on the number of nonzero entries in the matrix when eigenvalue deflation is used the preconditioners turn out to be suitable for solving sequences of slowly changing systems or linear systems with different right hand sides numerical results are provided including the case of sequences arising in nonnegative linear least squares problems solved by interior point methods

summability properties of gabor expansions abstract we show that there exist complete and minimal systems of time frequency shifts of gaussians in l 2 r which are not strong markushevich basis do not admit the spectral synthesis in particular it implies that there is no linear summation method for general gaussian gabor expansions on the other hand we prove that the spectral synthesis for such gabor systems holds up to one dimensional defect

preconditioners for generalized saddle point problems noindent we propose and examine block diagonal preconditioners and variants of indefinite preconditioners for block two by two generalized saddle point problems that is we consider the nonsymmetric nonsingular case where the 2 2 block is small in norm and we are particularly concerned with the case where the 1 2 block is different from the transposed 2 1 block we provide theoretical and experimental analyses of the convergence and eigenvalue distributions of the preconditioned matrices we also extend the results of de sturler and liesen siam j sci comput 26 2005 pp 1598 1619 to matrices with nonzero 2 2 block and to the use of approximate schur complements to demonstrate the effectiveness of these preconditioners we show convergence results spectra and eigenvalue bounds for two model navier stokes problems

convergence properties of trust region methods for linear and convex constraints we develop a convergence theory for convex and linearly constrained trust region methods which only requires that the step between iterates produce a sufficient reduction in the trust region subproblem global convergence is established for general convex constraints while the local analysis is for linearly constrained problems the main local result establishes that if the sequence converges to a nondegenerate stationary point then the active constraints at the solution are identified in a finite number of iterations as a consequence of the identification properties we develop rate of convergence results by assuming that the step is a truncated newton method our development is mainly geometrical this approach allows the development of a convergence theory without any linear independence assumptions

logarithmic order and type of indeterminate moment problems ii we show that there exist indeterminate stieltjes moment problems with prescribed common logarithmic order and type

newton s method for large bound constrained optimization problems we analyze a trust region version of newton s method for bound constrained problems our approach relies on the geometry of the feasible set not on the particular representation in terms of constraints the convergence theory holds for linearly constrained problems and yields global and superlinear convergence without assuming either strict complementarity or linear independence of the active constraints we also show that the convergence theory leads to an efficient implementation for large bound constrained problems

indeterminate moment problems and the theory of entire functions the set of solutions to an indeterminate hamburger moment problem is given by the nevanlinna parametrization involving four entire functions a b c d we review the properties of these functions and discuss recent progress in the still open problem of characterizing the class of entire functions which occur in this way they belong to the krein class and are of minimal exponential type we deduce the analogous parametrization of the set of solutions to an indeterminate stieltjes moment problem from the corresponding symmetric hamburger problem the survey ends with a discussion of three special cases where the functions a b c d are known explicitly

the efficient computation of sparse jacobian matrices using automatic differentiation this paper is concerned with the efficient computation of sparse jacobian matrices of nonlinear vector maps using automatic differentiation ad specifically we propose the use of a graph coloring technique bicoloring to exploit the sparsity of the jacobian matrix j and thereby allow for the efficient determination of j using ad software we analyze both a direct scheme and a substitution process we discuss the results of numerical experiments indicating significant practical potential of this approach

indeterminate symmetric moment problems let n x be a sequence of orthogonal polynomials with respect to a real distribution according to a classical theorem of reisz 191 x x is complete in l if and only if w is either the solution of a determined hamburger moment problem or an extremal solution of an indeterminate moment problem in the special case where the zeros of the orthogonal polynomials have a finite infimum a and the hamburger moment problem is indeterminate there is an extremal solution which is essentially the only solution whose spectrum is a subset of a co 4 this solution is in a sense the best solution from the viewpoint of orthogonal polynomials since its spectrum consists of the limit points of the zeros of the polynomials askey remarked to the author that in view of riesz theorem it is surprising that so little attention has been paid to the question of extremal solutions moak 8 has recently studied the extremal solutions associated with the q laguerre nee generalized stieltjes wigert polynomials askey suggested it would be desirable to investigate the existence of best extremal solutions in the general case or as a first step even in the symmetric case moments of odd order are 0 in this paper we will consider this special case and the related systems of orthogonal polynomials in particular we will show that an indeterminate symmetric moment problem has precisely two symmetric extremal solutions this result which we found surprizing at first appears after reflection quite natural and perhaps obvious since the two solutions are the symmetrizations of the best extremal solutions of the related moment problems on 0 co

approximate graph coloring by semidefinite programming we consider the problem of coloring k colorable graphs with the fewest possible colors we present a randomized polynomial time algorithm that colors a 3 colorable graph on n vertices with min o 1 3 log 1 2 log n o n 1 4 log 1 2 n colors where is the maximum degree of any vertex besides giving the best known approximation ratio in terms of n this marks the first nontrivial approximation result as a function of the maximum degree this result can be generalized to k colorable graphs to obtain a coloring using min o 1 2 k log 1 2 log n o n 1 3 k 1 log 1 2 n colors our results are inspired by the recent work of goemans and williamson who used an algorithm for semidefinite optimization problems which generalize linear programs to obtain improved approximations for the max cut and max 2 sat problems an intriguing outcome of our work is a duality relationship established between the value of the optimum solution to our semidefinite program and the lovasz t function we show lower bounds on the gap between the optimum solution of our semidefinite program and the actual chromatic number by duality this also demonstrates interesting new facts about the t function

a simple parallel algorithm for the maximal independent set problem two basic design strategies are used to develop a very simple and fast parallel algorithms for the maximal independent set mis problem the first strategy consists of assigning identical copies o

new acyclic and star coloring algorithms with application to computing hessians acyclic and star coloring problems are specialized vertex coloring problems that arise in the efficient computation of hessians using automatic differentiation or finite differencing when both sparsity and symmetry are exploited we present an algorithmic paradigm for finding heuristic solutions for these two np hard problems the underlying common technique is the exploitation of the structure of two colored induced subgraphs for a graph g on n vertices and m edges the time complexity of our star coloring algorithm is o n overline d 2 where overline d k a generalization of vertex degree denotes the average number of distinct paths of length at most k edges starting at a vertex in g the time complexity of our acyclic coloring algorithm is larger by a multiplicative factor involving the inverse of ackermann s function the space complexity of both algorithms is o m to the best of our knowledge our work is the first practical algorithm for the acyclic coloring problem for the star coloring problem our algorithm uses fewer colors and is considerably faster than a previously known o n overline d 3 time algorithm computational results from experiments on various large size test graphs demonstrate that the algorithms are fast and produce highly effective solutions the use of these algorithms in hessian computation is expected to reduce overall runtime drastically

distributed memory parallel algorithms for distance 2 coloring and related problems in derivative computation the distance 2 graph coloring problem aims at partitioning the vertex set of a graph into the fewest sets consisting of vertices pairwise at distance greater than 2 from each other its applications include derivative computation in numerical optimization and channel assignment in radio networks we present efficient distributed memory parallel heuristic algorithms for this np hard problem as well as for two related problems used in the computation of jacobians and hessians parallel speedup is achieved through graph partitioning speculative iterative coloring and a bulk synchronous parallel like organization of parallel computation results from experiments conducted on a pc cluster employing up to 96 processors and using large size real world as well as synthetically generated test graphs show that the algorithms are scalable in terms of quality of solution the algorithms perform remarkably well the numbers of colors used by the parallel algorithms are observed to be very close to the numbers used by their sequential counterparts which in turn are quite often near optimal moreover the experimental results show that the parallel distance 2 coloring algorithm compares favorably with the alternative approach of solving the distance 2 coloring problem on a graph mathcal g by first constructing the square graph mathcal g 2 and then applying a parallel distance 1 coloring algorithm on mathcal g 2 implementations of the algorithms are made available via the zoltan toolkit

distributed memory parallel algorithms for distance 2 coloring and their application to derivative computation the distance 2 graph coloring problem aims at partitioning the vertex set of a graph into the fewest sets consisting of vertices pairwise at distance greater than two from each other its applications include derivative computation in numerical optimization and channel assignment in radio networks we present efficient distributed memory parallel heuristic algorithms for this nphard problem as well as for two related problems used in the computation of jacobians and hessians parallel speedup is achieved through graph partitioning speculative iterative coloring and a bsplike organization of parallel computation results from experiments conducted on a pc cluster employing up to 96 processors and using large size real world as well as synthetically generated test graphs show that the algorithms are scalable in terms of quality of solution the algorithms perform remarkably well the number of colors used by the parallel algorithms was observed to be very close to the number used by the sequential counterparts which in turn are quite often near optimal moreover the experimental results show that the parallel distance 2 coloring algorithm compares favorably with the alternative approach of solving the distance 2 coloring problem on a graph g by first constructing the square graph g2 and then applying a parallel distance 1 coloring algorithm on g2 implementations of the algorithms are made available via the zoltan load balancing library

discrete hilbert transforms on sparse sequences weighted discrete hilbert transforms an n p n anvn z n from 2 to a weighted l 2 space are studied with n a sequence of distinct points in the complex plane and v vn a corresponding sequence of positive numbers in the special case when j nj grows at least exponentially bounded transforms of this kind are described in terms of a simple relative to the muckenhoupt a2 condition the special case when z is restricted to another sequenceis studied in detail it is shown that a bounded trans form satisfying a certain admissibility condition can be split into finitely many surjective transforms and precise geometric conditions are found for invertibility of such two weight transforms these results can be interpreted as statements about systems of reproducing kernels in certain hilbert spaces of which de branges spaces and model subspaces of h 2 are prime examples in particular a connection to the feichtinger conjecture is pointed out descriptions of carleson measures and riesz bases of normalized reproducing kernels for certain small de branges spaces follow from the results of this paper 1 introduction and main results this paper is concerned with the mapping properties of what we will call discrete hilbert transforms in the complex plane one aspect of this topic was treated in 7 where all unitary discrete hilbert transforms were described when we now turn to questions about boundedness surjectivity and invertibility results of the same generality seem at present out of reach the results to be presented below are complete only when the discrete hilbert transforms are defined on particularly sparse sequences we will nevertheless present the problems in the most general setting as we believe they merit further investigations we will emphasize the connection with topics such as carleson measures and riesz bases of normalized reproducing kernels in hilbert spaces of analytic functions this will lead us to the most intriguing general question namely whether or not the feichtinger conjecture holds true for systems of reproducing kernels in such spaces we begin by assuming that we are given an infinite sequence of distinct points n in c and a corresponding sequence of positive numbers v vn both indexed by the

a preconditioning framework for sequences of diagonally modified linear systems arising in optimization we propose a framework for building preconditioners for sequences of linear systems of the form a delta k x k b k where a is symmetric positive semidefinite and delta k is diagonal positive semidefinite such sequences arise in several optimization methods e g in affine scaling methods for bound constrained convex quadratic programming and bound constrained linear least squares as well as in trust region and overestimation methods for convex unconstrained optimization problems and nonlinear least squares for all the matrices of a sequence the preconditioners are obtained by updating any preconditioner for a available in the ldl t form the preconditioners in the framework satisfy the natural requirement of being effective on slowly varying sequences furthermore under an additional property they are also able to cluster eigenvalues of the preconditioned matrix when some entries of delta k are sufficiently large we present two low cost preconditioners sharing the above mentioned p

automatic preconditioning by limited memory quasi newton updating this paper proposes a preconditioner for the conjugate gradient method cg that is designed for solving systems of equations ax bi with different right hand side vectors or for solving a sequence of slowly varying systems ak x bk the preconditioner has the form of a limited memory quasi newton matrix and is generated using information from the cg iteration the automatic preconditioner does not require explicit knowledge of the coefficient matrix a and is therefore suitable for problems where only products of a times a vector can be computed numerical experiments indicate that the preconditioner has most to offer when these matrix vector products are expensive to compute and when low accuracy in the solution is required the effectiveness of the preconditioner is tested within a hessian free newton method for optimization and by solving certain linear systems arising in finite element models

estimation of sparse hessian matrices and graph coloring problems large scale optimization problems often require an approximation to the hessian matrix if the hessian matrix is sparse then estimation by differences of gradients is attractive because the number of required differences is usually small compared to the dimension of the problem the problem of estimating hessian matrices by diferences can be phrased as follows given the sparsity structure of a symmetric matrix a obtain vectors d 1 d 2 ldots d p such that ad 1 ad 2 ldots ad p determine a uniquely with p as small as possible we approach this problem from a graph theoretic point of view and show that both direct and indirect approaches to this problem have a natural graph coloring interpretation the complexity of the problem is analyzed and efficient practical heuristic procedures are developed numerical results illustrate the differences between the various approaches

improving the performance guarantee for approximate graph coloring the performance guarantee of a graph coloring algorithm is the worst case ratio between the number of colors it uses on the input graph and the chromauc number of this graph the previous best known polynomial time algorithm had a performance guarantee o n logn for graphs on n vertices this result stood unchallenged for eight years this paper presents an efficient algorithm with performance guarantee of o n loglog n 2 logn 2

free bits pcps and nonapproximability towards tight results this paper continues the investigation of the connection between probabilistically checkable proofs pcps and the approximability of np optimization problems the emphasis is on proving tight nonapproximability results via consideration of measures such as the free bit complexity and the amortized free bit complexity of proof systems the first part of the paper presents a collection of new proof systems based on a new error correcting code called the long code we provide a proof system that has amortized free bit complexity of 2 epsilon implying that approximating maxclique within n frac13 e and approximating the chromatic number within n frac15 e are hard assuming np neq corp for any e 0 we also derive the first explicit and reasonable constant hardness factors for min vertex cover msat 2 and max cut and we improve the hardness factor for msat 3 we note that our nonapproximability factors for maxsnp problems are appreciably close to the values known to be achievable by polynomial time algorithms finally we note a general approach to the derivation of strong nonapproximability results under which the problem reduces to the construction of certain gadgets the increasing strength of nonapproximability results obtained via the pcp connection motivates us to ask how far this can go and whether pcps are inherent in any way the second part of the paper addresses this the main result is a reversal of the connection due to feige et al fglss connection j acm 43 1996 pp 268 292 where the latter had shown how to translate proof systems for np into np hardness of approximation results for maxclique we show how any np hardness of approximation result for maxclique yields a proof system for np roughly our result says that for any constant f if maxclique is np hard to approximate within n 1 1 f then np subseteq overline fpcp log f the latter being the class of languages possessing proofs of logarithmic randomness and amortized free bit complexity f this suggests that pcps are inherent to obtaining nonapproximability results furthermore the tight relation suggests that reducing the amortized free bit complexity is necessary for improving the nonapproximability results for maxclique the third part of our paper initiates a systematic investigation of the properties of pcp and fpcp free pcp as a function of the following various parameters randomness query complexity free bit complexity amortized free bit complexity proof size etc we are particularly interested in triviality results which indicate which classes are not powerful enough to capture np we also distill the role of randomized reductions in this area and provide a variety of useful transformations between proof checking complexity classes

optimal accumulation of jacobian matrices by elimination methods on the dual computational graph the accumulation of the jacobian matrix f of a vector function equation here can be regarded as a transformation of its linearized computational graph into a subgraph of the directed complete bipartite graph kn m this transformation can be performed by applying different elimination techniques that may lead to varying costs for computing f this paper introduces face elimination as the basic technique for accumulating jacobian matrices by using a minimal number of arithmetic operations its superiority over both edge and vertex elimination methods is shown the intention is to establish the conceptual basis for the ongoing development of algorithms for optimizing the computation of jacobian matrices

what color is your jacobian graph coloring for computing derivatives graph coloring has been employed since the 1980s to efficiently compute sparse jacobian and hessian matrices using either finite differences or automatic differentiation several coloring problems occur in this context depending on whether the matrix is a jacobian or a hessian and on the specifics of the computational techniques employed we consider eight variant vertex coloring problems here this article begins with a gentle introduction to the problem of computing a sparse jacobian followed by an overview of the historical development of the research area then we present a unifying framework for the graph models of the variant matrix estimation problems the framework is based upon the viewpoint that a partition of a matrix into structurally orthogonal groups of columns corresponds to distance 2 coloring an appropriate graph representation the unified framework helps integrate earlier work and leads to fresh insights enables the design of more efficient algorithms for many problems leads to new algorithms for others and eases the task of building graph models for new problems we report computational results on two of the coloring problems to support our claims most of the methods for these problems treat a column or a row of a matrix as an atomic entity and partition the columns or rows or both a brief review of methods that do not fit these criteria is provided we also discuss results in discrete mathematics and theoretical computer science that intersect with the topics considered here

efficient preconditioning of sequences of nonsymmetric linear systems we present a new approach for approximate updates of factorized nonsymmetric preconditioners for solving sequences of linear algebraic systems this approach is algebraic and it is theoretically motivated it generalizes diagonal updates introduced by benzi and bertaccini bit 43 2003 pp 231 244 and bertaccini electron trans numer anal 18 2004 pp 49 64 it is shown experimentally that this approach can be very beneficial for example it is successful in significantly decreasing the number of iterations of a preconditioned iterative method for solving subsequent systems of a sequence when compared with freezing the preconditioner from the first system of the sequence in some cases the updated preconditioners offer a rate of convergence similar to or even higher than the rate obtained when preconditioning with recomputed preconditioners since the updates are typically cheap and straightforward their use is of practical interest they can replace recomputing preconditioners which is often expensive especially in parallel and matrix free environments