successive column correction algorithms for solving sparse nonlinear systems of equations this paper presents two algorithms for solving sparse nonlinear systems of equations the cm successive column correction algorithm and a modified cm successive column correction algorithm aq superlinear convergence theorem and anr convergence order estimate are given for both algorithms some numerical results and the detailed comparisons with some previously established algorithms show that the new algorithms have some promise of being very effective in practice

primes in short intervals contrary to what would be predicted on the basis of cramer s model con cerning the distribution of prime numbers we develop evidence that the distribution of x h x for 0 x n is approximately normal with mean h and variance h log n h when n h n 1

tracing the equilibrium path beyond compound critical points a unified presentation of some popular continuation procedures used in the non linear finite element analysis of structural mechanics is introduced an extension of the elliptical constraint equation proposed by crisfield is given it is shown that in the proposed procedure real roots can always be obtained in solving the iterative change of the load parameter updated weighting factors are introduced in the constraint equation in order to get better convergence characteristics in the case when localized deformations occur for bifurcation points a modification of rheinboldt s branching procedure is presented post critical response after limit and bifurcation points is determined in some numerical examples

optimal approximation of sparse hessians and its equivalence to a graph coloring problem we consider the problem of approximating the hessian matrix of a smooth non linear function using a minimum number of gradient evaluations particularly in the case that the hessian has a known fixed sparsity pattern we study the class of direct methods for this problem and propose two new ways of classifying direct methods examples are given that show the relationships among optimal methods from each class the problem of finding a non overlapping direct cover is shown to be equivalent to a generalized graph coloring problem the distance 2 graph coloring problem a theorem is proved showing that the general distance k graph coloring problem is np complete for all fixedk 2 and hence that the optimal non overlapping direct cover problem is also np complete some worst case bounds on the performance of a simple coloring heuristic are given an appendix proves a well known folklore result which gives lower bounds on the number of gradient evaluations needed in any possible approximation method

a numerical study of the limited memory bfgs method and the truncated newton method for large scale optimization this paper examines the numerical performances of two methods for large scale optimization a limited memory quasi newton method l bfgs and a discrete truncated newton method tn various ways of classifying test problems are discussed in order to better understand the types of problems that each algorithm solves well the l bfgs and tn methods are also compared with the polak ribiere conjugate gradient method

on the successive projections approach to least squares problems in this paper we suggest a generalized gauss seidel approach to sparse linear and nonlinear least squares problems the algorithm closely related to one given by elfving 1980 uses the work of curtis powell and reid 1974 as extended by coleman and more 1983 to divide the variables into nondisjoint groups of structurally orthogonal columns and then projects the updated residual into each column subspace of the jacobian in turn in the linear case this procedure can be viewed as an alternate ordering of the variables in the gauss seidel method preliminary tests indicate that this leads quickly to cheap solutions of limited accuracy for linear problems and that this approach is promising for an inexact gauss newton analog of the inexact newton approach of dembo eisenstat and steihaug 1982

a natural classification of vibration modes of polygonal ducts based on group theoretic analysis in this work the natural frequencies and mode shapes of polygonal ducts are obtained by a finite element analysis via a group theoretic approach we show that the group theoretic techniques provide a natural setting for the solution of the problem via optimum utilization of the symmetry in the problem the simplifications and insights due to the inherent symmetry is intuitively obvious in simple symmetric structures however a systematic exploitation of this symmetry in more complicated structures is less obvious and is possible perhaps only via useful theorems from group representation theory and the associated projection operator theory we illustrate all the main steps with the help of an example of dihedral groups this example is directly applicable to the problem addressed in this paper the group theoretic approach splits up the original problem into independent subproblems and thereby affects significant computational savings more importantly the approach captures the role of symmetry in the problem and provides insights which are otherwise not obvious the multiplicity of the natural frequencies arising out of the inherent symmetry is determined a priori and neatly separated into different subproblems this suggests a neat physical classification of the mode shapes on a symmetry basis which is unique to this approach the values of the natural frequencies and the mode shapes for triangular square and pentagonal ducts are shown to be in good agreement with the existing results reported in the literature the classification of mode shapes on this symmetry basis is used in the analysis of the polygonal ducts with number of sides varying up to 16

variable metric methods for unconstrainted optimization and nonlinear least squares variable metric or quasi newton methods are well known and commonly used in connection with unconstrained optimization since they have good theoretical and practical convergence properties although these methods were originally developed for small and moderate size dense problems their modifications based either on sparse partitioned or limited memory updates are very efficient on large scale sparse problems very significant applications of these methods also appear in nonlinear least squares approximation and nonsmooth optimization in this contribution we give an extensive review of variable metric methods and their use in various optimization fields

optimal estimation of jacobian and hessian matrices that arise in finite difference calculations in this paper the problem of estimating jacobian and hessian matrices arising in the finite difference approximation of partial differential equations is considered using the notion of computational molecule or stencil schemes are developed that require the minimum number of differences to estimate these matrices a procedure applicable to more complicated structures is also given 1 introduction in the past few years there has been a growing interest in studying efficient ways to obtain good estimates of sparse jacobian or hessian matrices two basic approaches have been followed in the first approach it is assumed that one has an estimate of the jacobian hessian which one wishes to revise given arbi trary changes in the variables and the corresponding changes in the functions gradients this has led to the development of several sparse quasi newton methods see 11 14 12 for example in the second approach it is assumed that one can specify the changes in the variables one then forms an estimate of the jacobian hessian matrix from the observed changes in the functions gradients when the dimension of the matrix is large and the evaluation of each function gradient vector is expensive it is necessary to take the sparsity structure of the matrix into account in order to make this approach efficient several methods for doing this for arbitrary sparsity structures have already appeared in the literature 2 5 7 10 13 in this paper we consider the second approach for cases where the sparsity structure arises from finite difference approximations to partial differential equa tions both symmetric and unsymmetric matrices are considered although the algorithms presented do not generalize to every pattern generated by finite dif ferences they cover the most useful structures and in some cases extend existing methods to more complex situations in the next section we describe the method of curtis powell and reid 5 henceforth referred to as the cpr method for estimating sparse jacobian matrices we then show how the so called computational molecule or stencil 1 of the finite difference operator associated with the jacobian matrix under study can be used to determine the groupings i e function differencing directions used by the

efficient estimation of sparse jacobian matrices by differences abstract in difference newton like methods for solving f x 0 the jacobian matrix f x is approximated by differences between values of f if f x is sparse a consistent partition of its columns can be exploited to approximate f x using relatively few values of f we provide a local convergence theory for the resulting methods a superlinearly convergent stable cyclic secant method in which at each iteration two values of f are required and several columns of the jacobian matrix approximation are updated simultaneously is developed

automated generation of high order partial derivative models a standard problem in science and engineering consists of developing mathematics and sensitivity models for complex applications for optimizing a candidate design a chain rule based evaluation technique is presented for analytically evaluating partial derivatives of nonlinear functions and differential equations defined by a high level language a coordinate embedding strategy is introduced that replaces all scalar variables with higher dimensional objects the higher dimensional objects are defined by a concatenation of the original scalar and its jacobian hessian and higher order partials exact sensitivity models are recovered for arbitrarily complex mathematical models an operator overloading technique is used to define generalized operators for basic and standard library functions the generalized operators encode the chain rule of calculus and store the results of partial derivative calculations in the artificial dimensions used to redefine the scalar operations hidden operations automatically generate and evaluate exact first through fourth order partial derivative models which are accurate to the working precision of the machine the new algorithm replaces a normally complex error prone time consuming and labor intensive process for producing the partials with an automatic procedure module functions encapsulate new data types and extended mathematic and library functions for handling vector matrix and tensor operations matrix operations are shown to generalize easily the algorithm has broad potential for impacting the design and use of mathematical programming tools for applications in science and engineering several applications are presented that demonstrate the effectiveness of the methods

a scalable fully implicit compressible euler solver for mesoscale nonhydrostatic simulation of atmospheric flows a fully implicit solver is developed for the mesoscale nonhydrostatic simulation of atmospheric flows governed by the compressible euler equations to spatially discretize the euler equations on a height based terrain following mesh we apply a cell centered finite volume scheme in which an advection upstream splitting ausm up method with a piecewise linear reconstruction is employed to achieve second order accuracy for the low mach flow a second order explicit first step single diagonal coefficient diagonally implicit runge kutta esdirk method with adaptive time stepping is applied to stabilize physically insignificant fast waves and accurately integrate the euler equations in time the nonlinear system arising at each time step is solved by using a jacobian free newton krylov schwarz nks algorithm to accelerate the convergence and improve the robustness we employ a class of additive schwarz preconditioners in which the subdomain jacobian matrix is constructed using a first order spatial discretization several test cases are used to validate the correctness of the scheme and examine the performance of the solver large scale results on a supercomputer with up to 18 432 processor cores are provided to show the parallel performance of the proposed method

a group theoretic approach to the global bifurcation analysis of an axially compressed cylindrical shell abstract the accurate prediction of the buckling load of thin shell structures is an important yet elusive goal it is particularly important in the aerospace industry where thin shell members are commonly used as structural elements due to a lack of adequate analytical results current practices in industry put heavy reliance on experimental testing and empirical data to supplement theoretical analysis see d bushnell computerized buckling analysis of shells martinus nijhoff dordrecht 1985 this paper focuses on recent results of the group theoretic approach to a numerical global postbuckling analysis of a perfect axially compressed cylindrical shell with built in end conditions the built in end conditions obviate the existence of a trivial membrane solution branch the example of an axially compressed cylindrical shell was chosen because it is well known that for thin shells the primary axisymmetric solution branch is riddled with closely spaced symmetry breaking bifurcation points in a numerical arc length continuation scheme the close proximity of the bifurcation points on the primary path manifests itself in severe ill conditioning of the tangent stiffness matrix group theory helps one systematically find an optimal set of basis vectors or symmetry modes which reflect the symmetry of a given solution path the immediate payoff in using these symmetry modes as basis vectors is that the tangent stiffness matrix block diagonalizes and the numerical ill conditioning is avoided thus an efficient and accurate technique for computing solution branches of a specific type and a subsequent diagnosis for symmetry breaking bifurcations is made relatively simple understanding the global behavior of the perfect structure is crucial in identifying critical imperfections and will ultimately diminish the heavy reliance on expensive experimental verification

a group theoretic approach to the linear free vibration analysis of shells with dihedral symmetry this paper deals with a group theoretic approach to the finite element analysis of linear free vibrations of shells with dihedral symmetry examples of such shell structures are cylindrical shells conical shells shells with circumferential stiffeners corrugated shells spherical shells etc the group theoretic approach is used to exploit the inherent symmetry in the problem for vibration analysis the group theoretic results give the correct symmetry adapted basis for the displacement field the stiffness matrix k and the mass matrix m are identically block diagonalized in this basis the generalized linear eigenvalue problem of free vibration gets split into independent subproblems due to this block diagonalization the simo element is used in the finite element formulation of the shell equilibrium equations numerical results for natural frequencies and natural modes of vibration of several dihedral shell structures are presented the results are shown to be in very good agreement with those reported in the literature the computational advantages and physical insights due to the group theoretic approach are also discussed

group theoretic exploitations of symmetry in computational solid and structural mechanics the use of group theory in simplifying the study of problems involving symmetry is a well established approach in various branches of physics and chemistry and major applications in these areas date back more than 70 years within the engineering disciplines the search for more systematic and more efficient strategies for exploiting symmetry in the computational problems of solid and structural mechanics has led to the development of group theoretic methods over the past 40 years this paper reviews the advances made in the application of group theory in areas such as bifurcation analysis vibration analysis and finite element analysis and summarizes the various implementation procedures currently available illustrative examples of typical solution procedures are drawn from recent work of the author it is shown how the group theoretic approach through the characteristic vector space decomposition enables considerable simplifications and reductions in computational effort to be achieved in many cases group theoretic considerations also allow valuable insights on the behaviour or properties of a system to be gained before any actual calculations are carried out copyright 2009 john wiley sons ltd

real time computation at the edge of chaos in recurrent neural networks depending on the connectivity recurrent networks of simple computational units can show very different types of dynamics ranging from totally ordered to chaotic we analyze how the type of dynamics ordered or chaotic exhibited by randomly connected networks of threshold gates driven by a time varying input signal depends on the parameters describing the distribution of the connectivity matrix in particular we calculate the critical boundary in parameter space where the transition from ordered to chaotic dynamics takes place employing a recently developed framework for analyzing real time computations we show that only near the critical boundary can such networks perform complex computations on time series hence this result strongly supports conjectures that dynamical systems that are capable of doing complex computational tasks should operate near the edge of chaos that is the transition from ordered to chaotic dynamics

some computational aspects of the stability analysis of nonlinear structures abstract the perturbation method and the continuation method are the two most popular techniques for the solution of finite element equations that describe instability phenomena this paper presents a summary of the principles involved and describes some trends in the development of these techniques

on the decomposition of generalized eigenproblems for the free vibration analysis of cyclically symmetric finite element models free vibration analysis is a major part of any dynamic analysis natural frequencies and related mode shapes may be obtained from free vibration analysis as the solutions of generalized eigenproblems although the eigensolutions of large scale structures require large computational efforts these solutions may be achieved simply for symmetric structures we present an efficient method for the decomposition of generalized eigenproblems related to finite element models with cyclic symmetry having nodes at the axis of symmetry into eigensubproblems with significantly smaller dimensions this decomposition is obtained by block diagonalization of a matrix with a special pattern known as a block circulant using the concept of the kronecker product and similarity transformations the proposed method is applied to three finite element models discretized by triangular and four node quadrilateral plate and shell elements and its efficiency accuracy and simplicity are evaluated copyright 2009 john wiley sons ltd

post critical analysis of structures with a nonlinear pre buckling state in the presence of imperfections in the context of the geometrically nonlinear analysis of structures by continuation methods a technique for the determination of the imperfection sensitivity is carried out the technique is based on galerkin s reduction of equilibrium equations in the subspace of extrapolation branches emanating from the bifurcation points the pre and post critical behaviour in the neighbourhood of bifurcation points however is well approximated provided that extrapolations have a large convergence domain to this end an efficient asymptotic representation of the primary and secondary equilibrium paths and procedures that overcome the ill conditioning of the systems defined in the critical points is used here

efficient calculation of jacobian and adjoint vector products in the wave propagational inverse problem using automatic differentiation wave propagational inverse problems arise in several applications including medical imaging and geophysical exploration in these problems one is interested in obtaining the parameters describing the medium from its response to excitations the problems are characterized by their large size and by the hyperbolic equation which models the physical phenomena the inverse problems are often posed as a nonlinear data fitting where the unknown parameters are found by minimizing the misfit between the predicted data and the actual data in order to solve the problem numerically using a gradient type approach one must calculate the action of the jacobian and its adjoint on a given vector in this paper we explore the use of automatic differentiation ad to develop codes that perform these calculations we show that by exploiting structure at 2 scales we can arrive at a very efficient code whose main components are produced by ad in the first scale we exploite the time stepping nature of the hyperbolic solver by using the extended jacobian framework in the second finer scale we exploit the finite difference stencil in order to make explicit use of the sparsity in the dependence of the output variables to the input variables the main ideas in this work are illustrated with a simpler one dimensional version of the problem numerical results are given for both one and two dimensional problems we present computational templates that can be used in conjunction with optimization packages to solve the inverse problem

optimization of unconstrained functions with sparse hessian matrices newton type methods newton type methods for unconstrained optimization problems have been very successful when coupled with a modified cholesky factorization to take into account the possible lack of positive definiteness in the hessian matrix in this paper we discuss the application of these method to large problems that have a sparse hessian matrix whose sparsity is known a priori

a chordal preconditioner for large scale optimization we propose an automatic preconditioning scheme for large sparse numerical optimization the strategy is based on an examination of the sparsity pattern of the hessian matrix using a graph theoretic heuristic a block diagonal approximation to the hessian matrix is induced the blocks are submatrices of the hessian matrix furthermore each block is chordal that is under a positive definiteness assumption each block can be cholesky factored without creating new nonzeroes fill therefore the preconditioner is space efficient we conduct a number of numerical experiments to determine the effectiveness of the preconditioner in the context of a linear conjugate gradient algorithm for optimization

on the calculation of limit and bifurcation points in stability problems of elastic shells abstract numerical methods for the computation of singular points of nonlinear equations g u 0 are discussed where and are real parameters simple and double limit points are treated in some detail and numerical algorithms are presented and applied to elastic shell stability problems the case of simple symmetry breaking bifurcation points is also treated with applications to nonsymmetric bifurcation from axisymmetric states of deformation of shells of revolution

a matrix free approach to build band preconditioners for large scale bound constrained optimization abstract we propose a procedure for building symmetric positive definite band preconditioners for large scale symmetric possibly indefinite linear systems when the coefficient matrix is not explicitly available but matrix vector products involving it can be computed we focus on linear systems arising in newton type iterations within matrix free versions of projected methods for bound constrained nonlinear optimization in this case the structure and the size of the matrix may significantly change in subsequent iterations and preconditioner updating algorithms that exploit information from previous steps cannot be easily applied our procedure is based on a recursive approach that incrementally improves the quality of the preconditioner while requiring a modest number of matrix vector products a strategy for dynamically choosing the bandwidth of the preconditioners is also presented numerical results are provided showing the performance of our preconditioning technique within a trust region newton method for bound constrained optimization

two dimensional fully adaptive solutions of solid solid alloying reactions abstract solid solid alloying reactions occur in a variety of pyrotechnical applications they arise when a mixture of powders composed of appropriate oxidizing and reducing agents is heated the large quantity of heat evolved produces a self propagating reaction front that is often very narrow with sharp changes in both the temperature and the concentrations of the reacting species solution of problems of this type with an equispaced or mildly nonuniform grid can be extremely inefficient in this paper we develop a two dimensional fully adaptive method for solving problems of this class the method adaptively adjusts the number of grid points needed to equidistribute a positive weight function over a given mesh interval in each direction at each time level we monitor the solution from one time level to another to ensure that the local error per unit step associated with the time differencing method is below some specified tolerance the method is applied to several examples involving exothermic diffusion controlled self propagating reactions in packed bed reactors

ilum a multi elimination ilu preconditioner for general sparse matrices standard preconditioning techniques based on incomplete lu ilu factorizations offer a limited degree of parallelism in general a few of the alternatives advocated so far consist of either using some form of polynomial preconditioning or applying the usual ilu factorization to a matrix obtained from a multicolor ordering in this paper we present an incomplete factorization technique based on independent set orderings and multicoloring we note that in order to improve robustness it is necessary to allow the preconditioner to have an arbitrarily high accuracy as is done with ilus based on threshold techniques the ilum factorization described in this paper is in this category it can be viewed as a multifrontal version of a gaussian elimination procedure with threshold dropping which has a high degree of potential parallelism the emphasis is on methods that deal specifically with general unstructured sparse matrices such as those arising from finite element methods on unstructured meshes

on the relation between two local convergence theories of least change secant update methods in this paper we show that the main results of the local convergence theory for least change secant update methods of dennis and walker siam j numer anal 18 1981 949 987 can be proved using the theory introduced recently by martinez math comp 55 1990 143 167 in addition we exhibit two generalizations of well known methods whose local convergence can be easily proved using martinez s theory

detection and analysis of irregular streaks in dermoscopic images of skin lesions irregular streaks are important clues for melanoma a potentially fatal form of skin cancer diagnosis using dermoscopy images this paper extends our previous algorithm to identify the absence or presence of streaks in a skin lesions by further analyzing the appearance of detected streak lines and performing a three way classification for streaks absent regular and irregular in a pigmented skin lesion in addition the directional pattern of detected lines is analyzed to extract their orientation features in order to detect the underlying pattern the method uses a graphical representation to model the geometric pattern of valid streaks and the distribution and coverage of the structure using these proposed features of the valid streaks along with the color and texture features of the entire lesion an accuracy of 76 1 and weighted average area under roc curve auc of 85 is achieved for classifying dermoscopy images into streaks absent regular or irregular on 945 images compiled from atlases and the internet without any exclusion criteria this challenging dataset is the largest validation dataset for streaks detection and classification published to date the data set has also been applied to the two class sub problems of absent present classification accuracy of 78 3 with auc of 83 2 and to regular irregular classification accuracy 83 6 with auc of 88 9 when the method was tested on a cleaned subset of 300 images randomly selected from the 945 images the auc increased to 91 8 93 2 and 90 9 for the absent regular irregular absent present and regular irregular problems respectively

tensor methods for large sparse nonlinear least squares problems this paper introduces tensor methods for solving large sparse nonlinear least squares problems where the jacobian either is analytically available or is computed by finite difference approximations tensor methods have been shown to have very good computational performance for small to medium sized dense nonlinear least squares problems in this paper we consider the application of tensor methods to large sparse nonlinear least squares problems where a sparse factorization of the jacobian can be stored this involves an entirely new way of solving the tensor model that is efficient for sparse problems a number of interesting linear algebraic implementation issues are addressed the test results of the tensor method applied to a set of sparse nonlinear least squares problems compared with those of the standard gauss newton method reveal that the tensor method is significantly more robust and efficient than the standard gauss newton method

multilevel schwarz methods for elliptic partial differential equations we investigate multilevel schwarz domain decomposition preconditioners to efficiently solve linear systems arising from numerical discretizations of elliptic partial differential equations by the finite element method in our analysis we deal with unstructured mesh partitions and with subdomain boundaries resulting from using the mesh partitioner we start from two level preconditioners with either aggregative or interpolative coarse level components then we focus on a strategy to increase the number of levels for all preconditioners we consider the additive residual update and its multiplicative variants within and between levels moreover we compare the preconditioners behaviour regarding scalability and rate of convergence numerical results are provided for elliptic boundary value problems including a convection diffusion problem when suitable stabilization becomes necessary c 2011 elsevier b v all rights reserved

estimation of sparse jacobian matrices and graph coloring blems given a mapping with a sparse jacobian matrix the problem of minimizing the number of function evaluations needed to estimate the jacobian matrix by differences is investigated this problem can be attacked as a graph coloring problem and this approach leads to very efficient algorithms the behavior of these algorithms is studied and in particular it is proved that two of the algorithms are optimal for band graphs numerical evidence is presented which indicates that these two algorithms are nearly optimal on practical problems

the structure factor of primes although the prime numbers are deterministic they can be viewed by some measures as pseudo random numbers in this article we numerically study the pair statistics of the primes using statistical mechanical methods particularly the structure factor in an interval with m large and l m smaller than unity we show that the structure factor of the prime number configurations in such intervals exhibits well defined bragg like peaks along with a small diffuse contribution this indicates that primes are appreciably more correlated and ordered than previously thought our numerical results definitively suggest an explicit formula for the locations and heights of the peaks this formula predicts infinitely many peaks in any non zero interval similar to the behavior of quasicrystals however primes differ from quasicrystals in that the ratio between the location of any two predicted peaks is rational we also show numerically that the diffuse part decays slowly as m and l increases this suggests that the diffuse part vanishes in an appropriate infinite system size limit

computational methods for bifurcation problems with symmetries with special attention to steady state and hopf bifurcation points we show how group theoretical methods can be employed to utilize the symmetry of a bifurcation problem in numerical computations we extend the approach by werner 1988 by presenting methods for the detection of bifurcation points and the computation of multiple hopf points the essential numerical point is the utilization of certain reduced instead of full systems involving appropriate subgroups of the underlying symmetry group the group theoretical tool is an a priori knowledge of the interaction of certain subgroups 0 and of at possibly multiple steady state or hopf bifurcation points minimal 0 breaking bifurcation we introduce a bifurcation graph which shows graphically this a priori information its edges represent possible symmetry breaking bifurcations our analysis follows the lines of golubitsky stewart and schaeffer 1988 but it is aimed to numerical applications we have chosen a 4 box brusselator model in order to explain our notions and ideas and to discuss the numerical procedure

cheaper jacobians by simulated annealing jacobian matrices can be accumulated by applying the chain rule to vector functions given as computer programs in different orders resulting in varying operations counts while yielding identical results up to round off the minimization of the number of operations performed leads to a computationally hard combinatorial optimization problem based on vertex elimination in computational graphs this paper discusses simulated annealing as a method for generating nearly optimal jacobian code

large scale sparse inverse covariance matrix estimation the estimation of large sparse inverse covariance matrices is a ubiquitous statistical problem in many application areas such as mathematical finance geology health and many others the ell 1

determination of the edge of criticality in echo state networks through fisher information maximization it is a widely accepted fact that the computational capability of recurrent neural networks rnns is maximized on the so called edge of criticality once the network operates in this configuration it performs efficiently on a specific application both in terms of 1 low prediction error and 2 high short term memory capacity since the behavior of recurrent networks is strongly influenced by the particular input signal driving the dynamics a universal application independent method for determining the edge of criticality is still missing in this paper we aim at addressing this issue by proposing a theoretically motivated unsupervised method based on fisher information for determining the edge of criticality in rnns it is proved that fisher information is maximized for finite size systems operating in such critical regions however fisher information is notoriously difficult to compute and requires the analytic form of the probability density function ruling the system behavior this paper takes advantage of a recently developed nonparametric estimator of the fisher information matrix and provides a method to determine the critical region of echo state networks esns a particular class of recurrent networks the considered control parameters which indirectly affect the esn performance are explored to identify those configurations lying on the edge of criticality and as such maximizing fisher information and computational performance experimental results on benchmarks and real world data demonstrate the effectiveness of the proposed method

preconditioners for generalized saddle point problems noindent we propose and examine block diagonal preconditioners and variants of indefinite preconditioners for block two by two generalized saddle point problems that is we consider the nonsymmetric nonsingular case where the 2 2 block is small in norm and we are particularly concerned with the case where the 1 2 block is different from the transposed 2 1 block we provide theoretical and experimental analyses of the convergence and eigenvalue distributions of the preconditioned matrices we also extend the results of de sturler and liesen siam j sci comput 26 2005 pp 1598 1619 to matrices with nonzero 2 2 block and to the use of approximate schur complements to demonstrate the effectiveness of these preconditioners we show convergence results spectra and eigenvalue bounds for two model navier stokes problems

branch switching for pitchfork bifurcation in nonlinear finite element method a general branch switching approach is proposed based on pseudo arc length continuation in the nonlinear finite element method fem to trace the branch solution curve at the pitchfork bifurcation point in parameter space by this approach branch direction can be determined without derivatives of tangent stiffness matrix this approach is proved mathematically and is inserted into a general fem code stability of equilibrium corresponding to each solution curve is estimated by the lagrange dirichlet criteria as an example the whole process of the elastic mode jumping for rectangular thin plates in which there are many pitchfork bifurcations in parameter space in loading and unloading is simulated by the improved fem code the numerical results are identical with that of an experiment done by previous researchers

improvement of the scaled corrector method for bifurcation analysis using symmetry exploiting block diagonalization in the nonlinear bifurcation analysis for large scaled structures the standard eigenanalysis of the tangent stiffness matrix yields important information but at the same time demands a large amount of computational cost the scaled corrector method was developed as a numerically efficient eigenanalysis free bifurcation analysis strategy which exploits byproducts of the numerical iteration for path tracing this method however has a problem in its accuracy especially when eigenvalues are nearly or exactly coincidental as a remedy for this we propose a new bifurcation analysis method through the implementation of bifurcation mechanism of a symmetric structure into the scaled corrector method the bifurcation mode is accurately approximated by decomposing a scaled corrector vector into a number of vectors by means of block diagonalization method in group theoretic bifurcation theory and in turn by choosing the predominant one among these vectors in order to demonstrate the usefulness of this method it is applied to the bifurcation analysis of reticulated regular hexagonal truss domes to compute accurately the locations of double bifurcation points and nearly coincidental bifurcation points and associated bifurcation modes

parallel multilevel methods for implicit solution of shallow water equations with nonsmooth topography on the cubed sphere high resolution and scalable parallel algorithms for the shallow water equations on the sphere are very important for modeling the global climate in this paper we introduce and study some highly scalable multilevel domain decomposition methods for the fully implicit solution of the nonlinear shallow water equations discretized with a second order well balanced finite volume method on the cubed sphere with the fully implicit approach the time step size is no longer limited by the stability condition and with the multilevel preconditioners good scalabilities are obtained on computers with a large number of processors the investigation focuses on the use of semismooth inexact newton method for the case with nonsmooth topography and the use of two and three level overlapping schwarz methods with different order of discretizations for the preconditioning of the jacobian systems we test the proposed algorithm for several benchmark cases and show numerically that this approach converges well with smooth and nonsmooth bottom topography and scales perfectly in terms of the strong scalability and reasonably well in terms of the weak scalability on machines with thousands and tens of thousands of processors

a fully implicit domain decomposition algorithm for shallow water equations on the cubed sphere popular approaches for solving shallow water equations swes for climate modeling are explicit and semiimplicit methods and both have certain constraints on the time step size in this paper we propose and study a fully implicit method which imposes no limit on the time step size but requires the solution of a large sparse nonlinear system of equations at every time step the focus of the paper is a parallel fully coupled newton krylov ras algorithm with a jacobian matrix explicitly calculated on a weakly nonmatching cubed sphere mesh here ras is a restricted additive schwarz method we show numerically that with such a preconditioned nonlinearly implicit method the time step size is no longer constrained by the cfl condition and we report superlinear speedup of the algorithm on machines with thousands of processors for problems with smooth and nonsmooth solutions

partitioning group correction cholesky techniques for large scale sparse unconstrained optimization this paper studies a successive partitioning group correction cholesky algorithm and its modified algorithm for solving large scale sparse unconstrained optimization problems these methods employ an initial cholesky factorization of their approximation hessians and then correct the partitioning group s of their diagonal factor and lower triangular factor directly and successively at each step iterations are generated using forward and backward substitution employing the update factorizations a self correcting property a q superlinear convergence result and an r convergence rate estimate show that the two methods both have good local convergence properties the numerical results show that the two methods especially the modified algorithm may be competitive with some current used algorithms

some computational aspects of a group theoretic finite element approach to the buckling and postbuckling analyses of plates and shells of revolution abstract progress in the application of group theoretic methods to the nonlinear bifurcation analysis of symmetric structures has provided new hope in understanding the global postbuckling behavior of symmetric thin shell structures however what is still lacking in the computational mechanics community is a general purpose fe code with a group theoretic capability designed to take optimal advantage of existing symmetry this paper addresses some of the important issues associated with developing such a code the long term goal being to provide structural analysts with new numerical tools for doing careful global analyses of imperfection sensitive structures which may eventually lead to more accurate predictions of the maximum load carrying capacity of thin imperfection sensitive shell structures

a hybrid algorithm for solving sparse nonlinear systems of equations this paper presents a hybrid algorithm for solving sparse nonlinear systems of equations the algorithm is based on dividing the columns of the jacobian into two parts and using different algorithms on each part the hybrid algorithm incorporates advantages of both component algorithms by exploiting the special structure of the jacobian to obtain a good approximation to the jacobian using as little effort as possible a kantorovich type analysis and a locally q superlinear convergence result for this algorithm are given

the efficient computation of sparse jacobian matrices using automatic differentiation this paper is concerned with the efficient computation of sparse jacobian matrices of nonlinear vector maps using automatic differentiation ad specifically we propose the use of a graph coloring technique bicoloring to exploit the sparsity of the jacobian matrix j and thereby allow for the efficient determination of j using ad software we analyze both a direct scheme and a substitution process we discuss the results of numerical experiments indicating significant practical potential of this approach

exact block diagonalization of large eigenvalue problems for structures with symmetry we consider large eigenvalue problems for skeletal structures with symmetry we present an algorithm based upon a novel combination of group theoretic ideas and substructuring techniques that block diagonalizes such systems exactly and efficiently the procedure requires only the structural matrices of a repeating substructure together with the symmetry modes which are obtained from symmetry considerations alone we first present a simple paradigmatic example and then follow with several non trivial applications involving large lattice structures

mean field theory of echo state networks dynamical systems driven by strong external signals are ubiquitous in nature and engineering here we study echo state networks networks of a large number of randomly connected nodes which represent a simple model of a neural network and have important applications in machine learning we develop a mean field theory of echo state networks the dynamics of the network is captured by the evolution law similar to a logistic map for a single collective variable when the network is driven by many independent external signals this collective variable reaches a steady state but when the network is driven by a single external signal the collective variable is non stationary but can be characterized by its time averaged distribution the predictions of the mean field theory including the value of the largest lyapunov exponent are compared with the numerical integration of the equations of motion

additive schwarz based fully coupled implicit methods for resistive hall magnetohydrodynamic problems a parallel fully coupled nonlinearly implicit newton krylov schwarz algorithm is proposed for the numerical simulation of a magnetic reconnection problem described by a system of resistive hall magnetohydrodynamics equations in slab geometry a key component of the algorithm is a restricted additive schwarz preconditioner defined for problems with doubly periodic boundary conditions we show numerically that with such a preconditioned nonlinearly implicit method the time step size is no longer constrained by the cfl number or the convergence of the newton solver we report the parallel performance of the algorithm and software on machines with thousands of processors

the computation of symmetry breaking bifurcation points we consider symmetry breaking simple singular points of a parameter dependent nonlinear equation g lambda x 0 such points are intimately connected to pitchfork bifurcation points and we sh

recent developments in constrained optimization constrained optimization problems occur in many applications of science engineering social science and medicine and hence it is useful for practitioners in these areas to be informed about numerical methods for solving such problems however improvements in optimization typically do not spread quickly to the non specialist literature this paper contains a brief summary of two recent developments in the solution of nonlinearly constrained problems direct use of the lagrangian function and linearization of nonlinear constraints in order to highlight the new approaches we also discuss techniques used in older methods to illustrate the capabilities of modern optimization methods a brief description is given of the development of methods to solve large nonlinearly constrained problems that arise in the electrical power industry the general nonlinear programming problem may be stated in the following form

unexpected biases in the distribution of consecutive primes although the sequence of primes is very well distributed in the reduced residue classes formula see text the distribution of pairs of consecutive primes among the permissible q 2 pairs of reduced residue classes formula see text is surprisingly erratic this paper proposes a conjectural explanation for this phenomenon based on the hardy littlewood conjectures the conjectures are then compared with numerical data and the observed fit is very good

bifurcation hierarchy of symmetric structures abstract a group theoretic method for the analysis of bifurcation behavior of regular polygonal symmetric structures is described possible bifurcation paths and points of these structures are categorized in terms of dihedral and cyclic groups which express the symmetry in particular we offer a complete description of those double bifurcation points which occur due to group symmetry the type the number and the stability of bifurcation paths branching at these points are determined by deriving bifurcation equations the existence of a potential function plays a substantial role for the existence of bifurcation paths as a result of these all possible bifurcation process can be known a priori as a natural consequence of bifurcation hierarchy before actual numerical analysis an implementation of this method in numerical analysis shows its validity and usability

conopt a grg code for large sparse dynamic nonlinear optimization problems the paper presents conopt an optimization system for static and dynamic large scale nonlinearly constrained optimization problems the system is based on the grg algorithm all computations involving the jacobian of the constraints use sparse matrix algorithm from linear programming modified to take optimal advantage of the nonlinearity the paper presents the main features of the system especially the inversion routines and their data structures the dynamic setting of tolerances in newton s algorithm and the user features in the overall packaging computational experience with some medium to large models is presented

tensor methods for large sparse systems of nonlinear equations this paper introduces tensor methods for solving large sparse systems of nonlinear equations tensor methods for nonlinear equations were developed in the context of solving small to medium sized dense problems they base each iteration on a quadratic model of the nonlinear equations where the second order term is selected so that the model requires no more derivative or function information per iteration than standard linear model based methods and hardly more storage or arithmetic operations per iteration computational experiments on small to medium sized problems have shown tensor methods to be considerably more efficient than standard newton based methods with a particularly large advantage on singular problems this paper considers the extension of this approach to solve large sparse problems the key issue considered is how to make efficient use of sparsity in forming and solving the tensor model problem at each iteration accomplishing this turns out to require an entirely new way of solving the tensor model that successfully exploits the sparsity of the jacobian whether the jacobian is nonsingular or singular we develop such an approach and based upon it an efficient tensor method for solving large sparse systems of nonlinear equations test results indicate that this tensor method is significantly more efficient and robust than an efficient sparse newton based method in terms of iterations function evaluations and execution time 1998 the mathematical programming society inc published by elsevier science b v

on the convergence of partitioning group correction algorithms this paper studies a successive partitioning group correction algorithm and its some modified algorithms for solving large scale sparse unconstrained optimization problems the methods depend on a symmetric consistent partition of the columns of the hessian matrix a q superlinear convergence result and an r convergence rate estimate show that the methods have good local convergence properties the numerical results show that the methods especially the modified algorithms may be competitive with some current used algorithms

distributed memory parallel algorithms for distance 2 coloring and their application to derivative computation the distance 2 graph coloring problem aims at partitioning the vertex set of a graph into the fewest sets consisting of vertices pairwise at distance greater than two from each other its applications include derivative computation in numerical optimization and channel assignment in radio networks we present efficient distributed memory parallel heuristic algorithms for this nphard problem as well as for two related problems used in the computation of jacobians and hessians parallel speedup is achieved through graph partitioning speculative iterative coloring and a bsplike organization of parallel computation results from experiments conducted on a pc cluster employing up to 96 processors and using large size real world as well as synthetically generated test graphs show that the algorithms are scalable in terms of quality of solution the algorithms perform remarkably well the number of colors used by the parallel algorithms was observed to be very close to the number used by the sequential counterparts which in turn are quite often near optimal moreover the experimental results show that the parallel distance 2 coloring algorithm compares favorably with the alternative approach of solving the distance 2 coloring problem on a graph g by first constructing the square graph g2 and then applying a parallel distance 1 coloring algorithm on g2 implementations of the algorithms are made available via the zoltan load balancing library

distributed memory parallel algorithms for distance 2 coloring and related problems in derivative computation the distance 2 graph coloring problem aims at partitioning the vertex set of a graph into the fewest sets consisting of vertices pairwise at distance greater than 2 from each other its applications include derivative computation in numerical optimization and channel assignment in radio networks we present efficient distributed memory parallel heuristic algorithms for this np hard problem as well as for two related problems used in the computation of jacobians and hessians parallel speedup is achieved through graph partitioning speculative iterative coloring and a bulk synchronous parallel like organization of parallel computation results from experiments conducted on a pc cluster employing up to 96 processors and using large size real world as well as synthetically generated test graphs show that the algorithms are scalable in terms of quality of solution the algorithms perform remarkably well the numbers of colors used by the parallel algorithms are observed to be very close to the numbers used by their sequential counterparts which in turn are quite often near optimal moreover the experimental results show that the parallel distance 2 coloring algorithm compares favorably with the alternative approach of solving the distance 2 coloring problem on a graph mathcal g by first constructing the square graph mathcal g 2 and then applying a parallel distance 1 coloring algorithm on mathcal g 2 implementations of the algorithms are made available via the zoltan toolkit

new acyclic and star coloring algorithms with application to computing hessians acyclic and star coloring problems are specialized vertex coloring problems that arise in the efficient computation of hessians using automatic differentiation or finite differencing when both sparsity and symmetry are exploited we present an algorithmic paradigm for finding heuristic solutions for these two np hard problems the underlying common technique is the exploitation of the structure of two colored induced subgraphs for a graph g on n vertices and m edges the time complexity of our star coloring algorithm is o n overline d 2 where overline d k a generalization of vertex degree denotes the average number of distinct paths of length at most k edges starting at a vertex in g the time complexity of our acyclic coloring algorithm is larger by a multiplicative factor involving the inverse of ackermann s function the space complexity of both algorithms is o m to the best of our knowledge our work is the first practical algorithm for the acyclic coloring problem for the star coloring problem our algorithm uses fewer colors and is considerably faster than a previously known o n overline d 3 time algorithm computational results from experiments on various large size test graphs demonstrate that the algorithms are fast and produce highly effective solutions the use of these algorithms in hessian computation is expected to reduce overall runtime drastically

a direct matrix method for computing analytical jacobians of discretized nonlinear integro differential equations in this article we present a simple direct matrix method for analytically computing the jacobian of nonlinear algebraic equations that arise from the discretization of nonlinear integro differential equations the method is based on a formulation of the discretized equations in vector form using only matrix vector products and component wise operations by applying simple matrix based differentiation rules the matrix form of the analytical jacobian can be calculated with little more difficulty than required to compute derivatives in single variable calculus after describing the direct matrix method we present numerical experiments demonstrating the computational performance of the method discuss its connection to the newton kantorovich method and apply it to illustrative 1d and 2d example problems from electrochemical transport

estimation of sparse hessian matrices and graph coloring problems large scale optimization problems often require an approximation to the hessian matrix if the hessian matrix is sparse then estimation by differences of gradients is attractive because the number of required differences is usually small compared to the dimension of the problem the problem of estimating hessian matrices by diferences can be phrased as follows given the sparsity structure of a symmetric matrix a obtain vectors d 1 d 2 ldots d p such that ad 1 ad 2 ldots ad p determine a uniquely with p as small as possible we approach this problem from a graph theoretic point of view and show that both direct and indirect approaches to this problem have a natural graph coloring interpretation the complexity of the problem is analyzed and efficient practical heuristic procedures are developed numerical results illustrate the differences between the various approaches

improving the performance guarantee for approximate graph coloring the performance guarantee of a graph coloring algorithm is the worst case ratio between the number of colors it uses on the input graph and the chromauc number of this graph the previous best known polynomial time algorithm had a performance guarantee o n logn for graphs on n vertices this result stood unchallenged for eight years this paper presents an efficient algorithm with performance guarantee of o n loglog n 2 logn 2

acyclic and star coloring of p4 reducible and p4 sparse graphs an acyclic coloring of a graph g is a proper vertex coloring such that g contains no bicolored cycles the more restricted notion of star coloring of g is an acyclic coloring in which each path of length 3 is not bicolored in this paper we mainly study on the acyclic and star coloring of p4 reducible and p4 sparse graphs moreover we list polynomial time algorithms for giving an optimal acyclic or star coloring of a p4 reducible or p4 sparse graph

detecting jacobian sparsity patterns by bayesian probing in this paper we describe an automatic procedure for successively reducing the set of possible nonzeros in a jacobian matrix until eventually the exact sparsity pattern is obtained the dependence information needed in this probing process consist of boolean jacobian vector products and possibly also vector jacobian products which can be evaluated exactly by automatic differentiation or approximated by divided differences the latter approach yields correct sparsity patterns provided there is no exact cancellation at the current argument starting from a user specified or by default initialized probability distribution the procedure suggests a sequence of probing vectors the resulting information is then used to update the probabilities that certain elements are nonzero according to bayes law the proposed probing procedure is found to require only o logn probing vectors on randomly generated matrices of dimension n with a fixed number of nonzeros per row or column this result has been proven for block banded matrices and for general sparsity pattern finite termination of the probing procedure can be guaranteed

what color is your jacobian graph coloring for computing derivatives graph coloring has been employed since the 1980s to efficiently compute sparse jacobian and hessian matrices using either finite differences or automatic differentiation several coloring problems occur in this context depending on whether the matrix is a jacobian or a hessian and on the specifics of the computational techniques employed we consider eight variant vertex coloring problems here this article begins with a gentle introduction to the problem of computing a sparse jacobian followed by an overview of the historical development of the research area then we present a unifying framework for the graph models of the variant matrix estimation problems the framework is based upon the viewpoint that a partition of a matrix into structurally orthogonal groups of columns corresponds to distance 2 coloring an appropriate graph representation the unified framework helps integrate earlier work and leads to fresh insights enables the design of more efficient algorithms for many problems leads to new algorithms for others and eases the task of building graph models for new problems we report computational results on two of the coloring problems to support our claims most of the methods for these problems treat a column or a row of a matrix as an atomic entity and partition the columns or rows or both a brief review of methods that do not fit these criteria is provided we also discuss results in discrete mathematics and theoretical computer science that intersect with the topics considered here

the evolution of the minimum degree ordering algorithm over the past fifteen years the implementation of the minimum degree algorithm has received much study and many important enhancements have been made to it this paper describes these various enhancements their historical development and some experiments showing how very effective they are in improving the execution time of the algorithm a shortcoming is also presented that exists in all of the widely used implementations of the algorithm namely that the quality of the ordering provided by the implementations is surprisingly sensitive to the initial ordering for example changing the input ordering can lead to an increase or decrease of as much as a factor of three in the cost of the subsequent numerical factorization this sensitivity is caused by the lack of an effective tie breaking strategy and the authors experiments illustrate the importance of developing such a strategy