on positive semidefinite modification schemes for incomplete cholesky factorization incomplete cholesky factorizations have long been important as preconditioners for use in solving large scale symmetric positive definite linear systems in this paper we focus on the relationship between two important positive semidefinite modification schemes that were introduced to avoid factorization breakdown namely the approach of jennings and malik and that of tismenetsky we present a novel view of the relationship between the two schemes and implement them in combination with a limited memory approach we explore their effectiveness using extensive numerical experiments involving a large set of test problems arising from a wide range of practical applications

experimental study of ilu preconditioners for indefinite matrices incomplete lu factorization preconditioners have been surprisingly successful for many cases of general nonsymmetric and indefinite matrices however their failure rate is still too high for them to be useful as black box library software for general matrices besides fatal breakdowns due to zero pivots the major causes of failure are inaccuracy and instability of the triangular solves when there are small pivots both these problems can occur but these problems can also occur without small pivots through examples from actual problems this paper shows how these problems evince themselves how these problems can be detected and how these problems can sometimes be circumvented through pivoting reordering scaling perturbing diagonal elements and preserving symmetric structure the goal of this paper is to gain a better practical understanding of ilu preconditioners and help improve their reliability

preconditioned conjugate gradient methods for three dimensional linear elasticity finite element modelling of three dimensional elasticity problems give rise to large sparse matrices various preconditioning methods are developed for use in preconditioned conjugate gradient iterative solution techniques incomplete factorizations based on levels of fill drop tolerance and a two level hierarchical basis are developed various techniques for ensuring that the incomplete factors have positive pivots are presented computational tests are carried out for problems generated using unstructured tetrahedral meshes quadratic basis functions are used the performance of the iterative methods is compared to a standard direct sparse matrix solver problems with up to 70 000 degrees of freedom and small 1 element aspect ratio are considered

a new matrix free algorithm for the large scale trust region subproblem we present a new method for the large scale trust region subproblem the method is matrix free in the sense that only matrix vector products are required we recast the trust region subproblem as a parameterized eigenvalue problem and compute an optimal value for the parameter we then find the solution of the trust region subproblem from the eigenvectors associated with two of the smallest eigenvalues of the parameterized eigenvalue problem corresponding to the optimal parameter the new algorithm uses a different interpolating scheme than existing methods and introduces a unified iteration that naturally includes the so called hard case we show that the new iteration is well defined and convergent at a superlinear rate we present computational results to illustrate convergence properties and robustness of the method

analysis of parallel incomplete point factorizations abstract incomplete point factorizations are analyzed regarding their parallelism and the norm of the error matrix and the relation between the two

on the performance of linear programming heuristics applied on a quadratic transformation in the classification problem abstract we have applied lp heuristics to a quadratic transformation of the observation data in the classification problem four real world research data sets in the literature have been used to test the method these data sets include continuous integer and binary attributes for the observations to be classified in all four research data sets the proposed quadratic transformation method outperformed quadratic discriminant analysis

incomplete cholesky factorizations with limited memory we propose an incomplete cholesky factorization for the solution of large scale trust region subproblems and positive definite systems of linear equations this factorization depends on a parameter p that specifies the amount of additional memory in multiples of n the dimension of the problem that is available there is no need to specify a drop tolerance our numerical results show that the number of conjugate gradient iterations and the computing time are reduced dramatically for small values of p we also show that in contrast with drop tolerance strategies the new approach is more stable in terms of number of iterations and memory requirements

scalable algorithms for three field mixed finite element coupled poromechanics abstract we introduce a class of block preconditioners for accelerating the iterative solution of coupled poromechanics equations based on a three field formulation the use of a displacement velocity pressure mixed finite element method combined with a first order backward difference formula for the approximation of time derivatives produces a sequence of linear systems with a 3 3 unsymmetric and indefinite block matrix the preconditioners are obtained by approximating the two level schur complement with the aid of physically based arguments that can be also generalized in a purely algebraic approach a theoretical and experimental analysis is presented that provides evidence of the robustness efficiency and scalability of the proposed algorithm the performance is also assessed for a real world challenging consolidation experiment of a shallow formation

multi level incomplete factorizations for the iterative solution of non linear finite element problems several engineering applications give rise quite naturally to linearized fe systems of equations possessing a multi level structure an example is provided by geomechanical models of layered and faulted geological formations for such problems the use of a multi level incomplete factorization mif as a preconditioner for krylov subspace methods can prove a robust and efficient solution accelerator allowing for a fine tuning of the fill in degree with a significant improvement in both the solver performance and the memory consumption the present paper develops two novel mif variants for the solution of multi level symmetric positive definite systems two correction algorithms are proposed with the aim of preserving the positive definiteness of the preconditioner thus avoiding possible breakdowns of the preconditioned conjugate gradient solver the mif variants are experimented with in the solution of both a single system and a long term quasi static simulation dealing with a multi level geomechanical application the numerical results show that mif typically outperforms by up to a factor 3 a more traditional algebraic preconditioner such as an incomplete cholesky factorization with partial fill in the advantage is emphasized in a long term simulation where the fine fill in tuning allowed for by mif yields a significant improvement for the computer memory requirement as well copyright 2009 john wiley sons ltd

an h likelihood method for spatial mixed linear models based on intrinsic auto regressions summary we consider sparse spatial mixed linear models particularly those described by besag and higdon and develop an h likelihood method for their statistical inference the method proposed allows for singular precision matrices as it produces estimates that coincide with those from the residual maximum likelihood based on appropriate differencing of the data and has a novel approach to estimating precision parameters by a gamma linear model furthermore we generalize the h likelihood method to include continuum spatial variations by making explicit use of scaling limit connections between gaussian intrinsic markov random fields on regular arrays and the de wijs process keeping various applications of spatial mixed linear models in mind we devise a novel sparse conjugate gradient algorithm that allows us to achieve fast matrix free statistical computations we provide two applications the first is an extensive analysis of an agricultural variety trial that brings forward various new aspects of nearest neighbour adjustment such as effects on statistical analyses to changes of scale and use of implicit continuum spatial formulation the second application concerns an analysis of a large cotton field which gives a focus to matrix free computations the paper closes with some further considerations such as applications to irregularly spaced data use of the parametric bootstrap and some generalizations to the gaussian matern mixed effect models

new lp based heuristics for the classification problem abstract three heuristic procedures for the two group discriminant problem are developed that minimize the total expected cost of misclassification in the training sample the approach is shown through a systematic experiment to generate near optimal solutions and to be efficient in terms of computational effort when compared to existing research

addressing the multigroup discriminant problem using multivariate statistics and mathematical programming in the paper we compare the performance of seven important multivariate and mathematical programming methods in the multigroup classification problem with simulated data our approach extends previous results from two to multiple groups and recognizes simultaneously different types of classification errors in a multivariate analysis of variance manova framework

efficient preconditioner updates for shifted linear systems we present a technique for building effective and low cost preconditioners for sequences of shifted linear systems a alpha i x alpha b where a is symmetric positive definite and alpha 0 this technique updates a preconditioner for a available in the form of an ldl t factorization by modifying only the nonzero entries of the l factor in such a way that the resulting preconditioner mimics the diagonal of the shifted matrix and reproduces its overall behavior this approach is supported by a theoretical analysis as well as by numerical experiments showing that it works efficiently for a broad range of values of alpha

on signed incomplete cholesky factorization preconditioners for saddle point systems limited memory incomplete cholesky factorizations can provide robust preconditioners for sparse symmetric positive definite linear systems in this paper the focus is on extending the approach to sparse symmetric indefinite systems in saddle point form a limited memory signed incomplete cholesky factorization of the form ldl t is proposed where the diagonal matrix d has entries pm 1 the main advantage of this approach is its simplicity as it avoids the use of numerical pivoting instead a global shift strategy involving two shifts one for the 1 1 block and one for the 2 2 block of the saddle point matrix is used to prevent breakdown and to improve performance the matrix is optionally prescaled and preordered using a standard sparse matrix ordering scheme that is then postprocessed to give a constrained ordering that reduces the likelihood of breakdown and need for shifts the use of intermediate memory memory used in the construction of the incomplete factorization but subsequently d

a generalized block fsai preconditioner for nonsymmetric linear systems the efficient solution to nonsymmetric linear systems is still an open issue especially on parallel computers in this paper we generalize to the unsymmetric case the block factorized sparse approximate inverse block fsai preconditioner which has already proved very effective on symmetric positive definite spd problems block fsai is a hybrid approach combining an inner preconditioner with the aim of transforming the system matrix structure to block diagonal with an outer one a block diagonal incomplete or exact factorization intended to improve the conditioning of each block the proposed algorithm is experimented with in a number of large size matrices showing both a good robustness and scalability

a scalable parallel algorithm for incomplete factor preconditioning we describe a parallel algorithm for computing incomplete factor ilu preconditioners the algorithm attains a high degree of parallelism through graph partitioning and a two level ordering strategy both the subdomains and the nodes within each subdomain are ordered to preserve concurrency we show through an algorithmic analysis and through computational results that this algorithm is scalable experimental results include timings on three parallel platforms for problems with up to 20 million unknowns running on up to 216 processors the resulting preconditioned krylov solvers have the desirable property that the number of iterations required for convergence is insensitive to the number of processors

branch and cut for the maximum feasible subsystem problem this paper presents a branch and cut algorithm for the np hard maximum feasible subsystem problem for a given infeasible linear inequality system determine a feasible subsystem containing as many inequalities as possible the complementary problem where one has to remove as few inequalities as possible in order to make the system feasible can be formulated as a set covering problem the rows of this formulation correspond to irreducible infeasible subsystems which can be exponentially many it turns out that the main issue of a branch and cut algorithm for the maximum feasible subsystem problem max fs is to efficiently find such infeasible subsystems we present three heuristics for the corresponding np hard separation problem and discuss cutting planes from the literature such as set covering cuts of balas and ng gomory cuts and 0 frac 1 2 cuts furthermore we compare a heuristic of chinneck and a simple greedy algorithm the main contribution of this paper is an extensive computational study on a variety of instances arising in a number of applications

a subspace minimization method for the trust region step we consider methods for large scale unconstrained minimization based on finding an approximate minimizer of a quadratic function subject to a two norm trust region constraint the steihaug toint method uses the conjugate gradient method to minimize the quadratic over a sequence of expanding subspaces until the iterates either converge to an interior point or cross the constraint boundary however if the conjugate gradient method is used with a preconditioner the steihaug toint method requires that the trust region norm be defined in terms of the preconditioning matrix if a different preconditioner is used for each subproblem the shape of the trust region can change substantially from one subproblem to the next which invalidates many of the assumptions on which standard methods for adjusting the trust region radius are based in this paper we propose a method that allows the trust region norm to be defined independently of the preconditioner the method solves the inequality constrained trust region subproblem over a sequence of evolving low dimensional subspaces each subspace includes an accelerator direction defined by a regularized newton method for satisfying the optimality conditions of a primal dual interior method a crucial property of this direction is that it can be computed by applying the preconditioned conjugate gradient method to a positive definite system in both the primal and dual variables of the trust region subproblem numerical experiments on problems from the cuter test collection indicate that the method can require significantly fewer function evaluations than other methods in addition experiments with general purpose preconditioners show that it is possible to significantly reduce the number of matrix vector products relative to those required without preconditioning

parabolic evolution equations and nonlinear boundary conditions on etudie le probleme aux valeurs limites et initiales u 7b a t u f t u 7b b t u g t u 0 t t u 0 u 0 qui peut etre considere comme une contrepartie abstraite a des problemes paraboliques semilineaires

a comparative study of the effect of the position of outliers on classical and nontraditional approaches to the two group classification problem abstract popularity of nontraditional approaches to the statistical classification problem has resulted from the potential of these techniques to outperform the standard parametric procedures under conditions when nonnormality is present thus proponents of these nontraditional models have recommended these models when outliers are in the data however research showing that these nontraditional models performances can vary widely depending on where the outlier data are located has not been fully illustrated the research in this paper demonstrates how the mathematical programming approaches and the nearest neighbor discriminant models can be affected by the position of contaminated normal data and that each of the models studied in this paper may not be robust to all types of outliers in the data the results of this paper are also important because the study compares two recently proposed mathematical programming models as well as two versions of the nearest neighbor model with the standard classical parametric models this combination of classification models does not appear to have been studied together under conditions of contaminated normal data in which numerous positions of the outliers are considered

a classification model based on goal programming with non standard preference functions with application to the prediction of cinema going behaviour this paper presents a goal programming model that allows for the flexible handling of the two group classification problem the goal programming model is based around the concepts of non standard preference functions and penalty function modelling an extension to a generalised distance metric case is given the inclusion of multiple levels of classification based upon different levels of certainty is incorporated into the model the model is tested on a real life data set pertaining to cinema going attendance and conclusions are drawn both in the context of the methodology and of the application

a mathematical programming approach to clusterwise regression model and its extensions abstract the clusterwise regression model is used to perform cluster analysis within a regression framework while the traditional regression model assumes the regression coefficient to be identical for all subjects in the sample the clusterwise regression model allows to vary with subjects of different clusters since the cluster membership is unknown the estimation of the clusterwise regression is a tough combinatorial optimization problem in this research we propose a generalized clusterwise regression model which is formulated as a mathematical programming mp problem a nonlinear programming procedure with linear constraints is proposed to solve the combinatorial problem and to estimate the cluster membership and simultaneously moreover by integrating the cluster analysis with the discriminant analysis a clusterwise discriminant model is developed to incorporate parameter heterogeneity into the traditional discriminant analysis the cluster membership and discriminant parameters are estimated simultaneously by another nonlinear programming model

preconditioning regularized least squares problems arising from high resolution image reconstruction from low resolution frames in this paper we study the problem of reconstructing a high resolution image from multiple undersampled shifted degraded frames with subpixel displacement errors from multisensors preconditioned conjugate gradient methods with cosine transform based preconditioners and incomplete factorization based preconditioners are applied to solve this image reconstruction problem numerical examples are given to demonstrate the efficiency of these preconditioners we find that cosine transform based preconditioners are effective when the number of shifted low resolution frames are large but are less effective when the number is small however incomplete factorization based preconditioners work quite well independent of the number of shifted low resolution frames

a hybrid genetic fuzzy neural network algorithm designed for classification problems involving several groups we propose a multigroup classification algorithm based on a hybrid genetic fuzzy neural net gfnn framework recent results on evolutionary computation and fuzzy neural network methodology are combined to effectively adapt the membership functions of the fuzzifier and the defuzzifier to the data set separate membership functions are defined for each dimension in the fuzzifier and for each fuzzy output group in the defuzzifier the signal inherent in the fuzzifier is aggregated by a suitable t norm and transmitted to the defuzzifier the defuzzifier aggregates the response i e the predicted group membership by a suitable conorm if misclassifications occur during training the membership functions of both the fuzzifier and the defuzzifier are adapted by a systematic robust procedure the algorithm is successfully tested with real economic data in total the gfnn performs as good as the best of the competing methods in our test the results suggest economically meaningful interpretations

on a finite difference scheme for blow up solutions for the chipot weissler equation in this paper we are interested in the numerical analysis of blow up for the chipot weissler equation u t u u p 1 u u q with dirichlet boundary conditions in bounded domain when p 1 and 1 q 2 p p 1 to approximate the blow up solution we construct a finite difference scheme and we prove that the numerical solution satisfies the same properties of the exact one and blows up in finite time

constrained regression median for measuring possible salary discrimination abstract this study proposes a new analytical method for empirically examining possible salary discrimination against a protected e g female or minority group the proposed approach is referred to as constrained regression median crm the method can be used for regressing salary with job qualifications as dependent variables the result of crm is transformed into a 2 2 contingency table and the degree of salary discrimination is determined by the result of the contingency table the approach can be easily explained to and understood by lawyers judges and other individuals involved in a discrimination case the crm method is particularly useful and practical when a data set maintains the problems of outlier and or multicollinearity between job qualifications and gender or race

minimizing deviations from the group mean a new linear programming approach for the two group classification problem abstract this paper proposes a new linear programming approach to solve the two group classification problem in discriminant analysis this new approach is based on an idea from cluster analysis that objects within the same group should be more similar than objects between groups consequently it is desirable for the classification score of an object to be nearer to its mean classification score but further from the mean classification score of the other group this objective is accomplished by minimizing the total deviation of the classification scores of the objects from their group mean scores in a linear programming approach when applied to an actual managerial problem and simulated data the proposed linear programming approach performs well both in groups separation and group membership predictions of new objects moreover this new approach has an advantage of obtaining more stable classification function across different samples than most of the existing linear programming approaches

a matrix free approach to build band preconditioners for large scale bound constrained optimization abstract we propose a procedure for building symmetric positive definite band preconditioners for large scale symmetric possibly indefinite linear systems when the coefficient matrix is not explicitly available but matrix vector products involving it can be computed we focus on linear systems arising in newton type iterations within matrix free versions of projected methods for bound constrained nonlinear optimization in this case the structure and the size of the matrix may significantly change in subsequent iterations and preconditioner updating algorithms that exploit information from previous steps cannot be easily applied our procedure is based on a recursive approach that incrementally improves the quality of the preconditioner while requiring a modest number of matrix vector products a strategy for dynamically choosing the bandwidth of the preconditioners is also presented numerical results are provided showing the performance of our preconditioning technique within a trust region newton method for bound constrained optimization

mixed integer programming approach of extended dea discriminant analysis abstract several unique features of data envelopment analysis dea are incorporated into discriminant analysis da such a combined use of the two approaches is referred to as dea da eur j operat res 1999 115 564 sueyoshi eur j operat res 2001 131 324 has recently extended dea da in a manner that it can deal with a negative value in data the new version is referred to as extended dea da an important feature of extended dea da is that it can estimate weights of a linear discriminant function by minimizing the total deviation of misclassified observations as an extension of his research this research proposes a new type of mixed integer programming formulations that provide these weight estimates by minimizing the total number of misclassified observations

insertion of triangulated surfaces into a meccano tetrahedral discretization by means of mesh refinement and optimization procedures this is the peer reviewed version of the following article ruiz girones e oliver a socorro g cascon j escobar j m montenegro r sarrate j insertion of triangulated surfaces into a meccano tetrahedral discretization by means of mesh refinement and optimization procedures international journal for numerical methods in engineering 2018 which has been published in final form at http onlinelibrary wiley com doi 10 1002 nme 5706 pdf this article may be used for non commercial purposes in accordance with wiley terms and conditions for self archiving

support vector machines with the ramp loss and the hard margin loss in the interest of deriving classifiers that are robust to outlier observations we present integer programming formulations of vapnik s support vector machine svm with the ramp loss and hard margin loss the ramp loss allows a maximum error of 2 for each training observation while the hard margin loss calculates error by counting the number of training observations that are in the margin or misclassified outside of the margin svm with these loss functions is shown to be a consistent estimator when used with certain kernel functions in computational studies with simulated and real world data svm with the robust loss functions ignores outlier observations effectively providing an advantage over svm with the traditional hinge loss when using the linear kernel despite the fact that training svm with the robust loss functions requires the solution of a quadratic mixed integer program qmip and is np hard while traditional svm requires only the solution of a continuous quadratic program qp we are able to find good solutions and prove optimality for instances with up to 500 observations solution methods are presented for the new formulations that improve computational performance over industry standard integer programming solvers alone

a new active set algorithm for box constrained optimization an active set algorithm asa for box constrained optimization is developed the algorithm consists of a nonmonotone gradient projection step an unconstrained optimization step and a set of rules for branching between the two steps global convergence to a stationary point is established for a nondegenerate stationary point the algorithm eventually reduces to unconstrained optimization without restarts similarly for a degenerate stationary point where the strong second order sufficient optimality condition holds the algorithm eventually reduces to unconstrained optimization without restarts a specific implementation of the asa is given which exploits the recently developed cyclic barzilai borwein cbb algorithm for the gradient projection step and the recently developed conjugate gradient algorithm cg descent for unconstrained optimization numerical experiments are presented using box constrained problems in the cuter and minpack 2 test problem libraries

evaluating the maximize minimum distance formulation of the linear discriminant problem abstract the maximize minimum distance mmd linear programming model for the two group discriminant problem has been noted to produce occasionally a trivial identically zero discriminant function one which classifies all observations into a single category in tests against other methods both parametric and nonparametric mmd has fared poorly in this paper we attribute the propensity of the mmd model to produce trivial solutions to a specific aspect of its formulation this same facet may also cause unnessarily high misclassification rates even when a nontrivial function is found we note a simple revision of a model which ensures an acceptable solution in those instancesin which the calibration samples can be classified with 100 accuracy by a single function this rises the question of whether the inferior performance of mmd in previous studies was due to inherent limitations in mmd or to the particular formulation used

a max plus approach to incomplete cholesky factorization preconditioners we present a new method for constructing incomplete cholesky factorization preconditioners for use in solving large sparse symmetric positive definite linear systems this method uses max plus algebra to predict the positions of the largest entries in the cholesky factor and then uses these positions as the sparsity pattern for the preconditioner our method builds on the max plus incomplete lu factorization preconditioner recently proposed in j hook and f tisseur incomplete lu preconditioner based on max plus approximation of lu factorization mims eprint 2016 47 manchester 2016 but applied to symmetric positive definite matrices which comprise an important special case for the method and its application a attractive feature of our approach is that the sparsity pattern of each column of the preconditioner can be computed in parallel numerical comparisons are made with other incomplete cholesky factorization preconditioners using problems from a range of practical applications we demonstrate that the new preconditioner can outperform traditional level based preconditioners and offer a parallel alternative to a serial limited memory based approach

simple but powerful goal programming models for discriminant problems abstract conventional statistical analysis includes the capacity to systematically assign individuals to groups we suggest alternative assignment procedures utilizing a set of interrelated goal programming formulations we further demonstrate via simple illustration the potential of these procedures to play a significant part in addressing the discriminant problem and indicate fundamental ideas that lay the foundation for other more sophisticated approaches

computing modified newton directions using a partial cholesky factorization the effectiveness of newton s method for finding an unconstrained minimizer of a strictly convex twice continuously differentiable function has prompted the proposal of various modified newton methods for the nonconvex case linesearch modified newton methods utilize a linear combination of a descent direction and a direction of negative curvature if these directions are sufficient in a certain sense and a suitable linesearch is used the resulting method will generate limit points that satisfy the second order necessary conditions for optimality the authors propose an efficient method for computing a descent direction and a direction of negative curvature that is based on a partial cholesky factorization of the hessian this factorization not only gives theoretically satisfactory directions but also requires only a partial pivoting strategy i e the equivalent of only two rows of the schur complement needs to be examined at each step

a sparse approximate inverse preconditioner for the conjugate gradient method a method for computing a sparse incomplete factorization of the inverse of a symmetric positive definite matrix a is developed and the resulting factorized sparse approximate inverse is used as an explicit preconditioner for conjugate gradient calculations it is proved that in exact arithmetic the preconditioner is well defined if a is an h matrix the results of numerical experiments are presented

second order mathematical programming formulations for discriminant analysis this paper introduces a nonparametric formulation based on mathematical programming mp for solving the classification problem in discriminant analysis which differs from previously proposed mp based models in that even though the final discriminant function is linear in terms of the parameters to be estimated the formulation is quadratic in terms of the predictor attribute variables including second order i e quadratic and cross product terms of the attribute variables in the model is similar in concept to the usual treatment of multiple predictor variables in statistical methods such as fisher s linear discriminant analysis and allows an analysis of how including nonlinear terms and interaction effects affect the predictive ability of the estimated classification function using simulation experiments involving data conditions for which nonlinear classifiers are appropriate the classificatory performance of this class of second order mp models is compared with that of existing statistical linear and quadratic and first order mp based formulations the results of these experiments show that the proposed formulation appears to be a very attractive alternative to previously introduced linear and quadratic statistical and linear mp based classification methods

preconditioning techniques for large linear systems a survey this article surveys preconditioning techniques for the iterative solution of large linear systems with a focus on algebraic methods suitable for general sparse matrices covered topics include progress in incomplete factorization methods sparse approximate inverses reorderings parallelization issues and block and multilevel extensions some of the challenges ahead are also discussed an extensive bibliography completes the paper

multigroup discriminant analysis using linear programming in this paper we introduce a nonparametric linear programming formulation for the general multigroup classification problem previous research using linear programming formulations has either been limited to the two group case or required complicated constraints and many zero one variables we develop general properties of our multigroup formulation and illustrate its use with several small example problems and previously published real data sets a comparative analysis on the real data sets shows that our formulation may offer an interesting robust alternative to parametric statistical formulations for the multigroup discriminant problem

minimization of a large scale quadratic function subject to a spherical constraint an important problem in linear algebra and optimization is the trust region subproblem minimize a quadratic function subject to an ellipsoidal or spherical constraint this basic problem has several important large scale applications including seismic inversion and forcing convergence in optimization methods existing methods to solve the trust region subproblem require matrix factorizations which are not feasible in the large scale setting this paper presents an algorithm for solving the large scale trust region subproblem that requires a fixed size limited storage proportional to the order of the quadratic and that relies only on matrix vector products the algorithm recasts the trust region subproblem in terms of a parameterized eigenvalue problem and adjusts the parameter with a superlinearly convergent iteration to find the optimal solution from the eigenvector of the parameterized problem only the smallest eigenvalue and corresponding eigenvector of the parameterized problem needs to be computed the implicitly restarted lanczos method is well suited to this subproblem

on detecting nonlinear patterns in discriminant problems we propose a two stage model for detecting nonlinear patterns in discriminant problems and for solving the problem the model deploys a linear programming based discriminator lpbd in stage one for treating linear patterns and a probabilistic neural network pnn in stage two for handling nonlinear patterns the lpbd in stage one divides the decision space into a clear zone where observations are almost linearly separable and a gray zone where nonlinear patterns are more likely to occur the pnn in stage two analyzes the gray zone and determines whether a significant nonlinear patterns exist in the observations our goal is to avoid using a nonlinear model unless the pnn strongly suggests so to maintain good interpretability and avoid overfitting our computational study demonstrates the effectiveness of the two stage model in both classification accuracy and computational efficiency

a matrix free preconditioner for sparse symmetric positive definite systems and least squares problems we analyze and discuss matrix free and limited memory preconditioners for sparse symmetric positive definite systems and normal equations of large and sparse least squares problems the preconditioners are based on a partial cholesky factorization and can be coupled with a deflation strategy the construction of the preconditioners requires only matrix vector products is breakdown free and does not need to form the matrix the memory requirements of the preconditioners are defined in advance and they do not depend on the number of nonzero entries in the matrix when eigenvalue deflation is used the preconditioners turn out to be suitable for solving sequences of slowly changing systems or linear systems with different right hand sides numerical results are provided including the case of sequences arising in nonnegative linear least squares problems solved by interior point methods

a mixed integer programming approach to multi class data classification problem this paper presents a new data classification method based on mixed integer programming traditional approaches that are based on partitioning the data sets into two groups perform poorly for multi class data classification problems the proposed approach is based on the use of hyper boxes for defining boundaries of the classes that include all or some of the points in that set a mixed integer programming model is developed for representing existence of hyper boxes and their boundaries in addition the relationships among the discrete decisions in the model are represented using propositional logic and then converted to their equivalent integer constraints using boolean algebra the proposed approach for multi class data classification is illustrated on an example problem the efficiency of the proposed method is tested on the well known iris data set the computational results on the illustrative example and the iris data set show that the proposed method is accurate and efficient on multi class data classification problems

an active set truncated newton method for large scale bound constrained optimization an active set truncated newton method for large scale bound constrained optimization is proposed the active sets are guessed by an identification technique the search direction consists of two parts some of the components are simply defined the other components are determined by the truncated newton method the method based on a nonmonotone line search technique is shown to be globally convergent numerical experiments are presented using bound constrained problems in the cuter test problem library the numerical performance reveals that our method is effective and competitive with the famous algorithm tron

a comparison of standard and two stage mathematical programming discriminant analysis methods abstract classification models can be developed using standard or two stage mathematical programming mp discriminant analysis methods in standard mp discriminant analysis methods discriminant functions are generated by solving a single mp model in two stage mp methods observations that are difficult to classify are identified in the first stage with greater emphasis being given to these observations in the second stage mp model for discriminant function generation in this paper two two stage methods are described and compared with two standard mp models the model for minimisation of the sum of deviations and the model for maximisation of classification accuracy the performance of these mp discriminant analysis methods and fisher s linear discriminant analysis a parametric statistical technique is then evaluated on a published data set and on a number of simulated data sets

optimization approaches to supervised classification abstract the supervised classification problem one of the oldest and most recurrent problems in applied data analysis has always been analyzed from many different perspectives when the emphasis is placed on its overall goal of developing classification rules with minimal classification cost supervised classification can be understood as an optimization problem on the other hand when the focus is in modeling the uncertainty involved in the classification of future unknown entities it can be formulated as a statistical problem other perspectives that pay particular attention to pattern recognition and machine learning aspects of supervised classification have also a long history that has lead to influential insights and different methodologies in this review two approaches to supervised classification strongly related to optimization theory will be discussed and compared in particular we will review methodologies based on mathematical programming models that optimize observable criteria linked to the true objective of misclassification error or cost minimization and approaches derived from the minimization of known bounds on the true misclassification error the former approach is known as the mathematical programming approach to supervised classification while the latter is in the origin of the well known classification support vector machines throughout the review two group as well as general multi group problems will be considered and the review will conclude with a discussion of the most promising research directions in this area

the two group discriminant problem with equal group mean vectors an experimental evaluation of six linear nonlinear programming formulations abstract various parametric and nonparametric approaches to multiple discriminant analysis attempt to discriminate among or classify entities e g loan applicants customers employees businesses based on several of their distinguishing characteristics called discriminant variables statistical parametric procedures require that the mean vectors of discriminant variables for the populations of entities be different across groups this requirement may not always be met in practical settings this paper reports on a preliminary monte carlo simulation experiment which compares the performance of six 1 p norm distance models including two linear and four nonlinear formulations as well as two statistical procedures to address the discriminant problem under equal mean vectors the experimental data were generated from multivariate normal or nonnormal populations with equal or unequal dispersion matrices and with or without outliers the results indicate that when population mean vectors are equal the most significant characteristics which affects the performance of all of the methods is the similarity or dissimilarity of dispersion matrices the departure from normality and the presence of outliers and some of the interactions between these three factors are also statistically significant

newton s method for large bound constrained optimization problems we analyze a trust region version of newton s method for bound constrained problems our approach relies on the geometry of the feasible set not on the particular representation in terms of constraints the convergence theory holds for linearly constrained problems and yields global and superlinear convergence without assuming either strict complementarity or linear independence of the active constraints we also show that the convergence theory leads to an efficient implementation for large bound constrained problems

a semidefinite framework for trust region subproblems with applications to large scale minimization primal dual pairs of semidefinite programs provide a general framework for the theory and algorithms for the trust region subproblem trs this latter problem consists in minimizing a general quadratic function subject to a convex quadratic constraint and therefore it is a generalization of the minimum eigenvalue problem the importance of trs is due to the fact that it provides the step in trust region minimization algorithms the semidefinite framework is studied as an interesting instance of semidefinite programming as well as a tool for viewing known algorithms and deriving new algorithms for trs in particular a dual simplex type method is studied that solves trs as a parametric eigenvalue problem this method uses the lanczos algorithm for the smallest eigenvalue as a black box therefore the essential cost of the algorithm is the matrix vector multiplication and thus sparsity can be exploited a primal simplex type method provides steps for the so called hard case extensive numerical tests for large sparse problems are discussed these tests show that the cost of the algorithm is 1 n times the cost of finding a minimum eigenvalue using the lanczos algorithm where 0 n 1 is a fraction which decreases as the dimension increases

an algebraic multilevel multigraph algorithm we describe an algebraic multilevel multigraph algorithm many of the multilevel components are generalizations of algorithms originally applied to general sparse gaussian elimination indeed general sparse gaussian elimination with minimum degree ordering is a limiting case of our algorithm our goal is to develop a procedure which has the robustness and simplicity of use of sparse direct methods yet offers the opportunity to obtain the optimal or near optimal complexity typical of classical multigrid methods

linear goal programming in estimation of classification probability classification of objects into groups is a common practice in everyday life normally the scores of some pertinent criteria about the objects are available and form the most important factor in the classification decision discriminant analysis 4 logistic regression 1 3 and linear goal programming 2 5 are alternative techniques used to determine the criterion weights classification with these techniques is based upon the weighted sum of the criterion scores of each object the classification itself ignores more discriminating information about objects within groups for example dividing companies into bankrupt and non bankrupt groups ignores the fact that the non bankrupt group contains companies of varying qualities thus it is more appropriate to determine the group membership probabilities for each object instead of just the pure classification into one of the groups these probabilities can readily be used to discriminate objects within the same group

self similar subsolutions and blowup for nonlinear parabolic equations abstract for a wide class of nonlinear parabolic equations of the formut u f u u we prove the nonexistence of global solutions for large initial data we also estimate the maximal existence time to do so we use a method of comparison with suitable blowing up self similar subsolutions as a consequence we improve several known results onut u up on generalized burgers equations and on other semilinear equations this method can also apply to degenerate equations of porous medium type and provides a unified treatment for a large class of problems both semilinear and quasilinear

asymptotically self similar global solutions of a semilinear parabolic equation with a nonlinear gradient term we study the existence and the asymptotic behaviour of global solutions of the semilinear parabolic equation u 0 where a b q 1 p 1 for q 2p p 1 and 1 p 1 1 equivalently q n 2 n 1 we prove the existence of mild global solutions for small initial data with respect to some norm some of those solutions are asymptotically self similar

scalable nonlinear programming via exact differentiable penalty functions and trust region newton methods we present an approach for nonlinear programming based on the direct minimization of an exact differentiable penalty function using trust region newton techniques the approach provides desirable features required for scalability it can detect and exploit directions of negative curvature it is superlinearly convergent and it enables the scalable computation of the newton step through iterative linear algebra moreover it presents features that are desirable for parametric optimization problems that must be solved in a latency limited environment as is the case for model predictive control and mixed integer nonlinear programming these features are fast detection of activity efficient warm starting and progress on a primal dual merit function at every iteration we note that other algorithmic approaches fail to satisfy at least one of these features we derive general convergence results for our approach and demonstrate its behavior through numerical studies

examining the effect of second order terms in mathematical programming approaches to the classification problem abstract research on mathematical programming approaches to the classification problem has focused almost exclusively on linear discriminant functions with only first order terms while many of these first order models have displayed excellent classificatory performance when compared to fisher s linear discriminant method they cannot compete with smith s quadratic discriminant method on certain data sets in this paper we investigate the appropriateness of including second order terms in mathematical programming models various issues are addressed such as performance of models with small to moderate sample size need for crossproduct terms and loss of power by the mathematical programming models under conditions ideal for the parametric procedures a simulation study is conducted to assess the relative performance of first order and second order mathematical programming models to the parametric procedures the simulation study indicates that mathematical programming models using polynomial functions may be prone to overfitting on the training samples which in turn may cause rather poor fits on the validation samples the simulation study also indicates that inclusion of cross product terms may hurt a polynomial model s accuracy on the validation samples although omission of them means that the model is not invariant to nonsingular transformations of the data

mathematical programming based heuristics for improving lp generated classifiers for the multiclass supervised classification problem abstract mathematical programming is used as a nonparametric approach to supervised classification however mathematical programming formulations that minimize the number of misclassifications on the design dataset suffer from computational difficulties we present mathematical programming based heuristics for finding classifiers with a small number of misclassifications on the design dataset with multiple classes the basic idea is to improve an lp generated classifier with respect to the number of misclassifications on the design dataset the heuristics are evaluated computationally on both simulated and real world datasets

a comparison of a robust mixed integer approach to existing methods for establishing classification rules for the discriminant problem abstract a nonparametric mixed integer programming formulation to solve the classification problem in linear discriminant analysis is proposed the classification performance of this formulation is compared to the msd linear programming approach and two commonly used statistical methods fisher s linear discriminant function and the quadratic discriminant function using real data with highly nonnormal distributions the mixed integer formulation is shown to outperform any of the other three approaches to study the performance of the mixed integer formulation systematically a monte carlo simulation experiment is conducted sampling from several different distributions the results show that the mixed integer formulation outperforms the other three approaches on the training samples except when the variance covariances are heterogeneous in which case the quadratic function classifies better when holdout samples are used to evaluate the relative performance the mixed integer approach classifies well when the data are highly discrete and the variance covariances are homogeneous but does not fare as well under other data conditions therefore this study suggests that under certain conditions the mixed integer approach is an attractive alternative to establish classification methods

robust preconditioners for linear elasticity fem analyses summary this paper deals with two forms of preconditioner which can be easily used with a conjugate gradient solver to replace a direct solution subroutine in a traditional engineering nite element package they are tested in such a package final over a range of 2 d and 3 d elasticity problems from geotechnical engineering quadratic basis functions are used a number of modications to the basic incomplete choleski ic 0 factorization preconditioner are considered an algorithm to reduce positive o diagonal entries is shown in numerical experiments to ensure stability but at the expense of slow convergence an alternative algorithm of jennings and malik is more successful and a relaxation parameter is introduced which can make a further signicant improvement in performance while maintaining stability a heuristic for determining a near optimal value of is proposed a second form of preconditioning symmetrically scaled element by element due to bartelt is also shown to perform robustly over a range of problems it does not require assembly of the global stiness matrix and has great potential for parallelization 1997 by john wiley sons ltd

a combinatorial approach to the classification problem we study the two group classification problem which involves classifying an observation into one of two groups based on its attributes the classification rule is a hyperplane which misclassifies the fewest number of observations in the training sample exact and heuristic algorithms for solving the problem are presented computational results confirm the efficiency of this approach

dea discriminant analysis methodological comparison among eight discriminant analysis approaches abstract discriminant analysis da is a classification method that can predict the group membership of a newly sampled observation recently a new type of non parametric da approach is proposed to provide a set of weights of a discriminant function consequently yielding an evaluation score for the determination of group membership the non parametric da is referred to as data envelopment analysis discriminant analysis dea da because it maintains its discriminant capabilities by incorporating the non parametric feature of dea into da in this study a use of the mixed integer approach of dea da is compared with other da methods it is confirmed that it performs at least as well as the other well known da methods the proposed approach is further reformulated in a manner that it can deal with classification of more than two groups

an adaptive statistical method for the discriminant problem abstract this paper proposes an adaptive statistical method for the discriminant problem the method selects fisher s linear or smith s quadratic discriminant function or the nearest neighbor method for use on the holdout sample depending upon which method minimizes the sum of overall accuracy and balance on the estimation sample a simulation study examines the two group discriminant problem with variables generated from both bivariate normal and nonnormal distributions the resulting misclassification rates indicate that the adaptive method is an effective alternative to existing statistical and linear programming methods

a road to classification in high dimensional space the regularized optimal affine discriminant for high dimensional classification it is well known that naively performing the fisher discriminant rule leads to poor results due to diverging spectra and noise accumulation therefore researchers proposed independence rules to circumvent the diverging spectra and sparse independence rules to mitigate the issue of noise accumulation however in biological applications there are often a group of correlated genes responsible for clinical outcomes and the use of the covariance information can significantly reduce misclassification rates in theory the extent of such error rate reductions is unveiled by comparing the misclassification rates of the fisher discriminant rule and the independence rule to materialize the gain based on finite samples a regularized optimal affine discriminant road is proposed road selects an increasing number of features as the regularization relaxes further benefits can be achieved when a screening method is employed to narrow the feature pool before hitting the road an efficient constrained coordinate descent algorithm ccd is also developed to solve the associated optimization problems sampling properties of oracle type are established simulation studies and real data analysis support our theoretical results and demonstrate the advantages of the new classification procedure under a variety of correlation structures a delicate result on continuous piecewise linear solution path for the road optimization problem at the population level justifies the linear interpolation of the ccd algorithm

a piecewise linear programming approach to the two group discriminant problem an adaptation to fisher s linear discriminant function model abstract most existing linear programming lp models have optimization objectives that are very different from fisher s linear discriminant function fldf an lp technique that adapts to fldf to solve the two group classification problem is desirable as fldf is one of the most popular classification rules therefore this paper introduces a piecewise linear programming plp p approach that has an optimization objective very similar to that of fldf to solve the two group classification problem in discriminant analysis moreover the paper compares the classificatory performance between fldf and the new plp p model and shows that the results from both approaches are as good as each other when applied to three published data sets however the new plp p is more flexible than fldf in terms of adding different types of constraints and weighting individual observations the results of a simulation experiment confirm the value of our proposed approach

modified cholesky factorizations for sparse preconditioners in large scale unconstrained optimization problems preconditioned conjugate gradient techniques are often used to solve the newton equations approximately in certain applications natural sparse preconditioners can be derived from the structure of the problem and can accelerate convergence significantly since these preconditioners are not necessarily positive definite modified cholesky mc factorizations can be applied to construct related positive definite preconditioners this paper describes such an adaptation of two mc techniques gmw gill murray wright and se schnabel eskow in a truncated newton minimization method this paper then analyzes their effects on three interesting test problems of moderate size the preliminary results suggest that the two mc algorithms perform quite differently in practice trends can be noted for sparse problems that differ from the dense case differences in the size of the modifications and in their variances throughout the minimization are observed and relat

application of diagonally perturbed incomplete factorization preconditioned conjugate gradient algorithms for edge finite element analysis of helmholtz equations the diagonally perturbed incomplete factorization preconditioning scheme is applied to the conjugate gradient cg method for solving a large system of linear equations resulting from the use of edge based finite element method fem this scheme contains more global information about the coefficient matrix when compared with banded matrix schemes the efficient implementation of this preconditioned cg pcg algorithm is described in detail for complex coefficient matrix equation on several electromagnetic problems the pcg approach converges in cpu time which is 8 6 19 5 times shorter with respect to the cg approach by comparison with other preconditioned techniques the results demonstrate that incomplete factorization preconditioning strategy is especially effective for cg iterative method when edge fem is applied to solve large scale time harmonic electromagnetic field problems

blowup in diffusion equations a survey this paper deals with quasilinear reaction diffusion equations for which a solution local in time exists if the solution ceases to exist for some finite time we say that it blows up in contrast to linear equations blowup can occur even if the data are smooth and well defined for all times depending on the equation either the solution or some of its derivatives become singular we shall concentrate on those cases where the solution becomes unbounded in finite time this can occur in quasilinear equations if the heat source is strong enough there exist many theoretical studies on the question on the occurrence of blowup in this paper we shall recount some of the most interesting criteria and most important methods for analyzing blowup the asymptotic behavior of solutions near their singularities is only completely understood in the special case where the source is a power a better knowledge would be useful also for their numerical treatment thus not surprisingly the numerical analysis of this type of problems is still at a rather early stage the goal of this paper is to collect some of the known results and algorithms and to direct the attention to some open problems

guidelines for the usage of incomplete decompositions in solving sets of linear equations as they occur in practical problems abstract this paper presents incomplete decompositions for various types of matrices as they occur in the implicit discretisation of practical problems a review is given of methods for the usual five point discretisation of a self adjoint elliptic second order partial differential equation in two dimensions on a square the matrices which occur in this type of problem are symmetric m matrices of very regular structure the convergence behaviour of the different decompositions for this case is demonstrated by numerical experiments the paper also gives decompositions for the following type of matrices i symmetric m matrices of a different structure ii symmetric positive definite matrices iii non symmetric matrices

credit risk evaluation using multi criteria optimization classifier with kernel fuzzification and penalty factors abstract with the fast development of financial products and services bank s credit departments collected large amounts of data which risk analysts use to build appropriate credit scoring models to evaluate an applicant s credit risk accurately one of these models is the multi criteria optimization classifier mcoc by finding a trade off between overlapping of different classes and total distance from input points to the decision boundary mcoc can derive a decision function from distinct classes of training data and subsequently use this function to predict the class label of an unseen sample in many real world applications however owing to noise outliers class imbalance nonlinearly separable problems and other uncertainties in data classification quality degenerates rapidly when using mcoc in this paper we propose a novel multi criteria optimization classifier based on kernel fuzzification and penalty factors kfp mcoc firstly a kernel function is used to map input points into a high dimensional feature space then an appropriate fuzzy membership function is introduced to mcoc and associated with each data point in the feature space and the unequal penalty factors are added to the input points of imbalanced classes thus the effects of the aforementioned problems are reduced our experimental results of credit risk evaluation and their comparison with mcoc support vector machines svm and fuzzy svm show that kfp mcoc can enhance the separation of different applicants the efficiency of credit risk scoring and the generalization of predicting the credit rank of a new credit applicant

on ψ learning the concept of large margins have been recognized as an important principle in analyzing learning methodologies including boosting neural networks and support vector machines svms however this concept alone is not adequate for learning in nonseparable cases we propose a learning methodology called learning that is derived from a direct consideration of generalization errors we provide a theory for learning and show that it essentially attains the optimal rates of convergence in two learning examples finally results from simulation studies and from breast cancer classification confirm the ability of learning to outperform svm in generalization

an experimental comparison of several approaches to the discriminant problem with both qualitative and quantitative variables abstract this paper reports the results of an experimental comparison between a linear programming approach and the well known statistical procedure by fisher for solving discriminant analysis problems this work represents an extension of the work of bajgier and hill as we incorporate qualitative variables into our design as well as enlarge the estimation sample from 30 to 90 our results indicate that both methods are enhanced by the inclusion of the qualitative variables but that the fisher approach seems preferable lastly we discuss the effect of the different experimental factors on the relative performance of the two methods

semi implicit complementary volume scheme for solving level set like equations in image processing and curve evolution we introduce linear semi implicit complementary volume numerical scheme for solving level set like nonlinear degenerate diffusion equations arising in image processing and curve evolution problems we study discretization of image selective smoothing equation of mean curvature flow type given by alvarez lions and morel 3 solution of the level set equation of osher and sethian 26 30 is also included in the study we prove l infty and w 1 1 estimates for the proposed scheme and give existence of its generalized solution in every discrete time scale step efficiency of the scheme is given by its linearity and stability preconditioned iterative solvers are used for computing arising linear systems we present computational results related to image processing and plane curve evolution

a fuzzy neural network algorithm for multigroup classification we propose a multigroup classification algorithm based on a hybrid fuzzy neural net framework a key feature of the approach is the adaptation of membership functions to new data in this way learning is reflected in the shape of the membership functions by defining separate membership functions for each fuzzy output class we allow dynamic adjustment of the functions during training the algorithm is successfully tested with real economic data the results suggest economically meaningful interpretations

improper linear discriminant classifiers abstract mathematical programming formulations for determining linear discriminant classifiers have received considerable interest in recent years although their performance rivals classical methods and requires fewer assumptions many of the formulations exhibit undesirable properties in this paper we draw attention to two additional problems that the mathematical programming formulations face a solution for each type of problem is presented