benchmarking optimization software with performance profiles we propose performance profiles distribution functions for a performance metric as a tool for benchmarking and comparing optimization software we show that performance profiles combine the best features of other tools for performance evaluation

a shamanskii like self adaptive levenberg marquardt method for nonlinear equations abstract in this paper we are concerned with the shamanskii like self adaptive levenberg marquardt methods for nonlinear equations we consider two choices of levenberg marquardt parameter one of them is the standard self adaptive levenberg marquardt parameter the other is nonmonotone self adaptive levenberg marquardt parameter by using the nonmonotone technique of grippo lampariello and lucidi under the error bound condition which is weaker than nonsingularity we show that the shamanskii like self adaptive levenberg marquardt methods converge with q order m 1 numerical experiments show the new algorithms are efficient and promising

on the global convergence of the bfgs method for nonconvex unconstrained optimization problems this paper is concerned with the open problem of whether the bfgs method with inexact line search converges globally when applied to nonconvex unconstrained optimization problems we propose a cautious bfgs update and prove that the method with either a wolfe type or an armijo type line search converges globally if the function to be minimized has lipschitz continuous gradients

functional clustering by bayesian wavelet methods summary we propose a nonparametric bayes wavelet model for clustering of functional data the wavelet based methodology is aimed at the resolution of generic global and local features during clustering and is suitable for clustering high dimensional data based on the dirichlet process the nonparametric bayes model extends the scope of traditional bayes wavelet methods to functional clustering and allows the elicitation of prior belief about the regularity of the functions and the number of clusters by suitably mixing the dirichlet processes posterior inference is carried out by gibbs sampling with conjugate priors which makes the computation straightforward we use simulated as well as real data sets to illustrate the suitability of the approach over other alternatives

classification of non parametric regression functions in longitudinal data models summary we investigate a longitudinal data model with non parametric regression functions that may vary across the observed individuals in a variety of applications it is natural to impose a group structure on the regression curves specifically we may suppose that the observed individuals can be grouped into a number of classes whose members all share the same regression function we develop a statistical procedure to estimate the unknown group structure from the data moreover we derive the asymptotic properties of the procedure and investigate its finite sample performance by means of a simulation study and a real data example

global convergence of a cass of quasi newton methods on convex problems we study the global convergence properties of the restricted broyden class of quasi newton methods when applied to a convex objective function we assume that the line search satisfies a standard sufficient decrease condition and that the initial hessian approximation is any positive definite matrix we show global and superlinear convergence for this class of methods except for dfp this generalizes powell s well known result for the bfgs method the analysis gives us insight into the properties of these algorithms in particular it shows that dfp lacks a very desirable self correcting property possessed by bfgs

a tool for the analysis of quasi newton methods with application to unconstrained minimization the bfgs update formula is shown to have an important property that is inde pendent of the algorithmic context of the update and that is relevant to both constrained and unconstrained optimization the bfgs method for unconstrained optimization using a variety of line searches including backtracking is shown to be globally and superlinearly convergent on uniformly convex problems the analysis is particularly simple due to the use of some new tools introduced in this paper

a family of scaled factorized broyden like methods for nonlinear least squares problems in this paper a family of scaled factorized broyden like methods for unconstrained nonlinear least squares problems with zero or nonzero residual is presented this family is based on a full rank factorized form of a structured quasi newton update and a scaled approximation of the second order term using given information the resulting algorithms yield a q quadratic convergence for the zero residual case and a q superlinear convergence for the nonzero residual case and maintain at least locally the positive definiteness of the approximate hessian matrices

a new hybrid iteration method for solving algebraic equations abstract a new hybrid iteration method the hybrid s name was used by luo xing guo luo applied mathematics and computation 171 2 2005 1171 1183 has been proposed for solving a non linear algebraic equation f x 0 by using taylor s theorem in this paper we proposed a new hybrid iteration method and we show by one equation that this new hybrid iteration method is more quickly convergent than newton s method and hence than hybrid iteration method

a new iteration method for solving algebraic equations in this paper a nonlinear algebraic equation f x 0 by applying taylor s theorem is converted into a coupled iteration systeml x n 1 g x n 0andg x n f x n l x n wherel x ax 2 bx c the formula is of high convergence some examples are given

convergence properties of the bfgs algoritm the bfgs method is one of the most famous quasi newton algorithms for unconstrained optimization in 1984 powell presented an example of a function of two variables that shows that the polak ribiere polyak prp conjugate gradient method and the bfgs quasi newton method may cycle around eight nonstationary points if each line search picks a local minimum that provides a reduction in the objective function in this paper a new technique of choosing parameters is introduced and an example with only six cyclic points is provided it is also noted through the examples that the bfgs method with wolfe line searches need not converge for nonconvex objective functions

a new iteration method with cubic convergence to solve nonlinear algebraic equations abstract in this paper a new iteration scheme is proposed to solve the roots of an algebraic equation f x 0 given an initial guess x0 the roots of the equation can be obtained using the following iteration scheme x n 1 x n f x n f 2 x n 2 f x n f x n f x n this iteration scheme has unique convergence characteristics different from the well known newton s method it is shown that this iteration method has cubic local convergence in the neighborhood of the root using this scheme real or complex roots for specific algebraic equations can be found because there are two iteration directions for a given initial guess two solutions can be found for certain algebraic equations with multiple roots examples are presented and compared with other methods

multiscale clustering of nonparametric regression curves abstract in a wide range of modern applications one observes a large number of time series rather than only a single one it is often natural to suppose that there is some group structure in the observed time series when each time series is modeled by a nonparametric regression equation one may in particular assume that the observed time series can be partitioned into a small number of groups whose members share the same nonparametric regression function we develop a bandwidth free clustering method to estimate the unknown group structure from the data more precisely speaking we construct multiscale estimators of the unknown groups and their unknown number which are free of classical bandwidth or smoothing parameters in the theoretical part of the paper we analyze the statistical properties of our estimators our theoretical results are derived under general conditions which allow the data to be dependent both in time series direction and across different time series the technical analysis of the paper is complemented by simulated and real data examples

on the convergence of the modified levenberg marquardt method with a nonmonotone second order armijo type line search recently fan j fan the modified levenberg marquardt method for nonlinear equations with cubic convergence math comput 81 2012 447 466 proposed a modified levenberg marquardt mlm method for nonlinear equations using a trust region technique global and cubic convergence of the mlm method is proved by fan 2012 12 under the local error bound condition which is weaker than nonsingularity the purpose of the paper is to investigate the convergence properties of the mlm method with a line search technique since the search direction of the mlm method may be not a descent direction standard line searches can not be used directly in this paper we propose a nonmonotone second order armijo line search which guarantees the global convergence of the mlm method moreover we prove that the unit step will be always accepted finally then cubic convergence of the mlm method is preserved under the local error bound condition some preliminary numerical results are also reported

new family of iterative methods for nonlinear equations in this paper we suggest and analyze a new family of iterative methods for solving nonlinear equations using the system of coupled equations coupled with decomposition technique several numerical examples are given to illustrate the efficiency and performance of the new methods these new iterative methods may be viewed as an extension and generalization of the existing methods for solving nonlinear equations

new iterative schemes for nonlinear equations in this paper we suggest and analyze a new iterative method for solving nonlinear equations by using a new decomposition method we also discuss the convergence criteria of these iterative methods several numerical examples are given to illustrate the efficiency and performance of the new methods these new iterative methods may be viewed as an extension and generalization of the existing methods for solving nonlinear equations

on the use of product structure in secant methods for nonlinear least squares problems the problem of adjusting the second order term in secant methods for nonlinear least squares problems with zero residual is addressed the author uses the framework of structured secant methods to derive and investigate a new way to resize the approximation of the second order term using some exact information the resulting method is a self adjusting structured secant method for nonlinear least squares problems yielding a q quadratic convergence rate for zero residual and a q superlinear convergence rate for nonzero residual problems

a modified bfgs type quasi newton method with line search for symmetric nonlinear equations problems abstract in this paper using approximate gradient of the norm square metric function we present an inexact mbfgs method with line search for solving symmetric nonlinear equations which is a generalization of the mbfgs method proposed by li and fukushima 2001 for solving smooth unconstrained optimization the used approximate gradient of the proposed method only requires computing two residuals at every iteration and is easy to obtain by utilizing symmetric structure of the system we establish global convergence of the proposed method and report some numerical results

strong consistency of k means clustering a random sample is divided into the k clusters that minimise the within cluster sum of squares conditions are found that ensure the almost sure convergence as the sample size increases of the set of means of the k clusters the result is proved for a more general clustering criterion

new iterative methods for solving nonlinear equation by using homotopy perturbation method in this paper we suggest and analyze a new class of iterative methods for solving nonlinear equations by using the homotopy perturbation method convergence of their method is also considered here we also discuss the efficiency index and computational order of convergence of new methods several numerical examples are given to illustrate the efficiency and performance of these new methods these new iterative methods may be viewed as an extension and generalization of the existing methods for solving nonlinear equations

a fourth order iterative method for solving nonlinear equations the present paper illustrates an iterative numerical method to solve nonlinear equations of the form f x 0 especially those containing the partial and non partial involvement of transcendental terms comparative analysis shows that the present method is faster than newton raphson method hybrid iteration method new hybrid iteration method and others cost is also found to be minimum than these methods the beauty in our method can be seen because of the optimization in important effecting factors i e lesser number of iteration steps lesser number of functional evaluations and lesser value of absolute error in final as well as in individual step as compared to the other methods this work also demonstrates the higher order convergence of the present method as compared to others without going to the computation of second derivative

properties of principal component methods for functional and longitudinal data analysis the use of principal component methods to analyze functional data is appropriate in a wide range of different settings in studies of functional data analysis it has often been assumed that a sample of random functions is observed precisely in the continuum and without noise while this has been the traditional setting for functional data analysis in the context of longitudinal data analysis a random function typically represents a patient or subject who is observed at only a small number of randomly distributed points with nonnegligible measurement error nevertheless essentially the same methods can be used in both these cases as well as in the vast number of settings that lie between them how is performance affected by the sampling plan in this paper we answer that question we show that if there is a sample of n functions or subjects then estimation of eigenvalues is a semiparametric problem with root n consistent estimators even if only a few observations are made of each function and if each observation is encumbered by noise however estimation of eigenfunctions becomes a nonparametric problem when observations are sparse the optimal convergence rates in this case are those which pertain to more familiar function estimation settings we also describe the effects of sampling at regularly spaced points as opposed to random points in particular it is shown that there are often advantages in sampling randomly however even in the case of noisy data there is a threshold sampling rate depending on the number of functions treated above which the rate of sampling either randomly or regularly has negligible impact on estimator performance no matter whether eigenfunctions or eigenvectors are being estimated

a central limit theorem for k means clustering a set of n points in euclidean space is partitioned into the k groups that minimize the within groups sum of squares under the assumption that the n points come from independent sampling on a fixed distribution conditions are found to assure asymptotic normality of the vector of means of the k groups the method of proof makes novel application of a functional central limit theorem for empirical processes a generalization of donsker s theorem due to dudley

global convergence of a new hybrid gauss newton structured bfgs method for nonlinear least squares problems in this paper we propose a hybrid gauss newton structured bfgs method with a new update formula and a new switch criterion for the iterative matrix to solve nonlinear least squares problems we approximate the second term in the hessian by a positive definite bfgs matrix under suitable conditions global convergence of the proposed method with a backtracking line search is established moreover the proposed method automatically reduces to the gauss newton method for zero residual problems and the structured bfgs method for nonzero residual problems in a neighborhood of an accumulation point a locally quadratic convergence rate for zero residual problems and a locally superlinear convergence rate for nonzero residual problems are obtained for the proposed method some numerical results are given to compare the proposed method with some existing methods

three step iterative methods for nonlinear equations in this paper we suggest and analyze a new three step iterative method for solving nonlinear equations we show that this new iterative method has third order convergence several numerical examples are given to illustrate the efficiency and performance of the new method

local and superlinear convergence for partially known quasi newton methods this paper develops a unified theory for establishing the local and q superlinear convergence of quasi newton methods from the convex class when part of the hessian matrix is known one first proves the bounded deterioration principle due to dennis and popularized by broyden dennis and more for the appropriate modifications of all update formulas in the convex broyden class using standard conditions on the quasi newton updates one then deduces local and q superlinear convergence particular cases of these methods are the sqp augmented scale bfgs and dfp secant methods for constrained optimization problems introduced by tapia and a generalization of the al baali and fletcher modification of the structured secant method considered by dennis gay and welsch for the nonlinear least squares problem in all cases bounded deterioration is proved for the approximate hessian not for its inverse

clustering for sparsely sampled functional data we develop a flexible model based procedure for clustering functional data the technique can be applied to all types of curve data but is particularly useful when individuals are observed at a sparse set of time points in addition to producing final cluster assignments the procedure generates predictions and confidence intervals for missing portions of curves our approach also provides many useful tools for evaluating the resulting models clustering can be assessed visually via low dimensional representations of the curves and the regions of greatest separation between clusters can be determined using a discriminant function finally we extend the model to handle multiple functional and finite dimensional covariates and show how it can be applied to standard finite dimensional clustering problems involving missing data

comparing the shapes of regression functions summary does a regression function follow a specified shape do two regression functions have the same shape how can regression functions be grouped based on shape these questions can occur when investigating monotonicity when counting local maxima or when studying variation in families of curves one can address these questions by considering the rank correlation coefficient between two functions this correlation is a generalisation of the rank correlation between two finite sets of numbers and is equal to one if and only if the two functions have the same shape a sample rank correlation based on smoothed estimates of the regression functions consistently estimates the true correlation this sample rank correlation can be used as a measure of similarity between functions in cluster analysis and as a measure of monotonicity or modality

comparing nonparametric versus parametric regression fits in general there will be visible differences between a parametric and a nonparametric curve estimate it is therefore quite natural to compare these in order to decide whether the parametric model could be justified an asymptotic quantification is the distribution of the integrated squared difference between these curves we show that the standard way of bootstrapping this statistic fails we use and analyse a different form of bootstrapping for this task we call this method the wild bootstrap and apply it to fitting engel curves in expenditure data analysis

iterative methods with fourth order convergence for nonlinear equations in this paper we suggest and analyze a new three step iterative method for solving nonlinear equations involving only first derive of the function using a new decomposition technique which is due to noor m aslam noor numerical analysis and optimization lecture notes comsats institute of information technology islamabad pakistan 2006 m aslam noor new iterative schemes for nonlinear equations appl math comput in press and noor and noor m aslam noor k inayat noor some iterative schemes for nonlinear equations appl math comput in press we show that this new iterative method has fourth order convergence several numerical examples are given to illustrate the efficiency and performance of the new method

heterogeneous panel data models with cross sectional dependence abstract this paper considers a semiparametric panel data model with heterogeneous coefficients and individual specific trending functions where the random errors are assumed to be serially correlated and cross sectionally dependent we propose mean group estimators for the coefficients and trending functions involved in the model it can be shown that the proposed estimators can achieve an asymptotic consistency with rates of root n t and root n t h respectively as n t where n is allowed to increase faster than t furthermore a statistic for testing homogeneous coefficients is constructed based on the difference between the mean group estimator and a pooled estimator its asymptotic distributions are established under both the null and a sequence of local alternatives even if the difference between these estimators vanishes considerably fast can achieve root n t 2 rate at most under the null and no consistent estimator for the covariance matrix is required the finite sample performance of the proposed estimators together with the size and local power properties of the test are demonstrated by simulated data examples and an empirical application with the oecd health care expenditure dataset is also provided

on modified newton methods for solving a non linear algebraic equations abstract in this paper we present a new modified newton method for solving a non linear algebraic equation by using the well known nedzhibov s method d k r babajee m z dauhoo an analysis of the properties of the variants of newton s method with third order convergence applied mathematics and computation 183 1 2006 659 684 we proposed a new iteration method and we show by one equation that this new iteration method is more quickly convergence than newton s method

the bfgs method with exact line searches fails for non convex objective functions this work shows that the bfgs method and other methods in the broyden class with exact line searches may fail for non convex objective functions

on the convergence of a modified regularized newton method for convex optimization with singular solutions in this paper we propose a modified regularized newton method for convex minimization problems whose hessian matrices may be singular the proposed method is proved to converge globally if the gradient and hessian of the objective function are lipschitz continuous under the local error bound condition we first show that the method converges quadratically which implies that x k x is equivalent to dist x k x where x is the solution set and x k x x then we in turn prove the cubic convergence of the proposed method under the same local error bound condition which is weaker than nonsingularity

panel threshold regressions with latent group structures abstract in this paper we consider the least squares estimation of a panel structure threshold regression pstr model where both the slope coefficients and threshold parameters may exhibit latent group structures we study the asymptotic properties of the estimators of the latent group structure and the slope and threshold coefficients we show that we can estimate the latent group structure correctly with probability approaching 1 and the estimators of the slope and threshold coefficients are asymptotically equivalent to the infeasible estimators that are obtained as if the true group structures were known we study likelihood ratio based inferences on the group specific threshold parameters under the shrinking threshold effect framework we also propose two specification tests one tests whether the threshold parameters are homogeneous across groups and the other tests whether the threshold effects are present when the number of latent groups is unknown we propose a bic type information criterion to determine the number of groups in the data simulations demonstrate that our estimators and tests perform reasonably well in finite samples we apply our model to revisit the relationship between capital market imperfection and the investment behavior of firms and to examine the impact of bank deregulation on income inequality we document a large degree of heterogeneous effects in both applications that cannot be captured by conventional panel threshold regressions

approximate gauss newton methods for nonlinear least squares problems the gauss newton algorithm is an iterative method regularly used for solving nonlinear least squares problems it is particularly well suited to the treatment of very large scale variational data assimilation problems that arise in atmosphere and ocean forecasting the procedure consists of a sequence of linear least squares approximations to the nonlinear problem each of which is solved by an inner direct or iterative process in comparison with newton s method and its variants the algorithm is attractive because it does not require the evaluation of second order derivatives in the hessian of the objective function in practice the exact gauss newton method is too expensive to apply operationally in meteorological forecasting and various approximations are made in order to reduce computational costs and to solve the problems in real time here we investigate the effects on the convergence of the gauss newton method of two types of approximation used commonly in data assimilation first we examine truncated gauss newton methods where the inner linear least squares problem is not solved exactly and second we examine perturbed gauss newton methods where the true linearized inner problem is approximated by a simplified or perturbed linear least squares problem we give conditions ensuring that the truncated and perturbed gauss newton methods converge and also derive rates of convergence for the iterations the results are illustrated by a simple numerical example a practical application to the problem of data assimilation in a typical meteorological system is presented

cats clustering after transformation and smoothing cats clustering after transformation and smoothing is a technique for nonparametrically estimating and clustering a large number of curves our motivating example is a genetic microarray experiment but the method is very general the method includes transformation and smoothing multiple curves multiple nonparametric testing for screening out flat curves clustering curves with similar shape and nonparametrically inferring the clustering estimation error rate

factorized quasi newton methods for nonlinear least squares problems this paper provides a modification to the gauss newton method for nonlinear least squares problems the new method is based on structured quasi newton methods which yield a good approximation to the second derivative matrix of the objective function in particular we propose bfgs like and dfp like updates in a factorized form which give descent search directions for the objective function we prove local and q superlinear convergence of our methods and give results of computational experiments for the bfgs like and dfp like updates

functional clustering and identifying substructures of longitudinal data summary a functional clustering fc method k centres fc for longitudinal data is proposed the k centres fc approach accounts for both the means and the modes of variation differentials between clusters by predicting cluster membership with a reclassification step the cluster membership predictions are based on a non parametric random effect model of the truncated karhunen loeve expansion coupled with a non parametric iterative mean and covariance updating scheme we show that under the identifiability conditions derived the k centres fc method proposed can greatly improve cluster quality as compared with conventional clustering algorithms moreover by exploring the mean and covariance functions of each cluster thek centres fc method provides an additional insight into cluster structures which facilitates functional cluster analysis practical performance of the k centres fc method is demonstrated through simulation studies and data applications including growth curve and gene expression profile data

on a class of quadratically convergent iteration formulae in this paper a new class of iterative formulae having quadratic convergence is presented furthermore these algorithms are comparable to the well known method of newton and the computed results support this theory

a modified bfgs method and its global convergence in nonconvex minimization in this paper we propose a modification of the bfgs method for unconstrained optimization a remarkable feature of the proposed method is that it possesses a global convergence property even without convexity assumption on the objective function under certain conditions we also establish superlinear convergence of the method

nonparametric estimation and testing of fixed effects panel data models in this paper we consider the problem of estimating nonparametric panel data models with fixed effects we introduce an iterative nonparametric kernel estimator we also extend the estimation method to the case of a semiparametric partially linear fixed effects model to determine whether a parametric semiparametric or nonparametric model is appropriate we propose test statistics to test between the three alternatives in practice we further propose a test statistic for testing the null hypothesis of random effects against fixed effects in a nonparametric panel data regression model simulations are used to examine the finite sample performance of the proposed estimators and the test statistics

a note on the new iteration method for solving algebraic equations in ji huan he a new iteration method for solving algebraic equations appl math comput 135 2003 81 84 a new iteration method for solving algebraic equations has been proposed in this paper we show for one equation which given by the author and some other equations that the new iteration method fails to obtain expected results hence in contrast to previous claims it is no more quickly convergent than newton s method and may in fact be failed