convergence of newton like iterative methods newton like methods in which the intermediate systems of linear equations are solved by iterative techniques are examined by applying the theory of inexact newton methods radius of convergence and rate of convergence results are easily obtained the analysis is carried out in affine invariant terms the results are applicable to cases where the underlying newton like method is for example a difference newton like or update newton method

efficient estimation of sparse jacobian matrices by differences abstract in difference newton like methods for solving f x 0 the jacobian matrix f x is approximated by differences between values of f if f x is sparse a consistent partition of its columns can be exploited to approximate f x using relatively few values of f we provide a local convergence theory for the resulting methods a superlinearly convergent stable cyclic secant method in which at each iteration two values of f are required and several columns of the jacobian matrix approximation are updated simultaneously is developed

nonmonotone backtracking inexact quasi newton algorithms for solving smooth nonlinear equations a family of inexact quasi newton algorithms in association with nonmonotone backtracking line search technique is proposed for solving smooth nonlinear equations global convergence of the proposed algorithms are established under the reasonable conditions we characterize the order of local convergence based on convergence behaviour of the approximate matrix of the jacobian and indicate how to choose an inexact forcing sequence which preserves the rapid convergence of the proposed algorithms

the impact of cluster representatives on the convergence of the k modes type clustering as a leading partitional clustering technique k modes is one of the most computationally efficient clustering methods for categorical data in the k modes a cluster is represented by a x201c mode x201d which is composed of the attribute value that occurs most frequently in each attribute domain of the cluster whereas in real applications using only one attribute value in each attribute to represent a cluster may not be adequate as it could in turn affect the accuracy of data analysis to get rid of this deficiency several modified clustering algorithms were developed by assigning appropriate weights to several attribute values in each attribute although these modified algorithms are quite effective their convergence proofs are lacking in this paper we analyze their convergence property and prove that they cannot guarantee to converge under their optimization frameworks unless they degrade to the original k modes type algorithms furthermore we propose two different modified algorithms with weighted cluster prototypes to overcome the shortcomings of these existing algorithms we rigorously derive updating formulas for the proposed algorithms and prove the convergence of the proposed algorithms the experimental studies show that the proposed algorithms are effective and efficient for large categorical datasets

local convergence of difference newton like methods using affine invariant terms we give a local convergence analysis of difference newton like methods for solving the nonlinear equation f x 0 the convergence conditions are weaker than those standardly required for methods of this class the technique and results are valid for all currently known difference newton like methods which require evaluation of all components of f at the same point radius of convergence and rate of convergence results for particular difference newton like methods may easily be derived from the results reported here

concurrent forward bounding for distributed constraint optimization problems a distributed search algorithm for solving distributed constraints optimization problems dcops is presented the new algorithm scans the search space by using multiple search processes sps that run on all agents concurrently sps search in non intersecting parts of the global search space and perform branch bound search each search process sp uses the mechanism of forward bounding fb to prune efficiently its part of the global search space the concurrent forward bounding concfb algorithm enables all sps to share their upper bound across all parts of the global search space the number of concurrent sps is controlled dynamically by the concfb algorithm by performing dynamic splitting within each sp a dynamic variable ordering is employed in order to help control the balance of computational load among all agents and across different sps the concfb algorithm is evaluated experimentally and compared to all state of the art dcop algorithms the number of non concurrent logical operations non concurrent steps the total number of messages sent and cpu time are used as performance metrics the evaluation procedure considers different dcop problem types with a varying number of agents and different constraint graphs as problems become larger and denser concfb is shown to outperform all other evaluated algorithms by 2 3 orders of magnitude in all performance measures further evaluations comparing different variants of concfb provide important insights into the working of the algorithm and reveals the contribution of its different components

space structure and clustering of categorical data learning from categorical data plays a fundamental role in such areas as pattern recognition machine learning data mining and knowledge discovery to effectively discover the group structure inherent in a set of categorical objects many categorical clustering algorithms have been developed in the literature among which k modes type algorithms are very representative because of their good performance nevertheless there is still much room for improving their clustering performance in comparison with the clustering algorithms for the numeric data this may arise from the fact that the categorical data lack a clear space structure as that of the numeric data to address this issue we propose in this paper a novel data representation scheme for the categorical data which maps a set of categorical objects into a euclidean space based on the data representation scheme a general framework for space structure based categorical clustering algorithms sbc is designed this framework together with the applications of two kinds of dissimilarities leads two versions of the sbc type algorithms to verify the performance of the sbc type algorithms we employ as references four representative algorithms of the k modes type algorithms experiments show that the proposed sbc type algorithms significantly outperform the k modes type algorithms

concurrent search for distributed csps a distributed concurrent search algorithm for distributed constraint satisfaction problems discsps is presented concurrent search algorithms are composed of multiple search processes sps that operate concurrently and scan non intersecting parts of the global search space each sp is represented by a unique data structure containing a current partial assignment cpa that is circulated among the different agents search processes are generated dynamically started by the initializing agent and by any number of agents during search in the proposed concdb algorithm all search processes perform dynamic backtracking as a consequence of backjumping a search space can be found unsolvable by a different search process this enhances the efficiency of the concdb algorithm concurrent dynamic backtracking is an asynchronous distributed algorithm and is shown to be faster than former algorithms for solving discsps experimental evaluation of concdb on randomly generated discsps demonstrates that the network load of concdb is similar to the network load of synchronous backtracking and is much lower than that of asynchronous backtracking the advantage of concurrent search is more pronounced in the presence of imperfect communication when messages are randomly delayed

a modified chebyshev s iterative method with at least sixth order of convergence this paper is devoted to the construction and analysis of a high order variant of the classical chebyshev method the method has order of convergence at least six for simple roots the extension to system of equations and its semilocal convergence for nonlinear equations are presented finally an application to well known algebraic riccati equation is considered

soft subspace clustering of categorical data with probabilistic distance categorical data clustering is an important subject in pattern recognition currently subspace clustering of categorical data remains an open problem due to the difficulties in estimating attribute interestingness according to the statistics of categories in clusters in this paper a new algorithm is proposed for clustering categorical data with a novel soft feature selection scheme by which each categorical attribute is automatically assigned a weight that correlates with the smoothed dispersion of the categories in a cluster in the proposed algorithm dissimilarity between categorical data objects is measured using a probabilistic distance function based on kernel density estimation for categorical attributes we also make use of the probabilistic distances to define a cluster validity index for estimating the number of categorical clusters the suitability of the proposal is demonstrated in an empirical study done with some widely used real world data sets and synthetic data sets and the results show its outstanding performance highlightswe define the cluster scatter on object to cluster distances for categorical data we propose a probabilistic distance function using a kernel density estimation method categorical attributes are weighted based on the smoothed dispersion of categories two weighting schemes are offered depending on the attribute types significantly improve clustering performance compared to mode based algorithms

on the convergence of some projection methods with perturbation abstract we provide a general theorem for the convergence of some projection methods with perturbation under zabrejko nguen type assumptions our results extend and improve several situations already in the literature

a theoretical evaluation of selected backtracking algorithms abstract in recent years many new backtracking algorithms for solving constraint satisfaction problems have been proposed the algorithms are usually evaluated by empirical testing this method however has its limitations our paper adopts a different purely theoretical approach which is based on characterizations of the sets of search tree nodes visited by the backtracking algorithms a notion of inconsistency between instantiations and variables is introduced and is shown to be a useful tool for characterizing such well known concepts as backtrack backjump and domain annihilation the characterizations enable us to a prove the correctness of the algorithms and b partially order the algorithms according to two standard performance measures the number of nodes visited and the number of consistency checks performed among other results we prove the correctness of backjumping and conflict directed backjumping and show that forward checking never visits more nodes than backjumping our approach leads us also to propose a modification to two hybrid backtracking algorithms backmarking with backjumping bmj and backmarking with conflict directed backjumping bm cbj so that they always perform fewer consistency checks than the original algorithms

locating the phase transition in binary constraint satisfaction problems abstract the phase transition in binary constraint satisfaction problems i e the transition from a region in which almost all problems have many solutions to a region in which almost all problems have no solutions as the constraints become tighter is investigated by examining the behaviour of samples of randomly generated problems in contrast to theoretical work which is concerned with the asymptotic behaviour of problems as the number of variables becomes larger this paper is concerned with the location of the phase transition in finite problems the accuracy of a prediction based on the expected number of solutions is discussed it is shown that the variance of the number of solutions can be used to set bounds on the phase transition and to indicate the accuracy of the prediction a class of sparse problems for which the prediction is known to be inaccurate is considered in detail it is shown that for these problems the phase transition depends on the topology of the constraint graph as well as on the tightness of the constraints

an improved error analysis for newton like methods under generalized conditions we introduce new semilocal convergence theorems for newton like methods in a banach space setting using new and very general conditions we provide different sufficient convergence conditions than before this way we introduce more precise majorizing sequences which in turn lead to finer error estimates and a better information on the location of the solution moreover for special choices of majorizing functions our results reduce to earlier ones in the local case we obtain a larger convergence radius ball finally as an application we show that in the case of newton s method the famous newton kantorovich hypothesis can be weakened under the same information

forward bounding on pseudo trees for dcops and adcops abstract complete search algorithms for solving distributed constraint optimization problems dcops can be divided into two groups algorithms that use a pseudo tree and algorithms that do not use one the best performing algorithms that do not use a pseudo tree use some form of forward bounding in order to try and gain from both worlds a new algorithm which incorporates a hybrid approach of the two groups is presented the proposed algorithm pseudo tree forward bounding pt fb is shown to perform very well pt fb is next extended to be able to solve asymmetric dcops adcops here again its performance is better than its competitors an extensive experimental evaluation of the performance of each of the proposed algorithms is presented

rapid and brief communication a k populations algorithm for clustering categorical data in this paper the conventional k modes type algorithms for clustering categorical data are extended by representing the clusters of categorical data with k populations instead of the hard type centroids used in the conventional algorithms use of a population based centroid representation makes it possible to preserve the uncertainty inherent in data sets as long as possible before actual decisions are made the k populations algorithm was found to give markedly better clustering results through various experiments

an empirical study of phase transitions in binary constraint satisfaction problems abstract an empirical study of randomly generated binary constraint satisfaction problems reveals that for problems with a given number of variables domain size and connectivity there is a critical level of constraint tightness at which a phase transition occurs at the phase transition problems change from being soluble to insoluble and the difficulty of problems increases dramatically a theory developed by williams and hogg 44 and independently developed by smith 37 predicts where the hardest problems should occur it is shown that the theory is in close agreement with the empirical results except when constraint graphs are sparse

on the newton kantorovich hypothesis for solving equations the famous newton kantorovich hypothesis has been used for a long time as a sufficient condition for the convergence of newton s method to a solution of an equation in connection with the lipschitz continuity of the frechet derivative of the operator involved here using lipschitz and center lipschitz conditions we show that the newton kantorovich hypothesis can be weakened the error bounds obtained under our semilocal convergence result are more precise than the corresponding ones given by the dominating newton kantorovich theorem

local convergence of inexact newton like iterative methods and applications abstract we provide local convergence results in affine form for inexact newton like as well as quasi newton iterative methods in a banach space setting we use hypotheses on the second or on the first and second frechet derivative of the operator involved our results allow a wider choice of starting points since our radius of convergence can be larger than the corresponding one given in earlier results using hypotheses on the first frechet derivative only a numerical example is provided to illustrate this fact our results apply when the method is for example a difference newton like or update newton method furthermore our results have direct applications to the solution of autonomous differential equations

concerning the terra incognita between convergence regions of two newton methods abstract the majorizing principle is used to show local and semilocal convergence of newton methods to a locally unique solution of a nonlinear operator in a banach space when the frechet derivative of the operator involved satisfies a center holder and a holder continuity condition then we investigate an unknown area terra incognita between the convergence regions of newton s method and the corresponding modified newton s method our approach compares favorably with other corresponding ones in this direction

a steffensen s type method in banach spaces with applications on boundary value problems in this paper a modified steffensen s type iterative scheme for the numerical solution of a system of nonlinear equations is studied two convergence theorems are presented the numerical solution of boundary value problems by the multiple shooting method using the proposed iterative scheme is analyzed

convergence theorems for least change secant update methods the purpose of this paper is to present a convergence analysis of least change secant methods in which part of the derivative matrix being approximated is computed by other means the theorems and proofs given here can be viewed as generalizations of those given by broyden dennis more j inst math appl 12 1973 pp 223 246 and by dennis more math comp 28 1974 pp 549 560 the analysis is done in the orthogonal projection setting of dennis schnabel siam rev 21 1980 pp 443 459 and many readers might feel that it is easier to understand the theorems here readily imply local and q superlinear convergence of all the standard methods in addition to proving these results for the first time for the sparse symmetric method of marwil and toint and the nonlinear least squares method of dennis gay welsch

a fuzzy k modes algorithm for clustering categorical data this correspondence describes extensions to the fuzzy k means algorithm for clustering categorical data by using a simple matching dissimilarity measure for categorical objects and modes instead of means for clusters a new approach is developed which allows the use of the k means paradigm to efficiently cluster large categorical data sets a fuzzy k modes algorithm is presented and the effectiveness of the algorithm is demonstrated with experimental results

third order iterative methods under kantorovich conditions we study a class of third order iterative methods for nonlinear equations on banach spaces a characterization of the convergence under kantorovich type conditions and optimal estimates of the error are found though in general these methods are not very extended due to their computational costs we will show some examples in which they are competitive and even cheaper than other simpler methods we center our analysis in both analytic and computational aspects

symbolic clustering using a new dissimilarity measure abstract a new dissimilarity measure based on position span and content of symbolic objects is proposed for symbolic clustering the dissimilarity measure is new in the sense that it is not just another aspect of a similarity measure in the proposed hierarchical agglomerative clustering methodology composite symbolic objects are formed using a cartesian join operator whenever a mutual pair of symbolic objects is selected for agglomeration based on minimum dissimilarity the minimum dissimilarity values of different merging levels are used to compute the cluster indicator values and hence to determine the number of clusters in the data the results of the application of the algorithm on numeric data of known number of classes are described first so as to show the efficacy of the method subsequently the results of the experiments on two data sets of assertion type of symbolic objects drawn from the domains of fat oil and microcomputers are presented

incremental learning based multiobjective fuzzy clustering for categorical data the problem of clustering categorical data where attribute values cannot be naturally ordered as numerical values has gained more importance in recent time due to the special properties of categorical attributes the clustering of categorical data seems to be more complicated than that of numerical data although a few clustering algorithms that optimize single clustering objective have been proposed it has found that such single measure may not be appropriate for all kind of datasets hence in this article an incremental learning based multiobjective fuzzy clustering for categorical data is proposed for this purpose a multiobjective modified differential evolution based fuzzy clustering algorithm is developed thereafter it integrates with the well known supervised classifier called random forest using incremental learning to propose the aforementioned technique here the multiobjective algorithm produces a set of optimal clustering solutions known as pareto optimal solutions by optimizing two conflicting objectives simultaneously subsequently through incremental learning using random forest classifier final solution is evolved from the ensemble pareto optimal solutions the results of the proposed method are demonstrated quantitatively and visually in comparison with widely used state of the art methods for six synthetic and four real life datasets finally statistical test is conducted to show the superiority of the results produced by the proposed method

local convergence of inexact newton methods a simple affine invariant condition which ensures local convergence of inexact newton methods is provided this condition is closely linked with the characterization of superlinear convergence given by dennis and more radius of convergence and rate of convergence results are derived and we show the theory is in some sense sharp we apply the theory to various inexact newton methods including difference newton like methods and the general newton like method in which the iterates are perturbed by rounding errors in the computation