a descent algorithm without line search for unconstrained optimization in this paper a new descent algorithm for solving unconstrained optimization problem is presented its search direction is descent and line search procedure can be avoided except for the first iteration it is globally convergent under mild conditions the search direction of the new algorithm is generalized and convergence of corresponding algorithm is also proved numerical results show that the algorithm is efficient for given test problems

a new nonmonotone spectral residual method for nonsmooth nonlinear equations abstract in this paper a new spectral residual method is proposed to solve systems of large scale nonlinear equations where the steplength is obtained by minimizing the residue of an approximate secant equation especially the new steplength can be directly applied into solving strictly convex quadratic function combined with a new nonmonotone line search strategy a new derivative free algorithm called a nonmonotone spectral residual algorithm nsra is developed under mild assumptions global convergence is established for locally lipschitz continuous nonlinear systems compared with the state of the art algorithms available in the literatures the new algorithm is more efficient in solving large scale benchmark test problems

eikonal equations on the sierpinski gasket we study the eikonal equation on the sierpinski gasket in the spirit of the construction of the laplacian in kigami analysis on fractals cambridge tracts in mathematics 2001 we consider graph eikonal equations on the prefractals and we show that the solutions of these problems converge to a function defined on the fractal set we characterize this limit function as the unique metric viscosity solution to the eikonal equation on the sierpinski gasket according to the definition introduced in giga et al trans am math soc 367 49 66 2015

a tool for the analysis of quasi newton methods with application to unconstrained minimization the bfgs update formula is shown to have an important property that is inde pendent of the algorithmic context of the update and that is relevant to both constrained and unconstrained optimization the bfgs method for unconstrained optimization using a variety of line searches including backtracking is shown to be globally and superlinearly convergent on uniformly convex problems the analysis is particularly simple due to the use of some new tools introduced in this paper

convergence of line search methods for unconstrained optimization line search methods are traditional and successful methods for solving unconstrained optimization problems its convergence has attracted more attention in recent years in this paper we analyze the general results on convergence of line search methods with seven line search rules it is clarified that the search direction plays a main role in these methods and that step size guarantees the global convergence in some cases it is also proved that many line search methods have same convergence property these convergence results can enable us to design powerful effective and stable algorithms in practice finally a class of special line search methods is investigated

accelerated solution of multivariate polynomial systems of equations we propose new las vegas randomized algorithms for the solution of a square nondegenerate system of equations with well separated roots the algorithms use oc delta csttn d 2 log d log b arithmetic operations in addition to the operations required to compute the normal form of the boundary monomials modulo the ideal to approximate all real roots of the system as well as all roots lying in a fixed n dimensional box or disc here d is an upper bound on the number of all complex roots of the system e g bezout or bernshtein bound delta is the number of real roots or the roots lying in the box or disc and epsilon 2 b is the required upper bound on the output errors for computing the normal form modulo the ideal the efficient practical algorithms of b mourrain and p trebuchet in proceedings of the international symposium on symbolic and algebraic computation acm new york 2000 pp 231 238 or j c faugere j pure appl algebra 139 1999 pp 61 88 can be applied we also yield the bound oc csttn d 2 log d on the complexity of counting the numbers of all roots in a fixed box disc and all real roots for a large class of inputs and typically in practical computations the factor delta is much smaller than d delta o d this improves by the order of magnitude the known complexity estimates of the order of at least 3n d4 d3 log b or d4 which so far are the record estimates even for the approximation of a single root of a system and for each of the cited counting problems respectively our progress relies on proposing several novel techniques in particular we exploit the structure of matrices associated to a given polynomial system and relate it to the associated linear operators dual space of linear forms and normal forms of polynomials in the quotient algebra furthermore our techniques support the new nontrivial extension of the matrix sign and quadratic inverse power iterations to the case of multivariate polynomial systems where we emulate the recursive splitting of a univariate polynomial into factors of smaller degree

avoiding false local minima by proper initialization of connections the training of neural net classifiers is often hampered by the occurrence of local minima which results in the attainment of inferior classification performance it has been shown that the occurrence of local minima in the criterion function is often related to specific patterns of defects in the classifier in particular three main causes for local minima were identified such an understanding of the physical correlates of local minima suggests sensible ways of choosing the weights from which the training process is initiated a method of initialization is introduced and shown to decrease the possibility of local minima occurring on various test problems

a neural root finder of polynomials based on root moments this letter proposes a novel neural root finder based on the root moment method rmm to find the arbitrary roots including complex ones of arbitrary polynomials this neural root finder nrf was designed based on feedforward neural networks fnn and trained with a constrained learning algorithm cla specifically we have incorporated the a priori information about the root moments of polynomials into the conventional backpropagation algorithm bpa to construct a new cla the resulting nrf is shown to be able to rapidly estimate the distributions of roots of polynomials we study and compare the advantage of the rmm based nrf over the previous root coefficient method based nrf and the traditional muller and laguerre methods as well as the mathematica roots function and the behaviors the accuracies of the resulting root finders and their training speeds of two specific structures corresponding to this fnn root finder the log and the fnn we also analyze the effects of the three controlling parameters p0 p with the cla on the two nrfs theoretically and experimentally finally we present computer simulation results to support our claims

a comparison among various notions of viscosity solution for hamilton jacobi equations on networks abstract three definitions of viscosity solutions for hamilton jacobi equations on networks recently appeared in literature achdou et al 2013 1 imbert et al 2013 4 schieborn and camilli 2013 6 being motivated by various applications they appear to be considerably different the aim of this note is to establish their equivalence

convergence of quasi newton method with new inexact line search quasi newton method is a well known effective method for solving optimization problems since it is a line search method which needs a line search procedure after determining a search direction at each iteration we must decide a line search rule to choose a step size along a search direction in this paper we propose a new inexact line search rule for quasi newton method and establish some global convergent results of this method these results are useful in designing new quasi newton methods moreover we analyze the convergence rate of quasi newton method with the new line search rule

the average distance on the sierpinski gasket the canonical distance of points on the sierpinski gasket is considered and its expectation deduced the solution is surprising both for the value and for the method derived from an analysis of graphs connected with the tower of hanoi problem

a gradient related algorithm with inexact line searches in this paper a new gradient related algorithm for solving large scale unconstrained optimization problems is proposed the new algorithm is a kind of line search method the basic idea is to choose a combination of the current gradient and some previous search directions as a new search direction and to find a step size by using various inexact line searches using more information at the current iterative step may improve the performance of the algorithm this motivates us to find some new gradient algorithms which may be more effective than standard conjugate gradient methods uniformly gradient related conception is useful and it can be used to analyze global convergence of the new algorithm the global convergence and linear convergence rate of the new algorithm are investigated under diverse weak conditions numerical experiments show that the new algorithm seems to converge more stably and is superior to other similar methods in many situations

geometry and analysis of dirichlet forms let e be a regular strongly local dirichlet form on l2 x m and d the associated intrinsic distance assume that the topology induced by d coincides with the original topology on x and that x is compact satisfies a doubling property and supports a weak 1 2 poincare inequality we first discuss the non coincidence of the intrinsic length structure and the gradient structure under the further assumption that the ricci curvature of x is bounded from below in the sense of lott sturm villani the following are shown to be equivalent i the heat flow of e gives the unique gradient flow of u ii e satisfies the newtonian property iii the intrinsic length structure coincides with the gradient structure moreover for the standard resistance dirichlet form on the sierpinski gasket equipped with the kusuoka measure we identify the intrinsic length structure with the measurable riemannian and the gradient structures we also apply the above results to the coarse ricci curvatures and asymptotics of the gradient of the heat kernel

monotone circuits for matching require linear depth it is proven that monotone circuits computing the perfect matching function on n vertex graphs require o n depth this implies an exponential gap between the depth of monotone and nonmonotone circuits

inductive definitions over finite structures inductive definitions have played a central role in the foundations of mathematics for over a century they were used in the 1970s as the backbone of one major generalization of recursive function theory moschovakis 1974 aczel 1977 in recent years the relevance of inductive definitions in particular over finite structures to database theory to descriptive computational complexity and to logics of programs has been recognized a seminal paper on inductive definitions in database theory is chandra and hare1 1982 where they define a hierarchy of queries over finite structures within which minor steps successor ordinals correspond to first order quantifier alternations and major steps limit ordinals correspond to uses of lixpoints they left open the question of whether the hierarchy remains strict above the first major step level w this problem was answered in the negative by immerman 1986 since the collection of first order lixpoint queries over finite structures is closed under composition and first order operations other than negation moschovakis 1974 the main component of immerman s solution was

the steepest descent algorithm without line search for p laplacian in this paper the steepest descent algorithm without line search is proposed for p laplacian its search direction is the weighted preconditioned steepest descent one and step length is estimated by a formula except the first iteration continuation method is applied for solving the p laplacian with very large p lots of numerical experiments are carried out on these algorithms all numerical results show the algorithm without line search can cut down some computational time fast convergence of these new algorithms is displayed by their step length figures these figures show that if search direction is the steepest descent one exact step lengths can be substituted properly with step lengths obtained by the formula

a hybrid learning algorithm combined with generalized rls approach for radial basis function neural networks in this paper a new hybrid learning method for radial basis function neural networks based on generalized recursive least square algorithm is proposed firstly the generalized recursive least square grls model including a general quadratic weight decay term in the energy function for the training of rbf neural networks is described then combined with the grls approach a new hybrid learning method is proposed to meet the design goals improving the generalization ability of the trained network finally experimental results demonstrate that our approach can achieve a significantly improved generalization performance of the rbf networks

on preservation under homomorphisms and unions of conjunctive queries unions of conjunctive queries also known as select project join union queries are the most frequently asked queries in relational database systems these queries are definable by existential positive first order formulas and are preserved under homomorphisms a classical result of mathematical logic asserts that the existential positive formulas are the only first order formulas up to logical equivalence that are preserved under homomorphisms on all structures finite and infinite the question of whether the homomorphism preservation theorem holds for the class of all finite structures resisted solution for a long time it was eventually shown that unlike other classical preservation theorems the homomorphism preservation theorem does hold in the finite in this article we show that the homomorphism preservation theorem holds also for several restricted classes of finite structures of interest in graph theory and database theory specifically we show that this result holds for all classes of finite structures of bounded degree all classes of finite structures of bounded treewidth and more generally all classes of finite structures whose cores exclude at least one minor

on learning monotone dnf under product distributions we show that the class of monotone 2o logn term dnf formulae can be pac learned in polynomial time under the uniform distribution from random examples only this is an exponential improvement over the best previous polynomial time algorithms in this model which could learn monotone o log2 n term dnf we also show that various classes of small constant depth circuits which compute monotone functions are pac learnable in polynomial time under the uniform distribution all of our results extend to learning under any constant bounded product distribution

approximation of boolean functions by sigmoidal networks part i xor and other two variable functions we prove the existence of a manifold of exact solutions mean square error e 0 of weights and thresholds for sigmoidal networks for xor and other 2 variable boolean functions we also prove the existence of a manifold of local minima of e where e 0

global convergence properties of conjugate gradient methods for optimization this paper explores the convergence of nonlinear conjugate gradient methods without restarts and with practical line searches the analysis covers two classes of methods that are globally convergent on smooth nonconvex functions some properties of the fletcher reeves method play an important role in the first family whereas the second family shares an important property with the polak ribiere method numerical experiments are presented

improved sign based learning algorithm derived by the composite nonlinear jacobi process in this paper a globally convergent first order training algorithm is proposed that uses sign based information of the batch error measure in the framework of the nonlinear jacobi process this approach allows us to equip the recently proposed jacobi rprop method with the global convergence property i e convergence to a local minimiser from any initial starting point we also propose a strategy that ensures the search direction of the globally convergent jacobi rprop is a descent one the behaviour of the algorithm is empirically investigated in eight benchmark problems simulation results verify that there are indeed improvements on the convergence success of the algorithm

a new unconstrained optimization method for imprecise function and gradient values a new algorithm for unconstrained optimization is presented which is based on a modified one dimensional bisection method the algorithm actually uses only the signs of function and gradient values thus it can be applied to problems with imprecise function and gradient values it converges in one iteration on quadratic functions of n variables it rapidly minimizes general functions and it does not require evaluation or estimation of the matrix of second partial derivatives the algorithm has been implemented and tested performance information for wellknown test functions is reported q 1996 academic press inc

non cancellative boolean circuits a generalization of monotone boolean circuits cancellations are known to be helpful in efficient algebraic computation of polynomials over fields we define a notion of cancellation in boolean circuits and define boolean circuits that do not use cancellation to be non cancellative non cancellative boolean circuits are a natural generalization of monotone boolean circuits we show that in the absence of cancellation boolean circuits require super polynomial size to compute the determinant interpreted over gf 2 this non monotone boolean function is known to be in p in the spirit of monotone complexity classes we define complexity classes based on non cancellative boolean circuits we show that when the boolean circuit model is restricted by withholding cancellation p and popular classes within p are restricted as well but classes np and above remain unchanged

absolutely minimizing lipschitz extensions and infinity harmonic functions on the sierpinski gasket abstract aim of this note is to study the infinity laplace operator and the corresponding absolutely minimizing lipschitz extension problem on the sierpinski gasket in the spirit of the classical construction of kigami for the laplacian we introduce a notion of infinity harmonic functions on pre fractal sets and we show that these functions solve a lipschitz extension problem in the discrete setting then we prove that the limit of the infinity harmonic functions on the pre fractal sets solves the absolutely minimizing lipschitz extension problem on the sierpinski gasket

local frequency features for texture classification abstract we propose a texture feature extraction method which allows very small training sets and due to its very local nature is suitable for classification of small textured structures the method is based on detection of local extremas along a set of direction vectors in the spatial image representation the local extremas are interpreted as identifying local frequencies and used to design features corresponding to the basal frequency characteristics amplitude and wavelength the good texture discrimination ability of these features is demonstrated on two images and in a quantitative comparison with features from gabor filtered images and co occurrence matrix features of the three images used in the quantitative comparison the local frequency features performed best on two of them we conclude that the new method is a promising feature extraction method for non stochastic textures and that it should be further explored

determining the number of real roots of polynomials through neural networks the ability of feedforward neural networks to identify the number of real roots of univariate polynomials is investigated furthermore their ability to determine whether a system of multivariate polynomial equations has real solutions is examined on a problem of determining the structure of a molecule the obtained experimental results indicate that neural networks are capable of performing this task with high accuracy even when the training set is very small compared to the test set

multivariate spectral gradient method for unconstrained optimization multivariate spectral gradient method is proposed for solving unconstrained optimization problems combined with some quasi newton property multivariate spectral gradient method allows an individual adaptive stepsize along each coordinate direction which guarantees that the method is finitely convergent for positive definite quadratics especially it converges no more than two steps for positive definite quadratics with diagonal hessian and quadratically for objective functions with positive definite diagonal hessian moreover based on a nonmonotone line search global convergence is established for multivariate spectral gradient algorithm at last numerical results are reported which show that this method is promising and deserves further discussing

a class of gradient unconstrained minimization algorithms with adaptive stepsize in this paper the development convergence theory and numerical testing of a class of gradient unconstrained minimization algorithms with adaptive stepsize are presented the proposed class comprises four algorithms the first two incorporate techniques for the adaptation of a common stepsize for all coordinate directions and the other two allow an individual adaptive stepsize along each coordinate direction all the algorithms are computationally efficient and possess interesting convergence properties utilizing estimates of the lipschitz constant that are obtained without additional function or gradient evaluations the algorithms have been implemented and tested on some well known test cases as well as on real life artificial neural network applications and the results have been very satisfactory

autoregressive time series prediction by means of fuzzy inference systems using nonparametric residual variance estimation we propose an automatic methodology framework for short and long term prediction of time series by means of fuzzy inference systems in this methodology fuzzy techniques and statistical techniques for nonparametric residual variance estimation are combined in order to build autoregressive predictive models implemented as fuzzy inference systems nonparametric residual variance estimation plays a key role in driving the identification and learning procedures concrete criteria and procedures within the proposed methodology framework are applied to a number of time series prediction problems the learn from examples method introduced by wang and mendel w m is used for identification the levenberg marquardt l m optimization method is then applied for tuning the w m method produces compact and potentially accurate inference systems when applied after a proper variable selection stage the l m method yields the best compromise between accuracy and interpretability of results among a set of alternatives delta test based residual variance estimations are used in order to select the best subset of inputs to the fuzzy inference systems as well as the number of linguistic labels for the inputs experiments on a diverse set of time series prediction benchmarks are compared against least squares support vector machines ls svm optimally pruned extreme learning machine op elm and k nn based autoregressors the advantages of the proposed methodology are shown in terms of linguistic interpretability generalization capability and computational cost furthermore fuzzy models are shown to be consistently more accurate for prediction in the case of time series coming from real world applications

zeroing polynomials using modified constrained neural network approach this paper proposes new modified constrained learning neural root finders nrfs of polynomial constructed by backpropagation network bpn the technique is based on the relationships between the roots and the coefficients of polynomial as well as between the root moments and the coefficients of the polynomial we investigated different resulting constrained learning algorithms clas based on the variants of the error cost functions ecfs in the constrained bpn and derived a new modified cla mcla and found that the computational complexities of the cla and the mcla based on the root moment method rmm are the order of polynomial and that the mcla is simpler than the cla further we also discussed the effects of the different parameters with the cla and the mcla on the nrfs in particular considering the coefficients of the polynomials involved in practice to possibly be perturbed by noisy sources thus we also evaluated and discussed the effects of noises on the two nrfs finally to demonstrate the advantage of our neural approaches over the nonneural ones a series of simulating experiments are conducted

improving the convergence of the backpropagation algorithm using learning rate adaptation methods this article focuses on gradient based backpropagation algorithms that use either a common adaptive learning rate for all weights or an individual adaptive learning rate for each weight and apply the goldstein armijo line search the learning rate adaptation is based on descent techniques and estimates of the local lipschitz constant that are obtained without additional error function and gradient evaluations the proposed algorithms improve the backpropagation training in terms of both convergence rate and convergence characteristics such as stable learning and robustness to oscillations simulations are conducted to compare and evaluate the convergence behavior of these gradient based training algorithms with several popular training methods

nonlinear successive over relaxation we study the convergence of gauss seidel and nonlinear successive overrelaxation methods for finding the minimum of a strictly convex functional defined onr n

a constructive approach for finding arbitrary roots of polynomials by neural networks this paper proposes a constructive approach for finding arbitrary real or complex roots of arbitrary real or complex polynomials by multilayer perceptron network mlpn using constrained learning algorithm cla which encodes the a priori information of constraint relations between root moments and coefficients of a polynomial into the usual bp algorithm bpa moreover the root moment method rmm is also simplified into a recursive version so that the computational complexity can be further decreased which leads the roots of those higher order polynomials to be readily found in addition an adaptive learning parameter with the cla is also proposed in this paper an initial weight selection method is also given finally several experimental results show that our proposed neural connectionism approaches with respect to the nonneural ones are more efficient and feasible in finding the arbitrary roots of arbitrary polynomials

convergence of nonmonotone line search method in this paper we develop a new nonmonotone line search for general line search method and establish some global convergence theorems the new nonmonotone line search is a novel form of the nonmonotone armijo line search and allows one to choose a larger step size at each iteration which is available in constructing new line search methods and possibly reduces the function evaluations at each iteration moreover we analyze the convergence rate of some special line search methods with the new line search preliminary numerical results show that some line search methods with the new nonmonotone line search are available and efficient in practical computation

a new super memory gradient method with curve search rule in this paper we propose a new super memory gradient method with curve search rule for unconstrained optimization problems the method uses previous multi step iterative information and some curve search rules to generate new iterative points at each iteration this makes the new method converge stably and be more suitable for solving large scale optimization problems than other similar methods we analyze the global convergence and convergence rate under some mild conditions numerical experiments show that some new algorithms are available and effective in practical computation

homomorphism preservation theorems the homomorphism preservation theorem h p t a result in classical model theory states that a first order formula is preserved under homomorphisms on all structures finite and infinite if and only if it is equivalent to an existential positive formula answering a long standing question in finite model theory we prove that the h p t remains valid when restricted to finite structures unlike many other classical preservation theorems including the o tarski theorem and lyndon s positivity theorem applications of this result extend to constraint satisfaction problems and to database theory via a correspondence between existential positive formulas and unions of conjunctive queries a further result of this article strengthens the classical h p t we show that a first order formula is preserved under homomorphisms on all structures if and only if it is equivalent to an existential positive formula of equal quantifier rank

an algebraic approach to the estimation of the order of fir filters from complete and partial magnitude and phase specifications the problems addressed by this paper is the following given a set of measurements over the range of normalized frequencies thetas1 thetas2 on the magnitude and or phase of a real fir but otherwise unknown filter to estimate the order of the fir filter the range thetas1 thetas2 may be partial or it may cover the entire range of frequencies the purpose of the paper is to propose a new algebraic approach to solve the above collection of problems specific new results include fir order estimation from partial or complete noiseless measurements for a real system from a phase alone from b magnitude alone not necessarily piece wise constant and from c joint magnitude and phase the proposed approach is not only capable of dealing with specifications that go beyond the conventional formulas for the standard piece wise constant magnitude fir filter order estimation but it also furnishes a nexus for order estimation from phase or group delay specifications areas which have remained hitherto unaddressed the approach is based on the use of root moments in this context the novel concept of fractional root moments is used in a key fashion to provide an estimate on the number of zeros inside the unit circle open problems and new directions of exploration and research are mentioned in the body of the paper

global optimization through a stochastic perturbation of the polak ribiere conjugate gradient method abstract we develop a new modified polak ribiere conjugate gradient method by considering a random perturbation our approach is suitable for solving a large class of optimization problems on a rectangle of r n or unconstrained problems theoretical results ensure that the proposed method converges to a global minimizer numerical experiments are achieved on some typical test problems particularly the engineering problem of lennard jones clusters a comparison with well known methods is carried out to show the performance of our algorithm

finding the zeros of linear phase fir frequency sampling digital filters the need for finding the zeros of fir filters often arises in digital signal processing applications if we want to convert linear phase finite impulse response fir filters into minimum phase fir filters 1 2 the problem of finding the zeros of the filter is encountered this problem has been addressed in 3 but not solved here we introduce the idea of using frequency sampling filters for obtaining its zeros the advantage of using this kind of filter resides in the fact that all the zeros on the unit circle are known furthermore the zeros off the unit circle can be obtained with some simple processing this type of filter is especially useful when high order narrow band filters are involved finding the zeros of very high order equirriple low pass filters can be a difficult if not impossible task

a dimension reducing method for unconstrained optimization abstract a new method for unconstrained optimization in r n is presented this method reduces the dimension of the problem in such a way that it can lead to an iterative approximate formula for the computation of n 1 components of the optimum while its remaining component is computed separately using the final approximations of the other components it converges quadratically to a local optimum and it requires storage of order n 1 n 1 besides it does not require a good initial guess for one component of the optimum and it does not directly perform gradient evaluations thus it can be applied to problems with imprecise gradient values moreover a procedure for transforming the matrix formed by our method into a symmetric as well as into a diagonal one is presented furthermore the proposed dimension reducing scheme using finite difference gradient and hessian is presented the methods have been implemented and tested performance information for well known test functions is reported

bounding the search space for global optimization of neural networks learning error an interval analysis approach training a multilayer perceptron mlp with algorithms employing global search strategies has been an important research direction in the field of neural networks despite a number of signifcant results an important matter concerning the bounds of the search region typically defined as a box where a global optimization method has to search for a potential global minimizer seems to be unresolved the approach presented in this paper builds on interval analysis and attempts to define guaranteed bounds in the search space prior to applying a global search algorithm for training an mlp these bounds depend on the machine precision and the term guaranteed denotes that the region defined surely encloses weight sets that are global minimizers of the neural network s error function although the solution set to the bounding problem for an mlp is in general non convex the paper presents the theoretical results that help deriving a box which is a convex set this box is an outer approximation of the algebraic solutions to the interval equations resulting from the function implemented by the network nodes an experimental study using well known benchmarks is presented in accordance with the theoretical results

positive versions of polynomial time abstract we show that restricting a number of characterizations of the complexity class p to be positive in natural ways results in the same class of monotone problems which we denote by posp by a well known result of razborov posp is a proper subclass of the class of monotone problems in p we exhibit complete problems for posp via weak logical reductions as we do for other logically defined classes of problems our work is a continuation of research undertaken by grigni and sipser and subsequently stewart indeed we introduce the notion of a positive deterministic turing machine and consequently solve a problem posed by grigni and sipser

a new descent algorithm with curve search rule in the paper we present a new descent algorithm with curve search rule for unconstrained minimization problems at each iteration the next iterative point is determined by means of a curve search rule it is particular that the search direction and the step size is determined simultaneously at each iteration of the new algorithm the algorithm similarly to conjugate gradient methods avoids the computation and storage of some matrices associated with the hessian of objective functions unlike ode methods it is not required to solve ordinary differential equations at each iteration although the convergence rate of our algorithm is not as fast as that of newton like methods it is suitable to solve large scale minimization problems

monotone versus positive in connection with the least fixed point operator the following question was raised suppose that a first order formula p p is semantically monotone in a predicate symbol p on finite structures is p p necessarily equivalent on finite structures to a first order formula with only positive occurrences of p in this paper this question is answered negatively moreover the counterexample naturally gives a uniform sequence of constant depth polynomial size monotone boolean circuits that is not equivalent to any however nonuniform sequence of constant depth polynomial size positive boolean circuits

nonextensive statistical mechanics for hybrid learning of neural networks abstract this paper introduces a new hybrid approach for learning systems that builds on the theory of nonextensive statistical mechanics the proposed learning scheme uses only the sign of the gradient and combines adaptive stepsize local searches with global search steps that make use of an annealing schedule inspired from nonextensive statistics as proposed by tsallis the performance of the hybrid approach is empirically investigated through simulation in benchmark problems from the uci repository of machine learning databases preliminary results provide evidence that the synergy of techniques from nonextensive statistics provide neural learning schemes with significant benefits in terms of learning speed and convergence success

convergence of descent method without line search line search method is sometimes a useful and efficient technique for solving unconstrained optimization problems especially for solving small and middle scale problems however a line search procedure is necessary at each iteration leading to a significant amount of computation in order to reduce the evaluations of objective functions and gradients the line search procedure should be avoided in algorithm design in this paper we propose a new descent method without line search for unconstrained optimization problems the algorithm is required to estimate some parameters at each iteration we analyze theoretically the global convergence of the new algorithm under mild conditions these theoretical conclusions can help us in designing new efficient methods for optimization problems

optac a portable software package for analyzing and comparing optimization methods by visualization abstract a software package for analyzing and comparing optimization methods is presented this package displays using different colors the regions of convergence to the minima of a given function for various optimization methods it displays also the rate of their convergence as well as the regions of divergence of these methods moreover this package gives quantitative information regarding the total convergence area in a specific domain for various minima using optac optimization analysis and comparisons we are able to see in a picture the advantages and disadvantages of any optimization method as well as to compare various methods in order to choose the proper method for a given class of problems the optac package is self contained and conforms to the ansi 1977 fortran standards