non convergence of formal integrals of motion we consider formal integrals of motion in 2d hamiltonian dynamical systems calculated with the normal form method of giorgilli 1979 comput phys commun 16 331 three different non integrable and one integrable systems are considered the time variation di of the formal integral i is found as a function of the order of truncation n of the integral series an optimal order of truncation is found from the minima of the variations di the level lines of the integral i representing theoretical invariant curves on a poincare surface of section are compared with the real invariant curves when chaos is limited excellent agreement is found between the theoretical and the real invariant curves if the order of truncation is close to the optimal order the agreement is poor a far from the optimal order and b when chaos is pronounced the optimal order calculated as a function of the distance r from the origin decreases when r increases the decrease is rather smooth in the 1 1 resonance but it has abrupt steps in the case of a higher order 4 3 resonance in the case of an integrable hamiltonian a formal integral if is found which is a function of the exact integral i and of the hamiltonian given as power series of the canonical variables the series converges only within a domain of convergence the radius of convergence along a particular direction is calculated with the d alembert and cauchy methods the theoretical invariant curves agree with the real invariant curves only within the domain of convergence of if in the case of non integrable hamiltonians we calculate pseudo radii of convergence that tend to zero as the order of truncation n increases

storage of sets of correlated data in neural network memories in most applications of neural network models one has to consider sets of correlated data the authors study the problem of the storage of associated patterns in neural network memories they consider two basic types of correlation semantic ones among the different patterns and spatial ones among the different sites of the network they apply gardner s program to evaluate optimal storage conditions for both kinds of correlation the spatial ones worsen generally speaking the storage properties of a simple perceptron while they improve them for the hopfield network in the case of semantic correlations they obtain bounds of the critical capacity for both kinds of networks the storage ratio of the perceptron may be significantly increased in this case

critical storage capacity of the j 1 neural network for neural networks in which the coupling jij are allowed to take on the values jij 1 or jij 1 the authors determine numerically the critical storage capacity for random unbiased patterns as a function of the stability they use an exact enumeration scheme based on the gray code and a continuous distribution for the patterns to control finite size effects results are presented for n 25 they indicate an optimal storage capacity of c 0 82 n

periodic orbits and non integrability of henon heiles systems we apply the averaging theory of second order to study the periodic orbits for a generalized henon heiles system with two parameters which contains the classical henon heiles system two main results are shown the first result provides sufficient conditions on the two parameters of these generalized systems which guarantee that at any positive energy level the hamiltonian system has periodic orbits these periodic orbits form in the whole phase space a continuous family of periodic orbits parameterized by the energy the second result shows that for the non integrable henon heiles systems in the sense of liouville arnol d which have the periodic orbits analytically found with averaging theory cannot exist any second first integral of class in particular for any second first integral of class we prove that the classical henon heiles system and many generalizations of it are not integrable in the sense of liouville arnol d moreover the tools we use for studying the periodic orbits and the non liouville arnol d integrability can be applied to hamiltonian systems with an arbitrary number of degrees of freedom

rigorous proof of termination of smo algorithm for support vector machines sequential minimal optimization smo algorithm is one of the simplest decomposition methods for learning of support vector machines svms keerthi and gilbert have recently studied the convergence property of smo algorithm and given a proof that smo algorithm always stops within a finite number of iterations in this letter we point out the incompleteness of their proof and give a more rigorous proof

properties of neural networks storing spatially correlated patterns the author studies the behaviour of a feedforward neural network supplied with spatially organized data this inner structure is taken into account by a matrix cij whose coefficients equal the average correlation between two pixels i and j of the input patterns the storage capacity alpha is computed as a function of the required stability and of the eigenvalues of c the author proposes a geometrical transformation allowing an intuitive interpretation of these results numerical simulations using real and binary patterns show a very good agreement with the theory finally the author analyses the synaptic couplings correlations resulting from the training of the network with structured patterns focusing on exponentially decreasing correlations one and two dimensions the author finds that they exhibit a mexican hat profile the excitatory centre size of which depends on alpha

a new class of integrable systems we present a family of dynamical systems associated with the motion of a particle in two space dimensions these systems possess a second integral of motion quadratic in velocities apart from the hamiltonian and are thus completely integrable they were found through the derivation and subsequent resolution of the integrability condition in the form of a partial differential equation pde for the potential a most important point is that the same pde was derived through considerations on the analytic structure of the singularities of the solutions weak painleve property

content addressability and learning in neural networks the content addressability of patterns stored in ising spin neural network models with symmetric interactions is studied numerical results from simulations on the icl distributed array processor dap involving systems with up to 2048 neurons are presented behaviour consistent with finite size scaling characteristic of a first order phase transition is shown to be exhibited by the basins of attraction of the stored patterns both in the case of the hopfield model and for systems using a local iterative learning algorithm designed to optimise the basins of attraction estimates are obtained for the critical minimum overlaps which an input pattern must have with a stored pattern in order to successfully retrieve it

improvements to platt s smo algorithm for svm classifier design this article points out an important source of inefficiency in platt s sequential minimal optimization smo algorithm that is caused by the use of a single threshold value using clues from the kkt conditions for the dual problem two threshold parameters are employed to derive modifications of smo these modified algorithms perform significantly faster than the original smo on all benchmark data sets tried

finite size effects and bounds for perceptron models ablncl in lhis paper we consider two main aspsln of the binary perccptron problem the maximal capacity when random paltem are stored model a and its generalhation ability model b we have extended previous numerical estimates of critical capacilies and studied thermal properties of systems of small sizes to lest recent replica predictions we have also considered some simpler versions of thest models the discrete spherical vemions can be salved exactly using gardner s replica calculation for the spherical model and are shown to give a rigorous upper bound and lower bound on the capacities of models a and b respectively iby vemions of models a and b are soivcd in detail and provide information which is useful for interpreting the bnile size eeeciects present in the numerical studies of models a and b

integrability of hamiltonians with third and fourth degree polynomial potentials the weak painleve property as a criterion of integrability is applied to the case of simple hamiltonians describing the motion of a particle in two dimensional polynomial potentials of degree three and four this allows a complete identification of all the integrable cases of cubic potentials in the case of quartic potentials although our results are not exhaustive some new integrable cases are discovered in both cases the integrability is explicited by a direct calculation of the second integral of motion of the system

information and topology in attractor neural networks a wide range of networks including those with small world topology can be modeled by the connectivity ratio and randomness of the links both learning and attractor abilities of a neural network can be measured by the mutual information mi as a function of the load and the overlap between patterns and retrieval states in this letter we use mi to search for the optimal topology with regard to the storage and attractor properties of the network in an amari hopfield model we find that while an optimal storage implies an extremely diluted topology a large basin of attraction leads to moderate levels of connectivity this optimal topology is related to the clustering and path length of the network we also build a diagram for the dynamical phases with random or local initial overlap and show that very diluted networks lose their attractor ability

maximum gain working set selection for svms support vector machines are trained by solving constrained quadratic optimization problems this is usually done with an iterative decomposition algorithm operating on a small working set of variables in every iteration the training time strongly depends on the selection of these variables we propose the maximum gain working set selection algorithm for large scale quadratic programming it is based on the idea to greedily maximize the progress in each single iteration the algorithm takes second order information from cached kernel matrix entries into account we prove the convergence to an optimal solution of a variant termed hybrid maximum gain working set selection this method is empirically compared to the prominent most violating pair selection and the latest algorithm using second order information for large training sets our new selection scheme is significantly faster

periodic orbits and nonintegrability of generalized classical yang mills hamiltonian systems the averaging theory of first order is applied to study a generalized yang mills system with two parameters two main results are proved first we provide sufficient conditions on the two parameters of the generalized system to guarantee the existence of continuous families of isolated periodic orbits parameterized by the energy and these families are given up to first order in a small parameter second we prove that for the nonintegrable classical yang mills hamiltonian systems in the sense of liouville arnold which have the isolated periodic orbits found with averaging theory cannot exist in any second first integral of class c1 this is important because most of the results about integrability deals with analytic or meromorphic integrals of motion

simple proof of convergence of the smo algorithm for different svm variants in this brief we give a new proof of the asymptotic convergence of the sequential minimum optimization smo algorithm for both the most violating pair and second order rules to select the pair of coefficients to be updated the proof is more self contained shorter and simpler than previous ones and has a different flavor partially building upon gilbert s original convergence proof of its algorithm to solve the minimum norm problem for convex hulls it is valid for both support vector classification svc and support vector regression which are formulated under a general problem that encompasses them moreover this general problem can be further extended to also cover other support vector machines svm related problems such as svc or one class svms while the convergence proof of the slight variant of smo needed for them remains basically unchanged

global convergence of decomposition learning methods for support vector machines decomposition methods are well known techniques for solving quadratic programming qp problems arising in support vector machines svms in each iteration of a decomposition method a small number of variables are selected and a qp problem with only the selected variables is solved since large matrix computations are not required decomposition methods are applicable to large qp problems in this paper we will make a rigorous analysis of the global convergence of general decomposition methods for svms we first introduce a relaxed version of the optimality condition for the qp problems and then prove that a decomposition method reaches a solution satisfying this relaxed optimality condition within a finite number of iterations under a very mild condition on how to select variables

towards a theory of early visual processing we propose a theory of the early processing in the mammalian visual pathway the theory is formulated in the language of information theory and hypothesizes that the goal of this processing is to recode in order to reduce a generalized redundancy subject to a constraint that specifies the amount of average information preserved in the limit of no noise this theory becomes equivalent to barlow s redundancy reduction hypothesis but it leads to very different computational strategies when noise is present a tractable approach for finding the optimal encoding is to solve the problem in successive stages where at each stage the optimization is performed within a restricted class of transfer functions we explicitly find the solution for the class of encodings to which the parvocellular retinal processing belongs namely linear and nondivergent transformations the solution shows agreement with the experimentally observed transfer functions at all levels of signal to noise

on the capacity of neural networks with binary weights the authors study the critical capacity alpha c of multilayered networks with binary couplings they show that for any network presenting a tree like architecture after the first hidden layer no fixed internal representation is required using gardner s calculations they apply statistical mechanics to the simplest network with two layers of adaptive weights following the same approach as for the binary perceptron they find from the zero entropy point a critical capacity alpha c 0 92 they discuss the validity of this result in view of exhaustive search simulations on small networks

storage of correlated patterns in a perceptron we calculate the storage capacity of a perceptron for correlated gaussian patterns we find that the storage capacity alpha c can be less than 2 if similar patterns are mapped onto different outputs and vice versa as long as the patterns are in a general position we obtain in contrast to previous works that alpha c or 1 in agreement with cover s theorem numerical simulations confirm the results

optimal storage properties of neural network models the authors calculate the number p alpha n of random n bit patterns that an optimal neural network can store allowing a given fraction f of bit errors and with the condition that each right bit is stabilised by a local field at least equal to a parameter k for each value of alpha and k there is a minimum fraction fmin of wrong bits they find a critical line alpha c k with alpha c 0 2 the minimum fraction of wrong bits vanishes for alpha alpha c k the calculations are done using a saddle point method and the order parameters at the saddle point are assumed to be replica symmetric this solution is locally stable in a finite region of the k alpha plane including the line alpha c k but there is a line above which the solution becomes unstable and replica symmetry must be broken

learning algorithms with optimal stability in neural networks to ensure large basins of attraction in spin glass like neural networks of two state elements xi imu or 1 the authors propose to study learning rules with optimal stability delta where delta is the largest number satisfying delta or sigma j jij xi jmu xi imu mu 1 p i 1 n where n is the number of neurons and p is the number of patterns they motivate this proposal and provide optimal stability learning rules for two different choices of normalisation for the synaptic matrix jij in addition numerical work is presented which gives the value of the optimal stability for random uncorrelated patterns

the space of interactions in neural network models the typical fraction of the space of interactions between each pair of n ising spins which solve the problem of storing a given set of p random patterns as n bit spin configurations is considered the volume is calculated explicitly as a function of the storage ratio alpha p n of the value kappa 0 of the product of the spin and the magnetic field at each site and of the magnetisation m here m may vary between 0 no correlation and 1 completely correlated the capacity increases with the correlation between patterns from alpha 2 for correlated patterns with kappa 0 and tends to infinity as m tends to 1 the calculations use a saddle point method and the order parameters at the saddle point are assumed to be replica symmetric this solution is shown to be locally stable a local iterative learning algorithm for updating the interactions is given which will converge to a solution of given kappa provided such solutions exist

fast support vector data descriptions for novelty detection support vector data description svdd has become a very attractive kernel method due to its good results in many novelty detection problems however the decision function of svdd is expressed in terms of the kernel expansion which results in a run time complexity linear in the number of support vectors for applications where fast real time response is needed how to speed up the decision function is crucial this paper aims at dealing with the issue of reducing the testing time complexity of svdd a method called fast svdd f svdd is proposed unlike the traditional methods which all try to compress a kernel expansion into one with fewer terms the proposed f svdd directly finds the preimage of a feature vector and then uses a simple relationship between this feature vector and the svdd sphere center to re express the center with a single vector the decision function of f svdd contains only one kernel term and thus the decision boundary of f svdd is only spherical in the original space hence the run time complexity of the f svdd decision function is no longer linear in the support vectors but is a constant no matter how large the training set size is in this paper we also propose a novel direct preimage finding method which is noniterative and involves no free parameters the unique preimage can be obtained in real time by the proposed direct method without taking trial and error for demonstration several real world data sets and a large scale data set the extended mit face data set are used in experiments in addition a practical industry example regarding liquid crystal display micro defect inspection is also used to compare the applicability of svdd and our proposed f svdd when faced with mass data input the results are very encouraging

generalized core vector machines kernel methods such as the support vector machine svm are often formulated as quadratic programming qp problems however given m training patterns a naive implementation of the qp solver takes o m 3 training time and at least o m2 space hence scaling up these qps is a major stumbling block in applying kernel methods on very large data sets and a replacement of the naive method for finding the qp solutions is highly desirable recently by using approximation algorithms for the minimum enclosing ball meb problem we proposed the core vector machine cvm algorithm that is much faster and can handle much larger data sets than existing svm implementations however the cvm can only be used with certain kernel functions and kernel methods for example the very popular support vector regression svr cannot be used with the cvm in this paper we introduce the center constrained meb problem and subsequently extend the cvm algorithm the generalized cvm algorithm can now be used with any linear nonlinear kernel and can also be applied to kernel methods such as svr and the ranking svm moreover like the original cvm its asymptotic time complexity is again linear in m and its space complexity is independent of m experiments show that the generalized cvm has comparable performance with state of the art svm and svr implementations but is faster and produces fewer support vectors on very large data sets

learning drifting concepts with neural networks the learning of time dependent concepts with a neural network is studied analytically and numerically the linearly separable target rule is represented by an n vector whose time dependence is modelled by a random or deterministic drift process a single layer network is trained online using different hebb like algorithms training is based on examples which are chosen randomly and according to a query strategy the evolution of the generalization error can be calculated exactly in the thermodynamic limit n to infinity the rule is never learnt perfectly but can be tracked within a certain error margin the generalization performance of various learning rules is compared and simulations confirm the analytic results

structured patterns retrieval using a metric attractor network application to fingerprint recognition abstract the ability of a metric attractor neural networks mann to learn structured patterns is analyzed in particular we consider collections of fingerprints which present some local features rather than being modeled by random patterns the network retrieval proved to be robust to varying the pattern activity the threshold strategy the topological arrangement of the connections and for several types of noisy configuration we found that the lower the fingerprint patterns activity is the higher the load ratio and retrieval quality are a simplified theoretical framework for the unbiased case is developed as a function of five parameters the load ratio the finiteness connectivity the density degree of the network randomness ratio and the spatial pattern correlation linked to the latter appears a new neural dynamics variable the spatial neural correlation the theory agrees quite well with the experimental results

capacity of neural networks with discrete synaptic couplings the authors study the optimal storage capacity of neural networks with discrete local constraints on the synaptic couplings jij models with such constraints include those with binary couplings jij or 1 or jij 0 1 quantised couplings with larger synaptic range e g jij or 1 l or 2 l or 1 and in the limit continuous couplings confined to the hypercube mod jij mod or 1 box confinement they find that the optimal storage capacity alpha kappa is best determined by the vanishing of a suitably defined entropy as calculated in the replica symmetric approximation they also extend their results to cases with biased memories and make contact with sparse coding models

painleve conjecture revisited the discovery of new integrable two dimensional hamiltonian systems is reported the analytic structure of the solutions makes necessary the generalization of the painlev e conjecture a widely used integrability criterion such a generalization is presented which the authors believe should replace the usual conjecture for two dimensional hamiltonian systems it is indeed compatible with all the systems already found and in addition leads to still new integrable cases

a formal analysis of stopping criteria of decomposition methods for support vector machines in a previous paper the author 2001 proved the convergence of a commonly used decomposition method for support vector machines svms however there is no theoretical justification about its stopping criterion which is based on the gap of the violation of the optimality condition it is essential to have the gap asymptotically approach zero so we are sure that existing implementations stop in a finite number of iterations after reaching a specified tolerance here we prove this result and illustrate it by two extensions spl nu svm and a multiclass svm by crammer and singer 2001 a further result shows that in final iterations of the decomposition method only a particular set of variables are still being modified this supports the use of the shrinking and caching techniques in some existing implementations finally we prove the asymptotic convergence of a decomposition method for this multiclass svm discussions on the difference between this convergence proof and the one in another paper by lin are also included

asymptotic convergence of an smo algorithm without any assumptions the asymptotic convergence of c j lin 2001 can be applied to a modified smo sequential minimal optimization algorithm by s s keerthi et al 2001 with some assumptions the author shows that for this algorithm those assumptions are not necessary

on the periodic orbits and the integrability of the regularized hill lunar problem the classical hill s problem is a simplified version of the restricted three body problem where the distance of the two massive bodies say primary for the largest one and secondary for the smallest one is made infinity through the use of hill s variables the levi civita regularization takes the hamiltonian of the hill lunar problem into the form of two uncoupled harmonic oscillators perturbed by the coriolis force and the sun action polynomials of degree 4 and 6 respectively in this paper we study periodic orbits of the planar hill problem using the averaging theory moreover we provide information about the c1 integrability or non integrability of the regularized hill lunar problem