critical behavior of a kam surface i empirical results kolmogorov arnol d moser kam surfaces are studied in the context of a perturbed two dimensional twist map in particular we ask how a kam surface can disappear as the perturbation parameter is increased following greene we use cycles to numerically construct the kam curve and discover that at the critical coupling it shows structure at all length scales aspects of this structure are fitted by a scaling analysis critical indices and scaling functions are determined numerically some evidence is presented which suggests that the results are universal

multidimensional density shaping by sigmoids an estimate of the probability density function of a random vector is obtained by maximizing the output entropy of a feedforward network of sigmoidal units with respect to the input weights classification problems can be solved by selecting the class associated with the maximal estimated density newton s optimization method applied to the estimated density yields a recursive estimator for a random variable or a random sequence a constrained connectivity structure yields a linear estimator which is particularly suitable for real time prediction a gaussian nonlinearity yields a closed form solution for the network s parameters which may also be used for initializing the optimization algorithm when other nonlinearities are employed a triangular connectivity between the neurons and the input which is naturally suggested by the statistical setting reduces the number of parameters applications to classification and forecasting problems are demonstrated

what is the goal of sensory coding a number of recent attempts have been made to describe early sensory coding in terms of a general information processing strategy in this paper two strategies are contrasted both strategies take advantage of the redundancy in the environment to produce more effective representations the first is described as a compact coding scheme a compact code performs a transform that allows the input to be represented with a reduced number of vectors cells with minimal rms error this approach has recently become popular in the neural network literature and is related to a process called principal components analysis pca a number of recent papers have suggested that the optimal compact code for representing natural scenes will have units with receptive field profiles much like those found in the retina and primary visual cortex however in this paper it is proposed that compact coding schemes are insufficient to account for the receptive field properties of cells in the mammalian visual pathway in contrast it is proposed that the visual system is near to optimal in representing natural scenes only if optimality is defined in terms of sparse distributed coding in a sparse distributed code all cells in the code have an equal response probability across the class of images but have a low response probability for any single image in such a code the dimensionality is not reduced rather the redundancy of the input is transformed into the redundancy of the firing pattern of cells it is proposed that the signature for a sparse code is found in the fourth moment of the response distribution i e the kurtosis in measurements with 55 calibrated natural scenes the kurtosis was found to peak when the bandwidths of the visual code matched those of cells in the mammalian visual cortex codes resembling wavelet transforms are proposed to be effective because the response histograms of such codes are sparse i e show high kurtosis when presented with natural scenes it is proposed that the structure of the image that allows sparse coding is found in the phase spectrum of the image it is suggested that natural scenes to a first approximation can be considered as a sum of self similar local functions the inverse of a wavelet possible reasons for why sensory systems would evolve toward sparse coding are presented

universal transition in a dynamical system forced at two incommensurate frequencies abstract the various routes to chaos are explored for a nonlinear mechanical system mode locked to one of two incommensurate external frequencies two types of transitions are seen in our model system a saddle node transition with its associated intermittency can occur when the mode locking is lost while the attracting two torus remains smooth a less trivial transition can occur in which the attracting torus roughens and the power spectrum of the time series develops singular low frequency components after the breakdown mode locking persists with noisy small scale motions about the former torus at the multicritical point where these two transition lines meet scaling and universal low frequency power spectra are observed a renormalization group treatment is proposed the analysis might also be applicable to transitions in rotationally invariant systems

information maximization and independent component analysis is there a difference this article provides a detailed and rigorous analysis of the two commonly used methods for redundancy reduction linear independent component analysis ica posed as a direct minimization of a suitably chosen redundancy measure and information maximization infomax of a continuous stochastic signal transmitted through an appropriate nonlinear network the article shows analytically that ica based on the kullbackleibler information as a redundancy measure and infomax lead to the same solution if the parameterization of the output nonlinear functions in the latter method is sufficiently rich furthermore this work discusses the alternative redundancy measures not based on the kullback leibler information distance the practical issues of applying ica and infomax are also discussed and illustrated on the problem of extracting statistically independent factors from a linear pixel by pixel mixture of images

on lower subdifferentiable functions this paper studies the notion of lower subdifferentiability from the view point of generalized conjugation by this method the main properties of lower sub differentiable functions are analysed a related conjugation theory for holder and lipschitz functions is developed which is used to characterize those functions which can be expressed as a supremum of holder functions some applications to quasiconvex optimization and optimal control theory are examined

the rotation number for almost periodic potentials we define and analyze the rotation number for the almost periodic schrodinger operatorl d2 dx2 q x we use the rotation number to discuss i the spectrum ofl ii its relation to the korteweg de vries equation

a fast fixed point algorithm for independent component analysis we introduce a novel fast algorithm for independent component analysis which can be used for blind source separation and feature extraction we show how a neural network learning rule can be transformed into a fixedpoint iteration which provides an algorithm that is very simple does not depend on any user defined parameters and is fast to converge to the most accurate solution allowed by the data the algorithm finds one at a time all nongaussian independent components regardless of their probability distributions the computations can be performed in either batch mode or a semiadaptive manner the convergence of the algorithm is rigorously proved and the convergence speed is shown to be cubic some comparisons to gradient based algorithms are made showing that the new algorithm is usually 10 to 100 times faster sometimes giving the solution in just a few iterations

renormalization method for computing the threshold of the large scale stochastic instability in two degrees of freedom hamiltonian systems an approximate renormalization procedure is derived for the hamiltonianh v x t v2 2 m cosx p cosk x t it gives an estimate of the large scale stochastic instability threshold which agrees within 5 10 with the results obtained from direct numerical integration of the canonical equations it shows that this instability is related to the destruction of kam tori between the two resonances and makes the connection with kam theory possible improvements of the method are proposed the results obtained forh allow us to estimate the threshold for a large class of hamiltonian systems with two degrees of freedom

on a generalized bellman equation for the optimal time problem abstract in this paper an extension of some theorems of hajek 1976 is given using the notion of clarke s generalized gradient moreover a feedback law for optimal controls and bellman s equation are obtained

criteria for multichannel signal separation we consider the problem in which we want to separate two or more signals that are coupled to each other through an unknown multiple input multiple output linear system channel we prove that the signals can be decoupled or separated using only the condition that they are statistically independent and find even weaker sufficient conditions involving their cross polyspectra by imposing these conditions on the reconstructed signals we obtain a class of criteria for signal separation these criteria are universal in the sense that they do not require any prior knowledge or information concerning the nature of the source signals they may be communication signals or speech signals or any other 1 d or multidimensional signals e g images computationally efficient algorithms for implementing the proposed criteria that only involve the iterative solution to a linear least squares problem are presented

three coupled oscillators mode locking global bifurcations and toroidal chaos abstract many systems exhibit behaviour which can be described by three coupled oscillators provided the coupling is not too strong such systems can be modelled by maps of the two torus to itself in this paper we describe and explain those aspects of the bifurcation diagram for two parameter families of torus maps that involve change of mode locking type we introduce the concepts of partial and full mode locking for the coupled oscillators these notions correspond to the presence of one or two rational relations between the frequencies respectively numerical investigation of a particular family of torus maps reveals an intricate web of global bifurcations in order to explain the results we first show that we can approximate maps in the neighbourhood of a rational translation to arbitrary order by the time 1 map of a flow on the torus then we analyse those condimension 1 and 2 bifurcations for flows on the torus which change the set of frequency ratios we find a large variety of bifurcation diagrams in particular many involving homotopically non trivial saddle connections next we ask how the picture changes when the time 1 maps of a family of flows are perturbed to a general family of diffeomorphisms we find toroidal chaos in the neighbourhood of all these bifurcations meaning that there exist orbits which perform a pseudo random sequence of rotations in different directions around the torus the corresponding behaviour for the coupled oscillators is that the frequency ratios perform a random walk finally we show how all these ingredients can be put together to give global scenarios for bifurcation for families of torus maps which we believe to be of general applicability to physical systems with three weakly coupled modes of oscillation

semigroups of relations and subdifferentials of the minimal time function and of the geodesic distance given a closed subset s of a banach space x we study the minimal time function to reach a point x of x starting from s we relate this problem to the study of the geodesic distance from x to s associated with a riemannian metric or a finsler metric it appears that both cases can be treated simultaneously and yield solutions to a hamilton jacobi equation we detect simple links between the general framework we propose and multivalued semigroups new concepts of infinitesimal generators of such semigroups are considered

statistical independence and novelty detection with information preserving nonlinear maps according to barlow 1989 feature extraction can be understood as finding a statistically independent representation of the probability distribution underlying the measured signals the search for a statistically independent representation can be formulated by the criterion of minimal mutual information which reduces to decorrelation in the case of gaussian distributions if nongaussian distributions are to be considered minimal mutual information is the appropriate generalization of decorrelation as used in linear principal component analyses pca we also generalize to nonlinear transformations by only demanding perfect transmission of information this leads to a general class of nonlinear transformations namely symplectic maps conservation of information allows us to consider only the statistics of single coordinates the resulting factorial representation of the joint probability distribution gives a density estimation we apply this concept to the real world problem of electrical motor fault detection treated as a novelty detection task

learning in neural networks based on a generalized fluctuation theorem information maximization has been investigated as a possible mechanism of learning governing the self organization that occurs within the neural systems of animals within the general context of models of neural systems bidirectionally interacting with environments however the role of information maximization remains to be elucidated for bidirectionally interacting physical systems universal laws describing the fluctuation they exhibit and the information they possess have recently been discovered these laws are termed fluctuation theorems in the present study we formulate a theory of learning in neural networks bidirectionally interacting with environments based on the principle of information maximization our formulation begins with the introduction of a generalized fluctuation theorem employing an interpretation appropriate for the present application which differs from the original thermodynamic interpretation we analytically and numerically demonstrate that the learning mechanism presented in our theory allows neural networks to efficiently explore their environments and optimally encode information about them

doubling bifurcations of destroyed t 2 tori abstract two duffing oscillators driven with incommensurate frequencies and coupled additively are investigated with this system we observe a host of complex behavior which compete with each other in parameter space in particular the period doubling of an attracting t 2 torus and the destruction of a t 2 torus as the parameters are varied along a curve in parameter space we observe period doubling bifurcations of a destroyed t 2 torus this phenomenon is studied in greater detail via a simple recursive map

the nonlinear dynamics of the damped and driven toda chain ii fourier and lyapunov analysis of tori abstract the dynamics of a periodically damped and driven toda chain with periodic boundary conditions is investigated with the help of spectral energy bifurcation diagrams and lyapunov diagrams it can be shown that quasiperiodic motions on tori tk of dimension k 3 exist the technical problems concerning the fourier and lyapunov analysis of the motions on tori tk k 2 are addressed

on the sufficiency of the hamilton jacobi bellman equation for optimality of the controls in a linear optimal time problem abstract we study sufficient conditions for optimally of the controls in a linear autonomous optimal time problem with lipschitz continuous cost functional these conditions involve a generalized hamilton jacobi bellman equation

generalized jarzynski equality under nonequilibrium feedback control the jarzynski equality is generalized to situations in which nonequilibrium systems are subject to a feedback control the new terms that arise as a consequence of the feedback describe the mutual information content obtained by measurement and the efficacy of the feedback control our results lead to a generalized fluctuation dissipation theorem that reflects the readout information and they can be experimentally tested using small thermodynamic systems we illustrate our general results by introducing an information ratchet which can transport a brownian particle in one direction and extract positive work from the particle

convergent algorithm for sensory receptive field development an unsupervised developmental algorithm for linear maps is derived which reduces the pixel entropy using the measure introduced in previous work at every update and thus removes pairwise correlations between pixels since the measure of pixel entropy has only a global minimum the algorithm is guaranteed to converge to the minimum entropy map such optimal maps have recently been shown to possess cognitively desirable properties and are likely to be used by the nervous system to organize sensory information the algorithm derived here turns out to be one proposed by goodall for pairwise decorrelation it is biologically plausible since in a neural network implementation it requires only data available locally to a neuron in training over ensembles of two dimensional input signals with the same spatial power spectrum as natural scenes networks develop output neurons with center surround receptive fields similar to those of ganglion cells in the retina some technical issues pertinent to developmental algorithms of this sort such as symmetry fixing are also discussed

an information maximization approach to blind separation and blind deconvolution we derive a new self organizing learning algorithm that maximizes the information transferred in a network of nonlinear units the algorithm does not assume any knowledge of the input distributions and is defined here for the zero noise limit under these conditions information maximization has extra properties not found in the linear case linsker 1989 the nonlinearities in the transfer function are able to pick up higher order moments of the input distributions and perform something akin to true redundancy reduction between units in the output representation this enables the network to separate statistically independent components in the inputs a higher order generalization of principal components analysis we apply the network to the source separation or cocktail party problem successfully separating unknown mixtures of up to 10 speakers we also show that a variant on the network architecture is able to perform blind deconvolution cancellation of unknown echoes and reverberation in a speech signal finally we derive dependencies of information transfer on time delays we suggest that information maximization provides a unifying framework for problems in blind signal processing