model structure simplification of nonlinear systems via immersion this paper concerns the simplification of model structures of nonlinear systems while preserving their input output maps the basic technique is an immersion of a system which is a mapping of the initial state from the original system to another system of an identical input output map the necessary and sufficient conditions are presented for the immersibility of a given system into a state space representation of such particular structures as rational functions or polynomials with respect to the state the conditions are so mild that many types of nonlinear systems can be represented by rational or polynomial model structures moreover it is also shown that a polynomial state space representation can always be constructed to be at most quadratic with respect to the state

learning overcomplete representations in an overcomplete basis the number of basis vectors is greater than the dimensionality of the input and the representation of an input is not a unique combination of basis vectors overcomplete representations have been advocated because they have greater robustness in the presence of noise can be sparser and can have greater flexibility in matching structure in the data overcomplete codes have also been proposed as a model of some of the response properties of neurons in primary visual cortex previous work has focused on finding the best representation of a signal using a fixed overcomplete basis or dictionary we present an algorithm for learning an overcomplete basis by viewing it as probabilistic model of the observed data we show that overcomplete bases can yield a better approximation of the underlying statistical distribution of the data and can thus lead to greater coding efficiency this can be viewed as a generalization of the technique of independent component analysis and provides a method for bayesian reconstruction of signals in the presence of noise and for blind source separation when there are more sources than mixtures

brief paper estimating the region of attraction for large scale systems with uncertainties in this paper a new approach is proposed for estimating regions of attraction for large scale dynamic systems in designing control laws for such systems it is essential to incorporate the underlying information structure constraints while keeping the number of optimization variables at a minimum the proposed method successfully accomplishes both of these objectives it is computationally efficient and can produce decentralized control laws without imposing structural constraints on the lyapunov function a feature that can considerably improve the quality of the estimate the design algorithm is based on linear matrix inequalities and can easily accommodate various types of uncertainties in the system model an example with 300 states is provided to demonstrate the suitability of this approach for large sparse systems

detecting changes in the basin of attraction of a dynamical system application to the postural restoring system a method that provides a three dimensional representation of the basin of attraction of a dynamical system from experimental data was applied to the problem of dynamic balance restoration the method is based on the density of the data on the phase space of the system under study and makes use of modeling and numerical curve fitting tools for the dynamical system of balance restoration the shape and the size of the basin of attraction depend on the dynamics of the postural restoring mechanisms and contain important information regarding the biomechanical as well as the neuromuscular condition of the individual the aim of this work was to examine the ability of the method to detect through the observed changes in the shape and or the size of the calculated basins of attraction a the inherent differences between different systems in the current application postural restoring systems of different individuals and b induced changes in the same system the postural restoring system of an individual the results of the study confirm the validity of the method and furthermore justify its robustness

enlarging the domain of attraction and maximising convergence rate for delta operator systems with actuator saturation in this paper we present some new approaches to improve the feedback property of delta operator systems with actuator saturation both enlarging the domain of attraction and maximising the convergence rate are the desired feedback properties the lifting technique is used to enlarge the domain of attraction for the delta operator systems subject to actuator saturation the maximisation of convergence rate is realised by designing control gain inside a given ellipsoid a necessary and sufficient condition is proposed for the contractive invariance of the given ellipsoid simulation results are provided to demonstrate the effectiveness of the developed techniques

gtm the generative topographic mapping latent variable models represent the probability density of data in a space of several dimensions in terms of a smaller number of latent or hidden variables a familiar example is factor analysis which is based on a linear transformations between the latent space and the data space in this paper we introduce a form of non linear latent variable model called the generative topographic mapping for which the parameters of the model can be determined using the em algorithm gtm provides a principled alternative to the widely used self organizing map som of kohonen 1982 and overcomes most of the significant limitations of the som we demonstrate the performance of the gtm algorithm on a toy problem and on simulated data from flow diagnostics for a multi phase oil pipeline

emergence of phase and shift invariant features by decomposition of natural images into independent feature subspaces olshausen and field 1996 applied the principle of independence maximization by sparse coding to extract features from natural images this leads to the emergence of oriented linear filters that have simultaneous localization in space and in frequency thus resembling gabor functions and simple cell receptive fields in this article we show that the same principle of independence maximization can explain the emergence of phase and shift invariant features similar to those found in complex cells this new kind of emergence is obtained by maximizing the independence between norms of projections on linear subspaces instead of the independence of simple linear filter outputs the norms of the projections on such independent feature subspaces then indicate the values of invariant features

self organization of topographic bilinear networks for invariant recognition we present a model for the emergence of ordered fiber projections that may serve as a basis for invariant recognition after invariance transformations are self organized so called control units competitively activate fiber projections for different transformation parameters the model builds on a well known ontogenetic mechanism activity based development of retinotopy and it employs activity blobs of varying position and size to install different transformations we provide a detailed analysis for the case of 1d input and output fields for schematic input patterns that shows how the model is able to develop specific mappings we discuss results that show that the proposed learning scheme is stable for complex biologically more realistic input patterns finally we show that the model generalizes to 2d neuronal fields driven by simulated retinal waves

arc length based maximal lyapunov functions and domains of attraction estimation for polynomial nonlinear systems in the phase space of a dynamical system containing an asymptotically stable equilibrium point the arc length function alf is defined as sum of length differential elements of phase trajectories starting from state points and ending at the equilibrium point it is shown that receding from the origin and verging on the domain of attraction doa boundary causes alf to be increased drastically according to the latter issue and regarding to other properties it is shown that alf is a maximal lyapunov function to this end a numerical method to approximate the alf as a polynomial function is proposed to ensure that the approximated function has positive value and negative derivative inside the desired region the homotopy continuation method is used thus the approximated alf presents an ensured lyapunov behavior to estimate doa the efficacy of the proposed method is demonstrated by several simulation examples

conditional high order boltzmann machines for supervised relation learning relation learning is a fundamental problem in many vision tasks recently high order boltzmann machine and its variants have shown their great potentials in learning various types of data relation in a range of tasks but most of these models are learned in an unsupervised way i e without using relation class labels which are not very discriminative for some challenging tasks e g face verification in this paper with the goal to perform supervised relation learning we introduce relation class labels into conventional high order multiplicative interactions with pairwise input samples and propose a conditional high order boltzmann machine chbm which can learn to classify the data relation in a binary classification way to be able to deal with more complex data relation we develop two improved variants of chbm 1 latent chbm which jointly performs relation feature learning and classification by using a set of latent variables to block the pathway from pairwise input samples to output relation labels and 2 gated chbm which untangles factors of variation in data relation by exploiting a set of latent variables to multiplicatively gate the classification of chbm to reduce the large number of model parameters generated by the multiplicative interactions we approximately factorize high order parameter tensors into multiple matrices then we develop efficient supervised learning algorithms by first pretraining the models using joint likelihood to provide good parameter initialization and then finetuning them using conditional likelihood to enhance the discriminant ability we apply the proposed models to a series of tasks including invariant recognition face verification and action similarity labeling experimental results demonstrate that by exploiting supervised relation labels our models can greatly improve the performance

separation theorem for independent subspace analysis and its consequences independent component analysis ica the theory of mixed independent non gaussian sources has a central role in signal processing computer vision and pattern recognition one of the most fundamental conjectures of this research field is that independent subspace analysis isa the extension of the ica problem where groups of sources are independent can be solved by traditional ica followed by grouping the ica components the conjecture called isa separation principle i has been rigorously proven for some distribution types recently ii forms the basis of the state of the art isa solvers iii enables one to estimate the unknown number and the dimensions of the sources efficiently and iv can be extended to generalizations of the isa task such as different linear controlled post nonlinear complex valued partially observed problems as well as to problems dealing with nonparametric source dynamics here we shall review the advances on this field

a stochastic self organizing map for proximity data we derive an efficient algorithm for topographic mapping of proximity data tmp which can be seen as an extension of kohonen s self organizing map to arbitrary distance measures the tmp cost function is derived in a baysian framework of folded markov chains for the description of autoencoders it incorporates the data by a dissimilarity matrix and the topographic neighborhood by a matrix of transition probabilities from the principle of maximum entropy a nonfactorizing gibbs distribution is obtained which is approximated in a mean field fashion this allows for maximum likelihood estimation using an expectation maximization algorithm in analogy to the transition from topographic vector quantization to the self organizing map we suggest an approximation to tmp that is computationally more efficient in order to prevent convergence to local minima an annealing scheme in the temperature parameter is introduced for which the critical temperature of the first phase transition is calculated in terms o

slow feature analysis unsupervised learning of invariances invariant features of temporally varying signals are useful for analysis and classification slow feature analysis sfa is a new method for learning invariant or slowly varying features from a vectorial input signal it is based on a nonlinear expansion of the input signal and application of principal component analysis to this expanded signal and its time derivative it is guaranteed to find the optimal solution within a family of functions directly and can learn to extract a large number of decorrelated features which are ordered by their degree of invariance sfa can be applied hierarchically to process high dimensional input signals and extract complex features sfa is applied first to complex cell tuning properties based on simple cell output including disparity and motion then more complicated input output functions are learned by repeated application of sfa finally a hierarchical network of sfa modules is presented as a simple model of the visual system the same unstructured network can learn translation size rotation contrast or to a lesser degree illumination invariance for one dimensional objects depending on only the training stimulus surprisingly only a few training objects suffice to achieve good generalization to new objects the generated representation is suitable for object recognition performance degrades if the network is trained to learn multiple invariances simultaneously

an analytical method of estimating the domain of attraction for polynomial differential equations a system of nth order differential equations with polynomial right hand sides is considered and a simple analytical method of estimating the domain of attraction is developed the method involves an ordinary quadratic lyapunov function spl upsi x sup t q y x with a certain parameter vector y from the same state space all linear factors in the expression for spl upsi spl dot are bounded in the domain spl upsi spl les 1 and the derivative is bounded by a quadratic function the negativeness of which determines the restrictions for y the domain of attraction is estimated through a simple scaling of the obtained area or through its nonlinear transformation with optimization the method allows for obtaining domains for example with infinite volume that are comparable with ones obtained by complicated computational procedures a set of examples is presented

discriminative deep face shape model for facial point detection facial point detection is an active area in computer vision due to its relevance to many applications it is a nontrivial task since facial shapes vary significantly with facial expressions poses or occlusion in this paper we address this problem by proposing a discriminative deep face shape model that is constructed based on an augmented factorized three way restricted boltzmann machines model specifically the discriminative deep model combines the top down information from the embedded face shape patterns and the bottom up measurements from local point detectors in a unified framework in addition along with the model effective algorithms are proposed to perform model learning and to infer the true facial point locations from their measurements based on the discriminative deep face shape model 68 facial points are detected on facial images in both controlled and in the wild conditions experiments on benchmark data sets show the effectiveness of the proposed facial point detection algorithm against state of the art methods

bilinear sparse coding for invariant vision recent algorithms for sparse coding and independent component analysis ica have demonstrated how localized features can be learned from natural images however these approaches do not take image transformations into account we describe an unsupervised algorithm for learning both localized features and their transformations directly from images using a sparse bilinear generative model we show that from an arbitrary set of natural images the algorithm produces oriented basis filters that can simultaneously represent features in an image and their transformations the learned generative model can be used to translate features to different locations thereby reducing the need to learn the same feature at multiple locations a limitation of previous approaches to sparse coding and ica our results suggest that by explicitly modeling the interaction between local image features and their transformations the sparse bilinear approach can provide a basis for achieving transformation invariant vision

learning the lie groups of visual invariance a fundamental problem in biological and machine vision is visual invariance how are objects perceived to be the same despite transformations such as translations rotations and scaling in this letter we describe a new unsupervised approach to learning invariances based on lie group theory unlike traditional approaches that sacrifice information about transformations to achieve invariance the lie group approach explicitly models the effects of transformations in images as a result estimates of transformations are available for other purposes such as pose estimation and visuomotor control previous approaches based on first order taylor series expansions of images can be regarded as special cases of the lie group approach which utilizes a matrix exponential based generative model of images and can handle arbitrarily large transformations we present an unsupervised expectation maximization algorithm for learning lie transformation operators directly from image data containing examples of transformations our experimental results show that the lie operators learned by the algorithm from an artificial data set containing six types of affine transformations closely match the analytically predicted affine operators we then demonstrate that the algorithm can also recover novel transformation operators from natural image sequences we conclude by showing that the learned operators can be used to both generate and estimate transformations in images thereby providing a basis for achieving visual invariance

discrete restricted boltzmann machines we describe discrete restricted boltzmann machines probabilistic graphical models with bipartite interactions between visible and hidden discrete variables examples are binary restricted boltzmann machines and discrete naive bayes models we detail the inference functions and distributed representations arising in these models in terms of configurations of projected products of simplices and normal fans of products of simplices we bound the number of hidden variables depending on the cardinalities of their state spaces for which these models can approximate any probability distribution on their visible states to any given accuracy in addition we use algebraic methods and coding theory to compute their dimension

stability analysis of riccati differential equations related to affine diffusion processes we study a class of generalized riccati differential equations associated with affine diffusion processes these diffusions arise in financial econometrics and branching processes the generalized riccati equations determine the fourier transform of the diffusion s transition law we investigate stable regions of the dynamical systems and analyze their blow up times we discuss the implication of applying these results to affine diffusions and in particular to option pricing theory

learning to represent spatial transformations with factored higher order boltzmann machines to allow the hidden units of a restricted boltzmann machine to model the transformation between two successive images memisevic and hinton 2007 introduced three way multiplicative interactions that use the intensity of a pixel in the first image as a multiplicative gain on a learned symmetric weight between a pixel in the second image and a hidden unit this creates cubically many parameters which form a three dimensional interaction tensor we describe a low rank approximation to this interaction tensor that uses a sum of factors each of which is a three way outer product this approximation allows efficient learning of transformations between larger image patches since each factor can be viewed as an image filter the model as a whole learns optimal filter pairs for efficiently representing transformations we demonstrate the learning of optimal filter pairs from various synthetic and real image sequences we also show how learning about image transformations allows the model to perform a simple visual analogy task and we show how a completely unsupervised network trained on transformations perceives multiple motions of transparent dot patterns in the same way as humans

a fast fixed point algorithm for independent component analysis we introduce a novel fast algorithm for independent component analysis which can be used for blind source separation and feature extraction we show how a neural network learning rule can be transformed into a fixedpoint iteration which provides an algorithm that is very simple does not depend on any user defined parameters and is fast to converge to the most accurate solution allowed by the data the algorithm finds one at a time all nongaussian independent components regardless of their probability distributions the computations can be performed in either batch mode or a semiadaptive manner the convergence of the algorithm is rigorously proved and the convergence speed is shown to be cubic some comparisons to gradient based algorithms are made showing that the new algorithm is usually 10 to 100 times faster sometimes giving the solution in just a few iterations

further results on the bounds of the zeros of quasi critical polynomials on the basis of the relationship of the mth power of a polynomial and its modular form polynomial whose coefficients are the moduli of the coefficients of that polynomial we derive a necessary and sufficient condition for the modulus of the mth power of a polynomial for contacting its modular form on the boundary of a disc combined with the result about distribution of zeros of analytic function some new sufficient conditions are derived which give bounds of the absolute values of the roots of a quasi critical polynomial these results extend certain earlier similar tests for linear discrete time systems finally four examples are given to demonstrate the results example 2 1 gives a state feedback application examples 2 2 and 2 4 deal with r stability and example 2 3 display that our theorems give better results when m increases but at the cost of increasing complexity

sparse code shrinkage denoising of nongaussian data by maximum likelihood estimation sparse coding is a method for finding a representation of data in which each of the components of the representation is only rarely significantly active such a representation is closely related to redundancy reduction and independent component analysis and has some neurophysiological plausibility in this article we show how sparse coding can be used for denoising using maximum likelihood estimation of nongaussian variables corrupted by gaussian noise we show how to apply a soft thresholding shrinkage operator on the components of sparse coding so as to reduce noise our method is closely related to the method of wavelet shrinkage but it has the important benefit over wavelet methods that the representation is determined solely by the statistical properties of the data the wavelet representation on the other hand relies heavily on certain mathematical properties like self similarity that may be only weakly related to the properties of natural data

on power bounded operators in finite von neumann algebras abstract it is proved that any power bounded operator of class c 1 in a finite von neumann algebra is conjugate to a unitary this solves a conjecture stated by i kovacs in 1970 an important ingredient of the proof is the study of completely positive projections on some operator space

a spatio temporal rbm based model for facial expression recognition the ability to recognize facial expressions will be an important characteristic of next generation human computer interfaces towards this goal we propose a novel rbm based model to learn effectively the relationships or transformations between image pairs associated with different facial expressions the proposed model has the ability to disentangle these transformations e g pose variations and facial expressions by encoding them into two different hidden sets namely facial expression morphlets and non facial expression morphlets the first hidden set is used to encode facial expression morphlets through a factored four way sub model conditional to label units the second hidden set is used to encode non facial expression morphlets through a factored three way sub model with such a strategy the proposed model can learn transformations between image pairs while disentangling facial expression transformations from non facial expression transformations this is achieved using an algorithm dubbed quadripartite contrastive divergence reported experiments demonstrate the superior performance of the proposed model compared to state of the art highlightsintroducing a novel rbm based model to capture transformations between image pairs disentangling fer transformations from other transformations using two hidden sets introducing a quadripartite contrastive divergence algorithm to learn our model

learning invariance from transformation sequences the visual system can reliably identify objects even when the retinal image is transformed considerably by commonly occurring changes in the environment a local learning rule is proposed which allows a network to learn to generalize across such transformations during the learning phase the network is exposed to temporal sequences of patterns undergoing the transformation an application of the algorithm is presented in which the network learns invariance to shift in retinal position such a principle may be involved in the development of the characteristic shift invariance property of complex cells in the primary visual cortex and also in the development of more complicated invariance properties of neurons in higher visual areas

proof of the conjecture of a grothendieck on the fuglede kadison determinant let m be a von neumann algebra with a faithful semifinite trace for compact elements x m grothendieck introduced in seminaire bourbaki 1955 a function 1 x generalizing the fuglede kadison determinant and conjectured that 1 x y 1 x 1 y in this paper the inequality is proved as a corollary a direct proof of the clarkson inequalities is obtained

on the statistical analysis of dirty pictures may 7th 1986 professor a f m smith in the chair summary a continuous two dimensional region is partitioned into a fine rectangular array of sites or pixels each pixel having a particular colour belonging to a prescribed finite set the true colouring of the region is unknown but associated with each pixel there is a possibly multivariate record which conveys imperfect information about its colour according to a known statistical model the aim is to reconstruct the true scene with the additional knowledge that pixels close together tend to have the same or similar colours in this paper it is assumed that the local characteristics of the true scene can be represented by a nondegenerate markov random field such information can be combined with the records by bayes theorem and the true scene can be estimated according to standard criteria however the computational burden is enormous and the reconstruction may reflect undesirable largescale properties of the random field thus a simple iterative method of reconstruction is proposed which does not depend on these large scale characteristics the method is illustrated by computer simulations in which the original scene is not directly related to the assumed random field some complications including parameter estimation are discussed potential applications are mentioned briefly

images frames and connectionist hierarchies the representation of hierarchically structured knowledge in systems using distributed patterns of activity is an abiding concern for the connectionist solution of cognitively rich problems here we use statistical unsupervised learning to consider semantic aspects of structured knowledge representation we meld unsupervised learning notions formulated for multilinear models with tensor product ideas for representing rich information we apply the model to images of faces

a new image representation algorithm inspired by image submodality models redundancy reduction and learning in biological vision we develop a new biologically motivated algorithm for representing natural images using successive projections into complementary subspaces an image is first projected into an edge subspace spanned using an ica basis adapted to natural images which captures the sharp features of an image like edges and curves the residual image obtained after extraction of the sharp image features is approximated using a mixture of probabilistic principal component analyzers mppca model the model is consistent with cellular functional information theoretic and learning paradigms in visual pathway modeling we demonstrate the efficiency of our model for representing different attributes of natural images like color and luminance we compare the performance of our model in terms of quality of representation against commonly used basis like the discrete cosine transform dct independent component analysts ica and principal components analysis pca based on their entropies chrominance and luminance components of images are represented using codes having lower entropy than dct ica or pca for similar visual quality the model attains considerable simplification for learning from images by using a sparse independent code for representing edges and explicitly evaluating probabilities in the residual subspace

fast and robust fixed point algorithms for independent component analysis independent component analysis ica is a statistical method for transforming an observed multidimensional random vector into components that are statistically as independent from each other as possible we use a combination of two different approaches for linear ica comon s information theoretic approach and the projection pursuit approach using maximum entropy approximations of differential entropy we introduce a family of new contrast functions for ica these contrast functions enable both the estimation of the whole decomposition by minimizing mutual information and estimation of individual independent components as projection pursuit directions the statistical properties of the estimators based on such contrast functions are analyzed under the assumption of the linear mixture model and it is shown how to choose contrast functions that are robust and or of minimum variance finally we introduce simple fixed point algorithms for practical optimization of the contrast functions

rotation invariant hog descriptors using fourier analysis in polar and spherical coordinates the histogram of oriented gradients hog is widely used for image description and proves to be very effective in many vision problems rotation invariant analysis is necessary or preferred popular solutions are mainly based on pose normalization or learning neglecting some intrinsic properties of rotations this paper presents a method to build rotation invariant hog descriptors using fourier analysis in polar spherical coordinates which are closely related to the irreducible representation of the 2d 3d rotation groups this is achieved by considering a gradient histogram as a continuous angular signal which can be well represented by the fourier basis 2d or spherical harmonics 3d as rotation invariance is established in an analytical way we can avoid discretization artifacts and create a continuous mapping from the image to the feature space in the experiments we first show that our method outperforms the state of the art in a public dataset for a car detection task in aerial images we further use the princeton shape benchmark and the shrec 2009 generic shape benchmark to demonstrate the high performance of our method for similarity measures of 3d shapes finally we show an application on microscopic volumetric data

training products of experts by minimizing contrastive divergence it is possible to combine multiple latent variable models of the same data by multiplying their probability distributions together and then renormalizing this way of combining individual expert models makes it hard to generate samples from the combined model but easy to infer the values of the latent variables of each expert because the combination rule ensures that the latent variables of different experts are conditionally independent when given the data a product of experts poe is therefore an interesting candidate for a perceptual system in which rapid inference is vital and generation is unnecessary training a poe by maximizing the likelihood of the data is difficult because it is hard even to approximate the derivatives of the renormalization term in the combination rule fortunately a poe can be trained using a different objective function called contrastive divergence whose derivatives with regard to the parameters can be approximated accurately and efficiently examples are presented of contrastive divergence learning using several types of expert on several types of data

learning intermediate level representations of form and motion from natural movies we present a model of intermediate level visual representation that is based on learning invariances from movies of the natural environment the model is composed of two stages of processing an early feature representation layer and a second layer in which invariances are explicitly represented invariances are learned as the result of factoring apart the temporally stable and dynamic components embedded in the early feature representation the structure contained in these components is made explicit in the activities of second layer units that capture invariances in both form and motion when trained on natural movies the first layer produces a factorization or separation of image content into a temporally persistent part representing local edge structure and a dynamic part representing local motion structure consistent with known response properties in early visual cortex area v1 this factorization linearizes statistical dependencies among the first layer units making them learnable by the second layer the second layer units are split into two populations according to the factorization in the first layer the form selective units receive their input from the temporally persistent part local edge structure and after training result in a diverse set of higher order shape features consisting of extended contours multiscale edges textures and texture boundaries the motion selective units receive their input from the dynamic part local motion structure and after training result in a representation of image translation over different spatial scales and directions in addition to more complex deformations these representations provide a rich description of dynamic natural images and testable hypotheses regarding intermediate level representation in visual cortex

kernel reconstruction ica for sparse representation independent component analysis with soft reconstruction cost rica has been recently proposed to linearly learn sparse representation with an overcomplete basis and this technique exhibits promising performance even on unwhitened data however linear rica may not be effective for the majority of real world data because nonlinearly separable data structure pervasively exists in original data space meanwhile rica is essentially an unsupervised method and does not employ class information motivated by the success of the kernel trick that maps a nonlinearly separable data structure into a linearly separable case in a high dimensional feature space we propose a kernel rica krica model to nonlinearly capture sparse representation in feature space furthermore we extend the unsupervised krica to a supervised one by introducing a class driven discrimination constraint such that the data samples from the same class are well represented on the basis of the corresponding subset of basis vectors this discrimination constraint minimizes inhomogeneous representation energy and maximizes homogeneous representation energy simultaneously which is essentially equivalent to maximizing between class scatter and minimizing within class scatter at the same time in an implicit manner experimental results demonstrate that the proposed algorithm is more effective than other state of the art methods on several datasets

equivariant adaptive source separation source separation consists of recovering a set of independent signals when only mixtures with unknown coefficients are observed this paper introduces a class of adaptive algorithms for source separation that implements an adaptive version of equivariant estimation and is henceforth called equivariant adaptive separation via independence easi the easi algorithms are based on the idea of serial updating this specific form of matrix updates systematically yields algorithms with a simple structure for both real and complex mixtures most importantly the performance of an easi algorithm does not depend on the mixing matrix in particular convergence rates stability conditions and interference rejection levels depend only on the normalized distributions of the source signals closed form expressions of these quantities are given via an asymptotic performance analysis the theme of equivariance is stressed throughout the paper the source separation problem has an underlying multiplicative structure the parameter space forms a matrix multiplicative group we explore the favorable consequences of this fact on implementation performance and optimization of easi algorithms

a multifactor winner take all dynamics perceptual systems often have to disentangle different factors from mixed observations if each factor is represented by a set of variables each standing for a discrete value of the factor the factor values underlying an observation can be extracted by a winner take all wta mechanism over the direct product of the factors search in the product space however is expensive it is computationally attractive to work on the marginal factors in this letter we study the dynamics of a multifactor system modeled by a number of interacting wta dynamics one for each factor we give theoretical results on the stable fixed points of this system and show experimental results on invariant object recognition

modeling natural images using gated mrfs this paper describes a markov random field for real valued image modeling that has two sets of latent variables one set is used to gate the interactions between all pairs of pixels while the second set determines the mean intensities of each pixel this is a powerful model with a conditional distribution over the input that is gaussian with both mean and covariance determined by the configuration of latent variables which is unlike previous models that were restricted to using gaussians with either a fixed mean or a diagonal covariance matrix thanks to the increased flexibility this gated mrf can generate more realistic samples after training on an unconstrained distribution of high resolution natural images furthermore the latent variables of the model can be inferred efficiently and can be used as very effective descriptors in recognition tasks both generation and discrimination drastically improve as layers of binary latent variables are added to the model yielding a hierarchical model called a deep belief network

brief a new lyapunov design approach for nonlinear systems based on zubov s method the present research work proposes a new nonlinear controller synthesis approach that is based on the methodological principles of lyapunov design in particular it relies on a short horizon model based prediction and optimization of the rate of energy dissipation of the system as it is realized through the time derivative of an appropriately selected lyapunov function the latter is computed by solving zubov s partial differential equation based on the system s drift vector field a nonlinear state feedback control law with two adjustable parameters is derived as the solution of an optimization problem that is formulated on the basis of the aforementioned lyapunov function and closed loop performance characteristics a set of system theoretic properties of the proposed control law are examined as well finally the proposed lyapunov design method is evaluated in a chemical reactor example which exhibits nonminimum phase behaviour

bilinear models for spatio temporal point distribution analysis in this work we describe the usage of bilinear statistical models as a means of factoring the shape variability into two components attributed to inter subject variation and to the intrinsic dynamics of the human heart we show that it is feasible to reconstruct the shape of the heart at discrete points in the cardiac cycle provided we are given a small number of shape instances representing the same heart at different points in the same cycle we can use the bilinear model to establish this using a temporal and a spatial alignment step in the preprocessing of the shapes around half of the reconstruction errors were on the order of the axial image resolution of 2 mm and over 90 was within 3 5 mm from this we conclude that the dynamics were indeed separated from the inter subject variability in our dataset

characterization of stability region for general autonomous nonlinear dynamical systems the existing characterization of stability regions was developed under the assumption that limit sets on the stability boundary are exclusively composed of hyperbolic equilibrium points and closed orbits the characterizations derived in this technical note are a generalization of existing results in the theory of stability regions a characterization of the stability boundary of general autonomous nonlinear dynamical systems is developed under the assumption that limit sets on the stability boundary are composed of a countable number of disjoint and indecomposable components which can be equilibrium points closed orbits quasi periodic solutions and even chaotic invariant sets

polynomial level set method for polynomial system reachable set estimation in this paper we present a polynomial level set method for advecting a semi algebraic set for polynomial systems this method uses the sub level representation of sets the problem of flowing these sets under the advection map of a dynamical system is converted to a semi definite program which is then used to compute the coefficients of the polynomials the method presented in this paper does not require either the sets being positively invariant or star shaped hence the proposed algorithm can describe the behavior of system states both inside and outside the domain of attraction and can also be used to describe more complex shapes of sets we further address the related problems of constraining the degree of the polynomials various numerical examples are presented to show the effectiveness of advection approach

self organization of control circuits for invariant fiber projections assuming that patterns in memory are represented as two dimensional arrays of local features just as they are in primary visual cortices pattern recognition can take the form of elastic graph matching lades eti al 1993 neural implementation of this may be based on preorganized fiber projections that can be activated rapidly with the help of control units wolfrum wolff lucke von der malsburg 2008 each control unit governs a set of projection fibers that form part of a coherent mapping we describe a mathematical model for the ontogenesis of the underlying connectivity based on a principle of network self organization as described by the haussler system haussler von der malsburg 1983 modified to be sensitive to pattern similarity and to support formation of multiple mappings each under the command of a control unit the process takes the form of a soft winner take all where units compete for the representation of maps we show simulations for invariant point to point and feature to feature mappings

quasisimilarity of power bounded operators and blum hanson property abstract we construct a power bounded operator on a hilbert space which is not quasisimilar to a contraction to this aim we solve an open problem from operator ergodic theory showing that there are power bounded hilbert space operators without the blum hanson property we also find an example of a power bounded operator quasisimilar to a unitary operator which is not similar to a contraction thus answering negatively open questions raised by kerchy and cassier on the positive side we prove that contractions on l p spaces 1 p possess the blum hanson property

stability region analysis using polynomial and composite polynomial lyapunov functions and sum of squares programming we propose using bilinear sum of squares programming for obtaining inner bounds of roas for dynamical systems with polynomial vector fields we search for polynomial as well as composite lyapunov functions comprised of pointwise maximums of polynomial functions results for several examples from the literature are presented using the proposed methods and the penbmi solver

a maximum likelihood interpretation for slow feature analysis the brain extracts useful features from a maelstrom of sensory information and a fundamental goal of theoretical neuroscience is to work out how it does so one proposed feature extraction strategy is motivated by the observation that the meaning of sensory data such as the identity of a moving visual object is often more persistent than the activation of any single sensory receptor this notion is embodied in the slow feature analysis sfa algorithm which uses slowness as a heuristic by which to extract semantic information from multidimensional time series here we develop a probabilistic interpretation of this algorithm showing that inference and learning in the limiting case of a suitable probabilistic model yield exactly the results of sfa similar equivalences have proved useful in interpreting and extending comparable algorithms such as independent component analysis for sfa we use the equivalent probabilistic model as a conceptual springboard with which to motivate several novel extensions to the algorithm

contractions in von neumann algebras abstract we associate to any contraction t and more generally to any operator t of class c in a von neumann algebra m an operator kernel k t t when m is finite we use this kernel to give a short proof of the fuglede kadison theorem on the location of the trace and to prove that a contraction t in m is unitary if and only if its spectrum is contained in the unit circle by using a perturbation of the kernel k t we give for any operator t of class c acting on a separable hilbert space h a short proof of the power inequality for the numerical range and an accurate conjugacy to a contraction result for t we also get a generalized von neumann inequality which gives a good control of f rt x g rt x 0 r x h and f g in the disc algebra finally we associate to any c 1 contraction in a hilbert space an asymptotic kernel which allows us to describe new kinds of invariant subspaces for t from the positive solutions x of the operator equation t xt x in particular we recover some results of beauzamy based on the notion of invariant subspace of functional type

an information maximization approach to blind separation and blind deconvolution we derive a new self organizing learning algorithm that maximizes the information transferred in a network of nonlinear units the algorithm does not assume any knowledge of the input distributions and is defined here for the zero noise limit under these conditions information maximization has extra properties not found in the linear case linsker 1989 the nonlinearities in the transfer function are able to pick up higher order moments of the input distributions and perform something akin to true redundancy reduction between units in the output representation this enables the network to separate statistically independent components in the inputs a higher order generalization of principal components analysis we apply the network to the source separation or cocktail party problem successfully separating unknown mixtures of up to 10 speakers we also show that a variant on the network architecture is able to perform blind deconvolution cancellation of unknown echoes and reverberation in a speech signal finally we derive dependencies of information transfer on time delays we suggest that information maximization provides a unifying framework for problems in blind signal processing

constrained optimization for neural map formation a unifying framework for weight growth and normalization computational models of neural map formation can be considered on at least three different levels of abstraction detailed models including neural activity dynamics weight dynamics that abstract from the neural activity dynamics by an adiabatic approximation and constrained optimization from which equations governing weight dynamics can be derived constrained optimization uses an objective function from which a weight growth rule can be derived as a gradient flow and some constraints from which normalization rules are derived in this article we present an example of how an optimization problem can be derived from detailed nonlinear neural dynamics a systematic investigation reveals how different weight dynamics introduced previously can be derived from two types of objective function terms and two types of constraints this includes dynamic link matching as a special case of neural map formation we focus in particular on the role of coordinate transformations to derive different weight dynamics from the same optimization problem several examples illustrate how the constrained optimization framework can help in understanding generating and comparing different models of neural map formation the techniques used in this analysis may also be useful in investigating other types of neural dynamics

topographic independent component analysis in ordinary independent component analysis the components are assumed to be completely independent and they do not necessarily have any meaningful order relationships in practice however the estimated independent components are often not at all independent we propose that this residual dependence structure could be used to define a topographic order for the components in particular a distance between two components could be defined using their higher order correlations and this distance could be used to create a topographic representation thus we obtain a linear decomposition into approximately independent components where the dependence of two components is approximated by the proximity of the components in the topographic representation