some remarks on the computation of pythagoras numbers of real irreducible algebroid curves through gram matrices we clarify a difficulty that appears in r quarez j algebra 238 2001 139 to bound the pythagoras number of a real irreducible algebroid curve by its multiplicity

why do stabilizing controllers stabilize an alternative notion of a stabilizing controller is presented being defined in terms of the trajectories of plant and controller it is well suited for situations in which the controller s action is an interconnecting one in the second part of the paper a parametrization is given of the kernel representations of all controllers that give rise to an a priori specified closed loop behavior as a special case the parametrization yields all stabilizing controllers unlike the youla parametrization it also includes the so called singular stabilizing controllers

a behavioral approach to the h optimal control problem this paper considers the general h optimal control problem from a behavioral perspective a formalization of this problem is given that departs from the usual h control paradigm in the sense that system variables of the plant are treated in a symmetric way without distinguishing control inputs measurements exogenous inputs and to be controlled variables interconnection variables are introduced and controllers are allowed to constrain the interconnection variables of the plant necessary and sufficient conditions are given for the existence of controllers that achieve an h control objective for a given linear time invariant plant the set of all such controllers is parametrized by means of j spectral factorizations

observer synthesis in the behavioral approach analyzes the observer design problem in the behavioral context observability and detectability notions are first introduced and fully characterized necessary and sufficient conditions for the existence of an observer possibly an asymptotic or an exact one are introduced and a complete parameterization of all admissible observers is given the problem of obtaining observers endowed with a strictly proper transfer matrix and the design of observer based controllers are later addressed and solved finally the above issues are particularized to the case of state space systems thus showing they naturally generalize well known theorems of traditional system theory

functional clones and expressibility of partition functions abstract we study functional clones which are sets of non negative pseudo boolean functions functions 0 1 k r 0 closed under essentially multiplication summation and limits functional clones naturally form a lattice under set inclusion and are closely related to counting constraint satisfaction problems csps we identify a sublattice of interesting functional clones and investigate the relationships and properties of the functional clones in this sublattice

graph regularized restricted boltzmann machine the restricted boltzmann machine rbm has received an increasing amount of interest in recent years it determines good mapping weights that capture useful latent features in an unsupervised manner the rbm and its generalizations have been successfully applied to a variety of image classification and speech recognition tasks however most of the existing rbm based models disregard the preservation of the data manifold structure in many real applications the data generally reside on a low dimensional manifold embedded in high dimensional ambient space in this brief we propose a novel graph regularized rbm to capture features and learning representations explicitly considering the local manifold structure of the data by imposing manifold based locality that preserves constraints on the hidden layer of the rbm the model ultimately learns sparse and discriminative representations the representations can reflect data distributions while simultaneously preserving the local manifold structure of data we test our model using several benchmark image data sets for unsupervised clustering and supervised classification problem the results demonstrate that the performance of our method exceeds the state of the art alternatives

robust stability of feedback systems a geometric approach using the gap metric a geometric framework for robust stabilization of infinite dimensional time varying linear systems is presented the uncertainty of a system is described by perturbations of its graph and is measured in the gap metric necessary and sufficient conditions for robust stability are generalized from the time invariant case an example is given to highlight an important difference between the obstructions which limit the size of a stabilizable gap ball in the time varying and time invariant cases several results on the gap metric and the gap topology are established that are central in a geometric treatment of the robust stabilizability problem in the gap in particular the concept of a graphable subspace is introduced in the paper subspaces that fail to be graphable are characterized by an index condition on a certain semi fredholm operator

on the control of finite support behaviors abstract in the paper the main properties of finite support behaviors are introduced and characterized in terms of the polynomial matrices involved in their representation necessary and sufficient conditions for the existence of an input output possibly non anticipating description of a finite support behavior are given the control problem is also addressed and properly restated in the behavior theory context as an interconnection problem conditions for the existence of controllers that make the resulting connected system observable as well as a complete description of the class of characteristic polynomials that the resulting controlled system possibly exhibits are given finally the problem of controlling a dynamical system in input output from by means of a controller with the same structure is investigated and solved

paradigms and puzzles in the theory of dynamical systems a self contained exposition is given of an approach to mathematical models in particular to the theory of dynamical systems the basic ingredients form a triptych with the behavior of a system in the center and behavioral equations with latent variables as side panels the author discusses a variety of representation and parametrization problems in particular questions related to input output and state models the proposed concept of a dynamical system leads to a new view of the notions of controllability and observability and of the interconnection of systems in particular to what constitutes a feedback control law the final issue addressed is that of system identification it is argued that exact system identification leads to the question of computing the most powerful unfalsified model

stabilisation and control design by partial output feedback and by partial interconnection we study stabilisation and control design by partial output feedback on the one hand and by partial interconnection according to willems on the other hand this article is based on the paper design parametrization and pole placement of stabilizing output feedback compensators via injective cogenerator quotient signal modules by i blumthaler and u oberst and the same technique is used both discrete and continuous behaviours and various notions of stability are treated simultaneously in the case of partial output feedback we require that both the compensator and the feedback behaviour have proper transfer matrices without presuming properness of the plant we obtain conditions ensuring the existence of such stabilising compensators which perform additional control tasks like for instance tracking and an algorithm for constructing a large class of such compensators in the case of stabilisation by interconnection no input output structures of plant or compensator are assumed but the controlled be

universal approximation depth and errors of narrow belief networks with discrete units we generalize recent theoretical work on the minimal number of layers of narrow deep belief networks that can approximate any probability distribution on the states of their visible units arbitrarily well we relax the setting of binary units sutskever hinton 2008 le roux bengio 2008 2010 montufar ay 2011 to units with arbitrary finite state spaces and the vanishing approximation error to an arbitrary approximation error tolerance for example we show that a q ary deep belief network with layers of width for some can approximate any probability distribution on without exceeding a kullback leibler divergence of our analysis covers discrete restricted boltzmann machines and naive bayes models as special cases

a fuzzy restricted boltzmann machine novel learning algorithms based on the crisp possibilistic mean value of fuzzy numbers a fuzzy restricted boltzmann machine frbm is extended from a restricted boltzmann machine rbm by replacing all the real valued parameters with fuzzy numbers a new frbm that employs the crisp possibilistic mean value of a fuzzy number to defuzzify the fuzzy free energy function is presented this approach is much clearer and easier to obtain the expression of the defuzzified free energy function and its approximation than the centroid method several theorems that discuss the error bounds of the approximation to ensure the rationality and validity are also investigated learning algorithms are given for the designed frbm with symmetric triangular fuzzy numbers stfns asymmetric triangular fuzzy numbers and gaussian fuzzy numbers by appropriately choosing the parameters a theorem is concluded that all frbms with symmetric fuzzy numbers will have identical learning algorithm to that of frbms with stfns this is illustrated by a case of frbm with gaussian fuzzy numbers two experiments including the mnist handwriting recognition and the bars and stripes benchmark are carried out the results show that the proposed frbms significantly outperform rbms in learning accuracy and generalization ability especially when encountering unlearned samples and recovering incomplete images

fast modes in the set of minimal dissipation trajectories abstract in this paper we study the set of trajectories satisfying both a given lti system s laws and also laws of the corresponding adjoint system in other words trajectories in the intersection of the system s behavior and that of the adjoint system this intersection has important system theoretic significance for example it is known that the trajectories in this intersection are the ones with minimal dissipation underlying the notion of adjoint is that of a power supply it is with respect to this supply rate that the trajectories in the intersection are known to be stationary in this paper we deal with half line solutions to the differential equations governing both the system and its adjoint analysis of half line solutions plays a central role for example in initial value problems and in well posedness studies of an interconnection we interpret the set of half line trajectories allowed by a system and its adjoint as an interconnection of these two systems and thus address issues about well posedness ill posedness of the interconnection we formulate necessary and sufficient conditions for this intersection to be autonomous for the case of an ill posed interconnection and resulting autonomous system we derive conditions for existence of initial conditions that lead to impulsive solutions in the states of the system we link our conditions with the strongly reachable and weakly unobservable subspaces of a state space system we show that absence of impulsive initial conditions is equivalent to the well known subspace iteration algorithms for these subspaces converging in one step

deep narrow sigmoid belief networks are universal approximators in this note we show that exponentially deep belief networks can approximate any distribution over binary vectors to arbitrary accuracy even when the width of each layer is limited to the dimensionality of the data we further show that such networks can be greedily learned in an easy yet impractical way

pythagoras numbers of real irreducible algebroid curves and gram matrices abstract using elementary techniques of linear algebra in relation with gram matrices we bound the pythagoras number of a real irreducible algebroid curve by its multiplicity

what can the canonical controller in principle tell us given a plant and a desired specification our goal is to construct a controller system which when interconnected with the plant yields a system that behaves like the desired specification we can always construct the canonical controller introduced in van der schaft 2003 10 for linear systems there exists a controller which when interconnected to the plant yields the desired behaviour if and only if the canonical controller is itself one such controller see vinjamoor and van der schaft 2011 4 in this paper we extend this result to nonlinear systems it turns out that one has to look at the canonical controller together with its subsystems we obtain necessary and sufficient conditions for the existence of a controller for a class of nonlinear systems we end with examples which show that in certain cases looking at subsystems of the canonical controller also does not suffice

a fast learning algorithm for deep belief nets we show how to use complementary priors to eliminate the explaining away effects that make inference difficult in densely connected belief nets that have many hidden layers using complementary priors we derive a fast greedy algorithm that can learn deep directed belief networks one layer at a time provided the top two layers form an undirected associative memory the fast greedy algorithm is used to initialize a slower learning procedure that fine tunes the weights using a contrastive version of the wake sleep algorithm after fine tuning a network with three hidden layers forms a very good generative model of the joint distribution of handwritten digit images and their labels this generative model gives better digit classification than the best discriminative learning algorithms the low dimensional manifolds on which the digits lie are modeled by long ravines in the free energy landscape of the top level associative memory and it is easy to explore these ravines by using the directed connections to display what the associative memory has in mind

the achievable dynamics via control by interconnection we consider the problem of finding a controller such that when interconnected to the plant we obtain a system that is equivalent to a desired system here equivalence is formalized as bisimilarity we give necessary and sufficient conditions for the existence of such a controller the systems we consider are linear input state output systems a comparison is made to previously obtained results about achievable implementable behaviors in the behavioral approach to systems theory among the advantages of using the notion of bisimilarity is the fact that it directly applies to state space systems while the computations involved are operations on constant matrices

on the representational power of restricted boltzmann machines for symmetric functions and boolean functions restricted boltzmann machines rbms are used to build deep belief networks that are widely thought to be one of the first effective deep learning neural networks this paper studies the ability of rbms to represent distributions over 0 1 n via softplus hardplus rbm networks it is shown that any distribution whose density depends on the number of 1 s in their input can be approximated with arbitrarily high accuracy by an rbm of size 2n 1 which improves the result of a previous study by reducing the size from n 2 to 2n 1 a theorem for representing partially symmetric boolean functions by softplus rbm networks is established accordingly the representational power of rbms for distributions whose mass represents the boolean functions is investigated in comparison with that of threshold circuits and polynomial threshold functions it is shown that a distribution over 0 1 n whose mass represents a boolean function can be computed with a given margin delta by an rbm of size and parameters bounded by polynomials in n if and only if it can be computed by a depth 2 threshold circuit with size and parameters bounded by polynomials in n

universal approximation results for the temporal restricted boltzmann machine and the recurrent temporal restricted boltzmann machine the restricted boltzmann machine rbm has proved to be a powerful tool in machine learning both on its own and as the building block for deep belief networks multi layer generative graphical models the rbm and deep belief network have been shown to be universal approximators for probability distributions on binary vectors in this paper we prove several similar universal approximation results for two variations of the restricted boltzmann machine with time dependence the temporal restricted boltzmann machine trbm and the recurrent temporal restricted boltzmann machine rtrbm we show that the trbm is a universal approximator for markov chains and generalize the theorem to sequences with longer time dependence we then prove that the rtrbm is a universal approximator for stochastic processes with nite time dependence we conclude with a discussion on efficiency and how the constructions developed could explain some previous experimental results

totally dissipative systems in a totally dissipative behavior all non trivial trajectories dissipate energy a characterization of such behaviors is given in terms of properties of the one and two polynomial matrices associated with the supply rate and with their kernel and image representation

stabilization pole placement and regular implementability in this paper we study control by interconnection of linear differential systems we give necessary and sufficient conditions for regular implementability of a given linear differential system we formulate the problems of stabilization and pole placement as problems of finding a suitable regularly implementable sub behavior of the manifest plant behavior the problem formulations and their resolutions are completely representation free and specified only in terms of the system dynamics control is viewed as regular interconnection a controller is a system that constrains the plant behavior through a distinguished set of variables namely the control variables the issue of implementation of a controller in the feedback configuration and its relation to regularity of interconnection is addressed freedom of disturbances in a plant and regular interconnection with a controller also turn out to be inter related

on the parametrization of all regularly implementing and stabilizing controllers in this paper we deal with problems of controller parametrization in the context of behavioral systems given a full plant behavior a subbehavior of the manifest plant behavior is called regularly implementable if it can be achieved as the controlled behavior resulting from the interconnection of the full plant behavior with a suitable controller behavior in such a way that the controller does not impose restrictions that are already present in the plant we establish a parametrization of all controllers that regularly implement a given behavior we also obtain a parametrization of all stabilizing controllers

deep belief networks are compact universal approximators deep belief networks dbn are generative models with many layers of hidden causal variables recently introduced by hinton osindero and teh 2006 along with a greedy layer wise unsupervised learning algorithm building on le roux and bengio 2008 and sutskever and hinton 2008 we show that deep but narrow generative networks do not require more parameters than shallow ones to achieve universal approximation exploiting the proof technique we prove that deep but narrow feedforward neural networks with sigmoidal units can represent any boolean expression

impulse free interconnection of dynamical systems abstract in this paper we address the problem that impulses might occur when a to be controlled plant is connected to a suitable controller in the behavioral literature this issue is dealt when studying the so called regular feedback interconnection rfi of the plant and the controller behaviors we address the question of when a given controlled behavior satisfying desired specifications can be obtained by rfi of the plant and some suitable controller the paper presents an algorithm to construct a controller that yields the controlled behavior by rfi if such a controller exists and then parametrizes all such controllers we extend the results to the more general case when all the to be controlled variables might not be available for interconnection we show that any disturbances in the plant continue to remain free in the controlled behavior if and only if the interconnection is an rfi this is also equivalent to the well posedness of an interconnection this paper also makes concrete the intuitive relation between absence of inadmissibility of initial conditions and rfi a door closing mechanism is analysed in relation to the results in this paper

training products of experts by minimizing contrastive divergence it is possible to combine multiple latent variable models of the same data by multiplying their probability distributions together and then renormalizing this way of combining individual expert models makes it hard to generate samples from the combined model but easy to infer the values of the latent variables of each expert because the combination rule ensures that the latent variables of different experts are conditionally independent when given the data a product of experts poe is therefore an interesting candidate for a perceptual system in which rapid inference is vital and generation is unnecessary training a poe by maximizing the likelihood of the data is difficult because it is hard even to approximate the derivatives of the renormalization term in the combination rule fortunately a poe can be trained using a different objective function called contrastive divergence whose derivatives with regard to the parameters can be approximated accurately and efficiently examples are presented of contrastive divergence learning using several types of expert on several types of data

on the pythagoras numbers of real analytic curves we show that the pythagoras number of a real analytic curve is the supremum of the pythagoras numbers of its singularities or that supremum plus 1 this includes cases when the pythagoras number is infinite

refinements of universal approximation results for deep belief networks and restricted boltzmann machines we improve recently published results about resources of restricted boltzmann machines rbm and deep belief networks dbn required to make them universal approximators we show that any distribution on the set of binary vectors of length can be arbitrarily well approximated by an rbm with hidden units where is the minimal number of pairs of binary vectors differing in only one entry such that their union contains the support set of in important cases this number is half the cardinality of the support set of given in le roux bengio 2008 we construct a dbn with hidden layers of width that is capable of approximating any distribution on arbitrarily well this confirms a conjecture presented in le roux and bengio 2010

on interconnections control and feedback the purpose of this paper is to study interconnections and control of dynamical systems in a behavioral context we start with an extensive physical example which serves to illustrate that the familiar input output feedback loop structure is not as universal as we have been taught to believe this leads to a formulation of control problems in terms of interconnections subsequently we study interconnections of linear time invariant systems from this vantage point let us mention two of the results obtained the first one states that any polynomial can be achieved as the characteristic polynomial of the interconnection with a given plant provided the plant is not autonomous the second result states that any subsystem of a controllable system can be implemented by means of a singular feedback control law these results yield pole placement and stabilization of controllable plants as a special case these ideas are finally applied to the stabilization of a nonlinear system around an operating point

on the regular feedback interconnection problem we give a necessary and sufficient condition for a subsystem of a plant to be a regular feedback interconnection of an admissible controller with the plant we also parameterize the set of controllers that give rise to a desired controlled system

representational power of restricted boltzmann machines and deep belief networks deep belief networks dbn are generative neural network models with many layers of hidden explanatory factors recently introduced by hinton osindero and teh 2006 along with a greedy layer wise unsupervised learning algorithm the building block of a dbn is a probabilistic model called a restricted boltzmann machine rbm used to represent one layer of the model restricted boltzmann machines are interesting because inference is easy in them and because they have been successfully used as building blocks for training deeper models we first prove that adding hidden units yields strictly improved modeling power while a second theorem shows that rbms are universal approximators of discrete distributions we then study the question of whether dbns with more layers are strictly more powerful in terms of representational power this suggests a new and less greedy criterion for training rbms within dbns

stabilization lyapunov functions and dissipation for linear time invariant systems any stabilizing controller for a given plant can be associated with a supply rate with respect to which the plant in open loop is half line dissipative we also prove the equivalence between the stability of the interconnection of two systems and their half line dissipativity with respect to a supply rate and its negative

impulse controllability from descriptor systems to higher order daes impulsive solutions in lti dynamical systems have received ample attention but primarily for descriptor systems i e first order differential algebraic equations daes this paper focuses on the impulsive behavior of higher order dynamical systems and analyzes the causes of impulses in the context of interconnection of one or more dynamical systems we extend the definition of impulse controllability to the higher order case amongst the various nonequivalent notions of impulse controllability for first order systems available in the literature which mostly rely on the input output structure of the system our definition based on a so called state map obtained directly from the system equations generalizes many key first order results to the higher order case in particular we show that our higher order extension of the definition of impulse controllability generalizes the equivalence between impulse controllability and the ability to eliminate impulses in the closed loop by interconnecting with a suitable controller this requires an extension of the definition of regularity of interconnection from behaviors involving only smooth trajectories to behaviors on the positive half line involving impulsive smooth trajectories

output feedback pole placement with dynamic compensators derives a new rank condition which guarantees the arbitrary pole assignability of a given system by dynamic compensators of degree at most q by using this rank condition the authors establish several new sufficiency conditions which ensure the arbitrary pole assignability of a generic system the authors proofs also come with a concrete numerical procedure to construct a particular compensator which assigns a given set of closed loop poles

trajectory control and interconnection of 1d and nd systems in this paper we examine the relationship between control viewed as concatenation of trajectories and control viewed as interconnection of systems we show that for one dimensional linear time invariant systems the ability to obtain a given subsystem by regular interconnection a prerequisite for any feedback type structure is equivalent to the ability to drive any trajectory into that subsystem however in the case of multidimensional systems the former is a stronger property than the latter trajectory controllability can however be expressed as a regular interconnection of behaviors in an extended variable space by introducing latent or auxiliary variables this leads as a by product to the notion of controlling a system by means of latent variables