utility function estimation the entropy approach abstract the maximum entropy principle can be used to assign utility values when only partial information is available about the decision maker s preferences in order to obtain such utility values it is necessary to establish an analogy between probability and utility through the notion of a utility density function in this paper we explore the maximum entropy principle to estimate the utility function of a risk averse decision maker

nonlinearities enhance parameter convergence in output feedback systems while the parameter convergence properties of standard adaptive algorithms for linear systems are well established there are no similar results on the parameter convergence of adaptive controllers for nonlinear systems which have gained popularity in recent years in this paper we focus on a recently developed class of adaptive schemes for output feedback nonlinear systems and show that parameter convergence is guaranteed if and only if an appropriately defined signal vector which does not depend on closed loop signals is persistently exciting then we develop an analytic procedure which allows us given a specific nonlinear system and a specific reference signal to determine a priori whether or not this vector is persistently exciting pe and hence whether or not the parameter estimates will converge in the process we show that the presence of nonlinearities usually reduces the sufficient richness sr requirements on the reference signals and hence enhances parameter convergence this is the first result on the relationship between pe and sr for adaptive nonlinear control systems

identification of multivariable errors in variable models with dynamics this note considers the task of identifying a causal linear dynamic multivariable system excited by stationary zero mean noise of unknown spectrum and given measurements of the system inputs and outputs contaminated by independent additive noise also of unknown spectra although the solution is in general not unique finite dimensional parameterizations of the solution set are given even though the various spectra may not be rational

a bias correction method for identification of linear dynamic errors in variables models this paper considers the problem of identifying linear systems where the input is observed in white noise but the output is observed in colored noise which also includes process disturbances an efficient method is developed which can perform unbiased parameter estimation without utilizing a prefilter the developed method is characterized by attractive features direct use of the observed data without prefiltering no need to evaluate autocorrelation functions for the input noise no need to identify a high order augmented system and provision of a direct unbiased estimate of the system parameters without parameter extraction computer simulations are presented to illustrate its superior performance including its significantly reduced computational complexity

brief paper frequency domain maximum likelihood estimation of linear dynamic errors in variables models this paper studies the linear dynamic errors in variables problem for filtered white noise excitations first a frequency domain gaussian maximum likelihood ml estimator is constructed that can handle discrete time as well as continuous time models on a part s of the unit circle or imaginary axis next the ml estimates are calculated via a computationally simple and numerically stable gauss newton minimization scheme finally the cramer rao lower bound is derived

strong transitivity properties for operators abstract given a furstenberg family f of subsets of n an operator t on a topological vector space x is called f transitive provided for each non empty open subsets u v of x the set n z t n u v belongs to f we classify the topologically transitive operators with a hierarchy of f transitive subclasses by considering families f that are determined by various notions of largeness and density in z

dynamic errors in variables systems with three variables abstract errors in variables problems are considered for the case of three variables where the underlying relations among the noise free variables are linear and covariance information for the noisy variables is available the static problem where the variables are complex which can arise when narrowband filtering is used is analysed in detail some results are also presented for the case when the linear relations among the variables are defined by dynamic transfer functions

on line estimation of dynamic shock error models based on the kullback leibler information measure develops two sequential or on line estimation schemes in the time domain for dynamic shock error models which are special cases of errors in variables models the author s approach utilizes a state space representation of the model kalman filtering techniques and on line algorithms the first on line algorithm is based on the expectation maximization algorithm and uses a recursive gauss newton scheme to maximize the kullback leibler information measure the second on line algorithm the author proposes is a gradient based scheme and uses stochastic approximations to maximize the log likelihood in comparison to the off line maximum likelihood estimation scheme used in ghosh 1989 the author s on line algorithms have significantly reduced computational costs and negligible memory requirements simulations illustrate the satisfactory performance of the algorithms in estimating errors in variables systems with parameters that vary slowly with time or undergo infrequent jump changes

brief paper parameter convergence in adaptive extremum seeking control this paper addresses the problem of parameter convergence in adaptive extremum seeking control design an alternate version of the popular persistence of excitation condition is proposed for a class of nonlinear systems with parametric uncertainties the condition is translated to an asymptotic sufficient richness condition on the reference set point since the desired optimal set point is not known a priori in this type of problem the proposed method includes a technique for generating perturbation signal that satisfies this condition in closed loop this demonstrates its superiority in terms of parameter convergence the method guarantees parameter convergence with minimal but sufficient level of perturbation the effectiveness of the proposed method is illustrated with a simulation example

information theoretic framework for process control abstract this paper proposes a general framework for constructing control charts based on information theory the potential applications include developing information charts for monitoring moments and distributions of a process variable and process attributes in information theoretic process control itpc process moments are mapped to process distributions at in control and monitoring states and then to a control function via constrained maximization of entropy and minimization of kullback leibler function cross entropy variants of information charts can be developed without using distributional assumptions and based on a single criterion function the information discrepancy between two distributions an example of an information chart information mean variance chart imv chart for monitoring process mean and variance is developed the imv chart combines the standard x chart and s 2 chart and provides an information theoretic explication of the traditional procedures based on a run length study it is found that the imv chart singly possesses certain advantages over the standard two chart implementation of x chart and s 2 chart multivariate extension monitoring counts and proportions and monitoring distributional changes are briefly discussed

the set of observationally equivalent errors in variables models abstract linear dynamic errors in variables or factor models with uncorrelated noise components are considered we give a description of the set of all noise spectra which are oberservationally equivalent with respect to the second moments of observations

principal information theoretic approaches 2000 principal information theoretic approaches journal of the american statistical association vol 95 no 452 pp 1349 1353

renyi information measure for a used item recently in the literature a measure of tail heaviness has been proposed based on renyi entropy this measure is very useful in the sense that it can be used to measure tail heaviness even for the distributions for which the kurtosis measure b 2 m 4 m 2 2 does not exist nadarajah and zografos nadarajah zografos information sciences 153 2003 119 138 have derived the expression for this measure for different univariate continuous distributions but this measure can only be used for the lifetime of a new item in case of used item this measure needs some modification in this paper we have modified the measure accordingly so that it can be used in the case of used item and also for the new item we have also derived expressions for this measure for sixteen different univariate distributions and ten other standard distributions derived from the general distributions used in reliability and survival analysis the most of the results obtained in the literature in this direction can be obtained as particular cases of our general results

paper identification of scalar errors in variables models with dynamics the paper considers the task of identifying a causal linear dynamic system excited by stationary gaussian zero mean noise of unknown spectrum and given measurements of the system input and output contaminated by independent additive stationary noise signals of unknown spectra while the solution is normally not unique finite dimensional parameterizations of the solution set are given in some cases involving just one parameter even though the various spectra may not be rational

linear models based on noisy data and the frisch scheme we address the problem of identifying linear relations among variables based on noisy measurements this is a central question in the search for structure in large data sets often a key assumption is that measurement errors in each variable are independent this basic formulation has its roots in the work of charles spearman in 1904 and of ragnar frisch in the 1930s various topics such as errors in variables factor analysis and instrumental variables all refer to alternative viewpoints on this problem and on ways to account for the anticipated way that noise enters the data in the present paper we begin by describing certain fundamental contributions by the founders of the field and provide alternative modern proofs to certain key results we then go on to consider a modern viewpoint and novel numerical techniques to the problem the central theme is expressed by the frisch kalman dictum which calls for identifying a noise contribution that allows a maximal number of simultaneous linear relations a

from time series to linear system part iii approximate modelling abstract in the third part of this paper the problem of finding a linear time invariant complete system which models an observed time series will be continued however in this part it will be assumed that the model is required to explain the observations only approximately this raises the question of what is meant by an approximate model and how one should judge a model against data which in a strict sense falsify the model the initial sections of this paper introduce the methodology either the maximal admissible complexity is fixed and the best fitting model in the model class is sought or the maximally tolerated misfit is fixed and the least complex model in the model class is sought this is illustrated by means of finite dimensional models where natural complexity and misfit functions suggest themselves subsequently what is meant by the complexity of a linear time invariant complete system is discussed a concrete complexity function is proposed which is defined in terms of the richness of the behaviour on finite time intervals and is in one to one correspondence with the number of inputs the number of states and the observability indices of the underlying minimal i s o system the next definition is that of the misfit between an observed time series and a system a concrete misfit function is proposed which takes into consideration the lag structure of the system and in how far the observations fail to corroborate the ar equations which define the system s behaviour how the misfit may be computed by means of the correlation function of the data is then shown these concepts are subsequently used in order to set up two algorithms for approximate modelling of an observed q dimensional time series the first algorithm fixes the maximal complexity in a sense this comes down to fixing the number and the lags of the required ar equations the second algorithm fixes the tolerated misfit it is adaptive in nature in the sense that the number and the structure of the resulting ar equations will depend on the observations these ideas and algorithms are illustrated by means of two simulated examples

comment on identification in the linear errors in variables model on considere un modele etudie par kapteyn et wansbeek 1983 et on presente des conditions necessaires et suffisantes d identification

expressions for renyi and shannon entropies for bivariate distributions exact forms of renyi and shannon entropies are determined for 27 continuous bivariate distributions including the kotz type distribution truncated normal distribution distributions with normal and centered normal conditionals natural exponential distribution freund s exponential distribution marshall and olkin s exponential distribution exponential mixture distribution arnold and strauss s exponential distribution mckay s gamma distribution distribution with gamma conditionals gamma exponential distribution dirichlet distribution inverted beta distribution distribution with beta conditionals beta stacy distribution cuadras and auge s distribution farlie gumbel morgenstern distribution logistic distribution pearson type vii distribution pearson type ii distribution distribution with cauchy conditionals bilateral pareto distribution muliere and scarsini s pareto distribution distribution with pareto conditionals and the distribution with gumbel conditionals we believe that the results presented here will serve as an important reference for scientists and engineers in many areas

identification of the dynamic shock error model with autocorrelated errors abstract the paper gives necessary and sufficient conditions for the local identifiability of dynamic regression models with autocorrelated errors in the variables the conditions are simple counting rules combining the order parameters of a model and directly generalize the results of maravall and aigner a new method of identification is presented allowing a compact derivation of the results

stochastic system identification with noisy input output measurements using polyspectra two new classes of parametric frequency domain approaches are proposed for estimation of the parameters of scalar linear errors in variables models i e linear systems where measurements of both input and output of the system are noise contaminated the first approach consists of linear estimators where using the bispectrum or the integrated polyspectrum of the input and the cross bispectrum or the integrated cross polyspectrum the system transfer function is first estimated at a number of frequencies exceeding one half the number of unknown parameters the estimated transfer function is then used to estimate the unknown parameters using an overdetermined linear system of equations in the second class of approaches quadratic transfer function matching criteria are optimized by using the results of the linear estimators as initial guesses both classes of the parameter estimators are shown to be consistent in any measurement noise that has symmetric probability density function when the bispectral approaches are used the proposed parameter estimators are shown to be consistent in gaussian measurement noise when trispectral approaches are used

identification of dynamic errors in variables models this paper deals with the identifiability of scalar dynamic errors in variables models characterized by rational spectra the hypothesis of causality for the underlying dynamic system is taken into account by making use of stochastic realization theory and of the structural properties of state space representations it is shown that under mild assumptions the model is uniquely identified

extremum seeking for static maps with delays in this paper we address the design and analysis of multi variable extremum seeking for static maps subject to arbitrarily long time delays both gradient and newton based methods are considered multi input systems with different time delays in each individual input channel as well as output delays are dealt with the phase compensation of the dither signals and the inclusion of predictor feedback with a perturbation based averaging based estimate of the hessian allow to obtain local exponential convergence results to a small neighborhood of the optimal point even in the presence of delays the stability analysis is carried out using backstepping transformation and averaging in infinite dimensions capturing the infinite dimensional state due the time delay in particular a new backstepping like transformation is introduced to design the predictor for the gradient based extremum seeking scheme with multiple and distinct input delays the proposed newton based extremum seeking approach removes the dependence of the convergence rate on the unknown hessian of the nonlinear map to be optimized being user assignable as in the literature free of delays a source seeking example illustrates the performance of the proposed delay compensated extremum seeking schemes

unimodular eigenvalues uniformly distributed sequences and linear dynamics we study increasing sequences of positive integers nk k 1 with the following property every bounded linear operator t acting on a separable banach or hilbert space with supk 1 tnk has a countable set of unimodular eigenvalues whether this property holds or not depends on the distribution modulo one of sequences nk k 1 r or on the growth of nk 1 nk counterexamples to some conjectures in linear dynamics are given for instance a hilbert space operator which is frequently hypercyclic chaotic but not topologically mixing is constructed the situation of c0 semigroups is also discussed

identifiability in multivariate dynamic linear errors in variables models abstract this article considers multivariate causal transfer function systems with latent stationary inputs and outputs their observation is assumed to be disturbed by errors in variables ev the main identification results for such models so far consist of structure theory derived are descriptions of classes of ev transfer function systems compatible with given covariance structures of the observed variables under various causality constraints the case of unique determination is treated only for one dimensional models the article derives a great variety of conditions under which a multivariate ev transfer function system is uniquely determined by the second order moments of the observed variables for this purpose it considers a general model class of nonparametric systems and subclasses of systems with some parameterized components having fixed order parameters if parametric the transfer function is a rational matrix parametric inputs or errors follow vector autoregressive moving average arma

invariant gaussian measures for operators on banach spaces and linear dynamics we give conditions for an operator t on a complex separable banach space x with sufficiently many eigenvectors associated to eigenvalues of modulus 1 to admit a non degenerate invariant gaussian measure with respect to which it is weak mixing the existence of such a measure depends on the geometry of the banach space and on the possibility of parametrizing the mathbb t eigenvector fields of t in a regular way we also investigate the connection with frequent hypercyclicity

linear subsets of nonlinear sets in topological vector spaces for the last decade there has been a generalized trend in mathematics on the search for large algebraic structures linear spaces closed subspaces or infinitely generated algebras composed of mathematical objects enjoying certain special properties this trend has caught the eye of many researchers and has also had a remarkable influence in real and complex analysis operator theory summability theory polynomials in banach spaces hypercyclicity and chaos and general functional analysis this expository paper is devoted to providing an account on the advances and on the state of the art of this trend nowadays known as lineability and spaceability

simulation estimation of time series models abstract a formal econometric treatment of the estimation of the parameters of a fully specified stochastic equilibrium model is proposed the method estimation by simulation yields an estimator which is shown to have an asymptotic normal distribution a goodness of fit test based on a chi square statistic is also derived

minimum seeking for clfs universal semiglobally stabilizing feedback under unknown control directions employing extremum seeking es for seeking minima of control lyapunov function clf candidates we develop 1 the first systematic design of es controllers for unstable plants 2 a simple non model based universal feedback law that emulates in an average sense the l g v controllers for stabilization with inverse optimality and 3 a new strategy for stabilization of systems with unknown control directions as an alternative to nussbaum gain controllers that lack exponential stability lack transient performance guarantees and lack robustness to changes in the control direction the stability analysis that underlies our designs is inspired by an analysis approach synthesized in a recent work by durr stankovic and johansson which combines a lie bracket averaging result of gurvits and li with a semiglobal practical stability result under small parametric perturbations by moreau and aeyels

hypercyclicity and unimodular point spectrum abstract we show that an operator on a separable complex banach space with sufficiently many eigenvectors associated to eigenvalues of modulus 1 is hypercyclic we apply this result to construct hypercyclic operators with prescribed k unimodular point spectrum we show how eigenvectors associated to unimodular eigenvalues can be used to exhibit common hypercyclic vectors for uncountable families of operators and prove that the family of composition operators c on h 2 d where is a disk automorphism having 1 as attractive fixed point has a residual set of common hypercyclic vectors

linear dynamic errors in variables models some structure theory abstract this paper gives a survey of some recent results on problems of identifiability or more general of the relation between the observations and certain system characteristics for linear dynamic errors in variables models for a large part of the paper the noise components are assumed to be mutually uncorrelated after the general problem statement a rather complete analysis of the single input single output case is given also the case of three variables and the case where the number of inputs is equal to the number of outputs are discussed in detail finally the use of higher order cummulant spectra for identifiability is investigated

operators with the specification property abstract we study a version of the specification property for linear dynamics operators having the specification property are investigated and relationships with other well known dynamical notions such as mixing devaney chaos and frequent hypercyclicity are obtained

operators with dense invariant cyclic vector manifolds abstract we study a class of banach space operators patterned after the weighted backward shifts on hilbert space and show that any non scalar operator in the commutant of one of these generalized backward shifts has a dense invariant linear manifold whose non zero members are cyclic vectors under appropriate extra hypotheses on the commuting operator stronger forms of cyclicity are possible the most extreme being hypercyclicity density of an orbit motivated by these results we examine the cyclic behavior of two seemingly unrelated classes of operators adjoint multiplications on hilbert spaces of holomorphic functions and differential operators on the frechet space of entire functions we show that each of these operators other than the scalar multiples of the identity possesses a dense invariant linear submanifold each of whose non zero elements is hypercyclic finally we explore some connections with dynamics many of the hypercyclic operators discussed here are in at least one of the commonly accepted senses of the word chaotic

the rank of reduced dispersion matrices psychometricians working in factor analysis and econometricians working in regression with measurement error in all variables are both interested in the rank of dispersion matrices under variation of the diagonal elements psychometricians concentrate on cases in which low rank can be attained preferably rank one the spearman case econometricians cocentrate on cases in which the rank cannot be reduced below the number of variables minus one the frisch case in this paper we give an extensive historial discussion of both fields we prove the two key results in a more satisfactory and uniform way we point out various small errors and misunderstandings and we present a methodological comparison of factor analysis and regression on the basis of our results

from time series to linear system part i finite dimensional linear time invariant systems abstract in the first part of this paper the definition of a dynamical system as simply consisting of a family of time series will be developed in this context the notions of linearity time invariance and finite dimensionality will be introduced it will be shown that a given family of time series may be represented by a system of ar equations riw t l rl 1w t l 1 r0w t 0 or equivalently by a finite dimensional linear time invariant system x t 1 ax t bu t y t cx t du t w u y if and only if this family is linear shift invariant and complete or as is equivalent closed in the topology of pointwise convergence this yields a very high level and elegant set of axioms which characterize these familiar objects it is emphasized however that no a priori choice is made as to which components of w are inputs and which are outputs such a separation always exists in any specific linear time invariant model starting from these definitions the structural indices of such systems are introduced and it is shown how an ar representation of a system having a given behaviour can be constructed these results will be used in a modelling context in part ii of the paper

dynamics of the differentiation operator on weighted spaces of entire functions the continuity of the differentiation operator on weighted banach spaces of entire functions with sup norm has been characterized recently by harutyunyan and lusky we give necessary and sufficient conditions to ensure that the differentiation operator on these weighted banach spaces of entire functions is hypercyclic or chaotic when it is continuous

almost sure convergence of extremum seeking algorithm using stochastic perturbation abstract this paper presents an extremum seeking es algorithm where the perturbation signal is a martingale difference sequence m d s with a vanishing variance the measurement noise at the plant output is modeled by a superposition of deterministic component and a non stationary colored noise signal the optimizing set point of the uncertain reference to output equilibrium map is estimated by a stochastic approximation sa type algorithm the algorithm has a vanishing gain sequence dependent on the set point estimates by utilizing powerful tools of the martingale convergence theory it is proved that with probability one the set point estimates converge to the optimizing equilibrium point in spite of the presence of a measurement noise this result is derived without requiring boundedness or any prior condition on the set point estimates

an algorithm for the minimization of mixed l sub 1 and l sub 2 norms with application to bayesian estimation the regularizing functional approach is widely used in many estimation problems in practice the solution is defined as one minimum point of a suitable functional the main part of which accounts for the underlying physical model whereas the regularizing part represents some prior information about the unknowns in the bayesian interpretation one has a maximum a posteriori map estimator in which the main and regularizing parts are represented respectively by likelihood and prior distributions when either the prior or likelihood is a laplace distribution and the other is a gaussian distribution one is led to consider functionals that include both absolute and square norms the authors present a characterization of the minimum points of such functionals together with a descent type algorithm for numerical computations the results of monte carlo simulations are also reported

distributional chaos for linear operators abstract we characterize distributional chaos for linear operators on frechet spaces in terms of a computable condition dcc and also as the existence of distributionally irregular vectors a sufficient condition for the existence of dense uniformly distributionally irregular manifolds is presented which is very general and can be applied to many classes of operators distributional chaos is also analyzed in connection with frequent hypercyclicity and the particular cases of weighted shifts and composition operators are given as an illustration of the previous results

survival exponential entropies the multivariate survival function of a random vector x is used to define a broad class of entropy measures several properties of the proposed class are studied and explicit expressions of the measures derived for specific probabilistic models the cumulative residual entropy introduced by rao et al and wang et al is a particular case of the proposed class of measures

hypercyclic weighted shifts an operator t acting on a hilbert space is hypercyclic if for some vector x in the space the orbit tnx n 0 is dense in this paper we characterize hypercyclic weighted shifts in terms of their weight sequences and identify the direct sums of hypercyclic weighted shifts which are also hypercyclic as a consequence we show within the class of weighted shifts that multi hypercyclic shifts and direct sums of fixed hypercyclic shifts are both hypercyclic for general hypercyclic operators the corresponding questions were posed by d a herrero and they still remain open using a different technique we prove that i t is hypercycic whenever t is a unilateral backward weighted shift thus answering in more generality a question recently posed by k c chan and j h shapiro

hypercyclic operators that commute with the bergman backward shift the backward shift b on the bergman space of the unit disc is known to be hypercyclic meaning it has a dense orbit here we ask which operators that commute with b inherit its hypercyclicity we show that the problem reduces to the study of operators of the form 9p b where so is a holomorphic self map of the unit disc that multiplies the dirichlet space into itself and that the question of hypercyclicity for such an operator depends on how freely 9p z is allowed to approach the unit circle as lz s 1

persistent excitation in bilinear systems discrete time systems described by difference equations which are polynomial in the input and linear in the output are discussed bilinear systems are special members of this class two issues are examined first a multidimensional polynomial based algebraic condition which is necessary and sufficient for such systems to be identifiable is given second subject to an identifiability assumption a condition on the input sequence which guarantees persistent excitation is given the principle analytic tool used in this study involves multidimensional polynomials

brief paper speed regulation in steering based source seeking the simplest strategy for extremum seeking based source localization for sources with unknown spatial distributions and nonholonomic unicycle vehicles without position measurement employs a constant positive forward speed steering of the vehicle in the plane is performed using only the variation of the angular velocity while keeping the forward speed constant is a reasonable strategy motivated by implementation with aerial vehicles it leads to complexities in the asymptotic behavior of the vehicle since the vehicle cannot settle at best it can converge to a small size attractor around the source in this paper we regulate the forward velocity with the intent of bringing the vehicle to a stop or as close to a stop as possible the vehicle speed is controlled using simple derivative like feedback of the sensor measurement the derivative is approximated with a washout filter to which a speed bias parameter v c is added the angular velocity is tuned using standard extremum seeking we prove two results for v c in a certain range around zero we show that the vehicle converges to a ring around the source and on average the limit of the vehicle s heading is either directly away or towards the source for other values of v c 0 the vehicle converges to a ring around the source and it revolves around the source interestingly the average heading of this revolution around the source is more outward than inward this is possible because the vehicle s speed is not constant it is lower during the outward steering intervals and higher during the inward steering intervals the theoretical results are illustrated with simulations

estimation of quantized linear errors in variables models we present an estimation algorithm for dynamic shock error models dsem given 1 bit quantized noisy measurements of the input and output the algorithm is called the binary series estimation algorithm bsea bsea is computationally inexpensive since it involves counting the number of occurrences of particular patterns of zeros and ones in the input and output sequences it yields consistent parameter estimates that are asymptotically normal

brief adaptive extremum seeking control of nonlinear dynamic systems with parametric uncertainties we pose and solve an extremum seeking control problem for a class of nonlinear systems with unknown parameters extremum seeking controllers are developed to drive system states to the desired set points that extremize the value of an objective function the proposed adaptive extremum seeking controller is inverse optimal in the sense that it minimizes a meaningful cost function that incorporates penalty on both the performance error and control action simulation studies are provided to verify the effectiveness of the proposed approach

invariant measures for frequently hypercyclic operators abstract we investigate frequently hypercyclic and chaotic linear operators from a measure theoretic point of view among other things we show that any frequently hypercyclic operator t acting on a reflexive banach space admits an invariant probability measure with full support which may be required to vanish on the set of all periodic vectors for t that there exist frequently hypercyclic operators on the sequence space c 0 admitting no ergodic measure with full support and that if an operator admits an ergodic measure with full support then it has a comeager set of distributionally irregular vectors we also give some necessary and sufficient conditions which are satisfied by all the known chaotic operators for an operator t to admit an invariant measure supported on the set of its hypercyclic vectors and belonging to the closed convex hull of its periodic measures finally we give a baire category proof of the fact that any operator with a perfectly spanning set of unimodular eigenvectors admits an ergodic measure with full support

information theory as a unifying statistical approach for use in marketing research abstract information theory is shown to provide a unified approach to a wide range of problems in marketing research for instance this approach can be used to obtain characterizations parallel to those of the hendry system and other entropic approaches with great economy of assumptions and with the added flexibility that constraints can be easily identified for explicit consideration and implemented as needed goodness of fit tests and decision modelling structures are supplied from these same stochastic models with a range of applications that include market segmentation and brand shifting choices a basic approach to these and other procedures is therefore obtainable from information theoretic methods these methods can be used to address stochastic model selection problems and other probabilistic models of marketing choice for example minimum discrimination information mdi estimation logit multiplicative competitive interaction mci and other important choice models are also shown in this paper to arise naturally from information theoretic formulations with duality relations developed by charnes and cooper providing additional simplifications and interpretations

finite time parameter estimation in adaptive control of nonlinear systems this note presents a parameter estimation routine that allows exact reconstruction of the unknown parameters in finite time provided a given excitation condition is satisfied the robustness of the routine to an unknown bounded disturbance or modeling error is also shown the result is independent of the control and identifier structures employed the true parameter value is obtained without requiring the measurement or computation of the velocity state vector moreover the technique provides a direct solution to the problem of removing auxiliary perturbation signals when parameter convergence is achieved

differential log likelihood for evaluating and learning gaussian mixtures we introduce a new unbiased metric for assessing the quality of density estimation based on gaussian mixtures called differential log likelihood as an application we determine the optimal smoothness and the optimal number of kernels in gaussian mixtures furthermore we suggest a learning strategy for gaussian mixture density estimation and compare its performance with log likelihood maximization for a wide range of real world data sets

estimating linear statistical relationships this paper on estimating linear statistical relationships includes three lectures on linear functional and structural relationships factor analysis and simultaneous equations models the emphasis is on relating the several models by a general approach and on the similarity of maximum likelihood estimators under normality in the different models in the first two lectures the observable vector is decomposed into a systematic part and a random error the systematic part satisfies the linear relationships estimators are derived for several cases and some of their properties given estimation of the coefficients of a single equation in a simultaneous equations model is shown to be equivalent to estimation of linear functional relationships

information distinguishability with application to analysis of failure data abstract in maximum entropy me modeling the information discrepancy between two distributions is measured in terms of their entropy difference in discrimination information statistics the information discrepancy between two distributions is measured in terms of the kullback leibler function i e relative entropy or cross entropy this article presents an equivalence between kullback leibler functions and entropy differences involving an me distribution based on this equivalence the concept of information discrimination id distinguishability is introduced as a unifying framework for the two methods of measuring information discrepancy between distributions applications of id distinguishability as diagnostics for examining robustness of parametric procedures and sensitivity of nonparametric statistics across parametric families of distributions is proposed the equivalence result facilitates estimation of kullback leibler functions in terms of entropy estimates application of the id distinguish

some new examples of universal hypercyclic operators in the sense of glasner and weiss a bounded operator on a real or complex separable infinite dimensional banach space z is universal in the sense of glasner and weiss if for every invertible ergodic measure preserving transformation t of a standard lebesgue probability space x b there exists an a invariant probability measure on h with full support such that the two dynamical systems x b t and h bh a are isomorphic we present a general and simple criterion for an operator to be universal which allows us to characterize universal operators among unilateral or bilateral weighted shifts on p or c0 show the existence of universal operators on a large class of banach spaces and give a criterion for universality in terms of unimodular eigenvectors we also obtain similar results for operators which are universal for all ergodic systems not only for invertible ones and study necessary conditions for an operator on a hilbert space to be universal

a framework for extremum seeking control of systems with parameter uncertainties traditionally the design of extremum seeking algorithm treats the system as essentially a black box which for many applications means disregarding known information about the model structure in contrast to this approach there have been recent examples where a known plant structure with uncertain parameters has been used in the online optimization of plant operation however the results for these approaches have been restricted to specific classes of plants and optimization algorithms this paper seeks to provide general results and a framework for the design of extremum seekers applied to systems with parameter uncertainties general conditions for an optimization method and a parameter estimator are presented so that their combination guarantees convergence of the extremum seeker for both static and dynamic plants tuning guidelines for the closed loop scheme are also presented the generality and flexibility of the proposed framework is demonstrated through a number of parameter estimators and optimization algorithms that can be combined to obtain extremum seeking examples of anti lock braking and model reference adaptive control are used to illustrate the effectiveness of the proposed framework

capturing the intangible concept of information abstract the purpose of this article is to discuss the intricacies of quantifying information in some statistical problems the aim is to develop a general appreciation for the meanings of information functions rather than their mathematical use this theme integrates fundamental aspects of the contributions of kullback lindley and jaynes and bridges chaos to probability modeling a synopsis of information theoretic statistics is presented in the form of a pyramid with shannon at the vertex and a triangular base that signifies three distinct variants of quantifying information discrimination information kullback mutual information lindley and maximum entropy information jaynes examples of capturing information by the maximum entropy me method are discussed it is shown that the me approach produces a general class of logit models capable of capturing various forms of sample and nonsample information diagnostics for quantifying information captured by the me logit models are given and decom

global identification of the dynamic shock error model abstract the paper gives necessary and sufficient conditions for the global identifiability of dynamic regression models with errors in the variables the conditions are simple counting rules combining the order parameters of a model they are counterparts of conditions previously established for local identifiability in a series of papers

brief stability of extremum seeking feedback for general nonlinear dynamic systems while the mainstream methods of adaptive control both linear and nonlinear deal only with regulation to known set points or reference trajectories in many applications the set point should be selected to achieve a maximum of an uncertain reference to output equilibrium map the techniques of the so called extremum control or self optimizing control developed for this problem in the 1950 1960s have long gone out of fashion in the theoretical control literature because of the difficulties that arise in a rigorous analytical treatment in this paper we provide the first proof of stability of an extremum seeking feedback scheme by employing the tools of averaging and singular perturbation analysis our scheme is much more general that the existing extremum control results which represent the plant as a static nonlinear map possibly cascaded with a linear dynamic block we allow the plant to be a general nonlinear dynamic system possibly non affine in control and open loop unstable whose reference to output equilibrium map has a maximum and whose equilibria are locally exponentially stabilizable

brief paper on the identifiability of errors in variables models with white measurement errors we discuss identifiability of dynamic siso errors in variables eiv models with white measurement errors although this class of models turns out to be generically identifiable it has been pointed out that in certain circumstances there may be two eiv models which are indistinguishable from external input output experiments this lack of global identifiability may be prejudicial to identification and needs better understanding the identifiability conditions found in the literature guarantee uniqueness under certain coprimality assumptions on the rational transfer function of the ideal true system and the spectral density of the noiseless true input unfortunately these conditions are not testable since they concern precisely the unknowns of the problem which are not available to the experimenter we provide new identifiability conditions which are instead expressible in terms of the external description of the observable signals namely their joint power spectral densities

efficient method of moments estimation of a stochastic volatility model a monte carlo study abstract we perform an extensive monte carlo study of efficient method of moments emm estimation of a stochastic volatility model emm uses the expectation under the structural model of the score from an auxiliary model as moment conditions we examine the sensitivity to the choice of auxiliary model using arch garch and egarch models for the score as well as nonparametric extensions emm efficiency approaches that of maximum likelihood for larger sample sizes inference is sensitive to the choice of auxiliary model in small samples but robust in larger samples specification tests and t tests show little size distortion

information indices unification and applications the unified framework of information theoretic statistics was established by kullback 1959 since then numerous information indices have been developed in various contexts this paper represents many of these indices in a unified context the unification thread is the discrimination information function information indices are all logarithmic measures of discrepancy between two probability distributions first we present a summary of informational aspects of the basic information functions a unification of various information theoretic modeling approaches and some explication in terms of traditional measures we then tabulate a unified representation of assortments of information indices developed in the literature for maximum entropy modeling covariate information and influence diagnostics the subjects of these indices include parametric model fitting nonparametric entropy estimation categorical data analysis the linear and exponential family regression and time series the coverage however is not exhaustive the tabulation includes sampling theory and bayesian indices but the focus is on interpretation as descriptive measures and inferential properties are noted tangentially finally applications of some information indices are illustrated through modeling duration data for sprint s churned customer and choice of long distance provider

recurrence properties of hypercyclic operators we generalize the notions of hypercyclic operators mathfrak u frequently hypercyclic operators and frequently hypercyclic operators by introducing a new concept in linear dynamics namely mathcal a hypercyclicity we then state an mathcal a hypercyclicity criterion inspired by the hypercyclicity criterion and the frequent hypercyclicity criterion and we show that this criterion characterizes the mathcal a hypercyclicity for weighted shifts we also investigate which density properties can the sets n x u n in mathbb n t nx in u have for a given hypercyclic operator and we study the new notion of reiteratively hypercyclic operators

estimating continuous time stochastic volatility models of the short term interest rate abstract we obtain consistent parameter estimates of continuous time stochastic volatility diffusions for the u s risk free short term interest rate sampled weekly over 1954 1995 using the efficient method of moments procedure of gallant and tauchen the preferred model displays mean reversion and incorporates level effects and stochastic volatility in the diffusion function extensive diagnostics indicate that the cox ingersoll ross model with an added stochastic volatility factor provides a good characterization of the short rate process further they suggest that recently proposed garch models fail to approximate the discrete time short rate dynamics while level egarch models perform reasonably well

qualitative and asymptotic performance of snp density estimators abstract the snp estimator is the most convenient nonparametric method for simultaneously estimating the parameters of a nonlinear model and the density of a latent process by maximum likelihood to determine if this convenience comes at a price we assess the qualitative behavior of snp in finite samples using the marron wand test suite and verify theoretical convergence rates by monte carlo simulation our results suggest that there is no price for convenience because the snp estimator is both qualitatively and asymptotically similar to the kernel estimator which is optimal

the shape of neural dependence the product moment correlation coefficient is often viewed as a natural measure of dependence however this equivalence applies only in the context of elliptical distributions most commonly the multivariate gaussian where linear correlation indeed sufficiently describes the underlying dependence structure should the true probability distributions deviate from those with elliptical contours linear correlation may convey misleading information on the actual underlying dependencies it is often the case that probability distributions other than the gaussian distribution are necessary to properly capture the stochastic nature of single neurons which as a consequence greatly complicates the construction of a flexible model of covariance we show how arbitrary probability densities can be coupled to allow greater flexibility in the construction of multivariate neural population models

formulas for renyi information and related measures for univariate distributions in a recent paper song j stat plan infer 93 2001 51 considered renyi information of order and established its connection to the loglikelihood from this relation an intrinsic distribution measure was proposed and analytic expressions of this measure and renyi information were derived for some standard continuous distributions in this paper we derive analytical formulas for renyi and shannon entropies as well as for song s measure for 26 flexible families of univariate continuous distributions we believe that the results presented here will serve as an important reference for scientists and engineers in many areas

the identification of multivariate linear dynamic errors in variables models abstract the paper establishes necessary and sufficient conditions for the local identifiability of multivariate linear time series models with measurement errors the models have a linear relation with rational transfer matrix function and variables specified by unrestricted vector arma processes of fixed orders or more general variables the identifiability conditions finally are simple counting rules combining the order parameters of a model the paper derives the results bydisplaying in detail a new identification method which is based on the factorization and the expansion into partial fractions of rational matrices

extremum seeking control of state constrained nonlinear systems an extremum seeking control problem is posed for a class of nonlinear systems with unknown dynamical parameters whose states are subject to convex pointwise inequality constraints using a barrier function approach an adaptive method is proposed for generating setpoints online which converge to the feasible minimizer of a convex objective function containing the unknown dynamic parameters a tracking controller regulates system states to the generated setpoint via state feedback while maintaining feasibility of the state constraints a simulation example demonstrates application of the method

hilbertian jamison sequences and rigid dynamical systems a strictly increasing sequence nk k 0 of positive integers is said to be a hilbertian jamison sequence if for any bounded operator t on a separable hilbert space such that supk 0 tnk the set of eigenvalues of modulus 1 of t is at most countable we first give a complete characterization of such sequences we then turn to the study of rigidity sequences nk k 0 for weakly mixing dynamical systems on measure spaces and give various conditions some of which are closely related to the jamison condition for a sequence to be a rigidity sequence we obtain on our way a complete characterization of topological rigidity and uniform rigidity sequences for linear dynamical systems and we construct in this framework examples of dynamical systems which are both weakly mixing in the measure theoretic sense and uniformly rigid

identification of multivariable stochastic linear systems via polyspectral analysis given noisy input output time domain data the paper considers the problem of identification of unknown parameters of multivariable linear errors invariables models attention is focused on frequency domain approaches where the integrated polyspectrum bispectrum or trispectrum of the input and the integrated cross polyspectrum respectively of the given time domain input output data are exploited two new classes of parametric frequency domain approaches are proposed and analyzed an integrated polyspectrum based persistence of excitation condition on system input is defined both classes of the parameter estimators are shown to be strongly consistent in any measurement noise sequences with vanishing bispectra when integrated bispectrum based approaches are used the proposed parameter estimators are shown to be strongly consistent in gaussian measurement noise when integrated trispectrum based approaches are used the input to the system need not be a linear process but must have nonvanishing bispectrum or trispectrum

reprojecting partially observed systems with application to interest rate diffusions abstract we introduce reprojection as a general purpose technique for characterizing the dynamic response of a partially observed nonlinear system to its observable history reprojection is the third step of a procedure wherein first data are summarized by projection onto a hermite series representation of the unconstrained transition density for observables second system parameters are estimated by minimum chi squared where the chi squared criterion is a quadratic form in the expected score of the projection and third the constraints on dynamics implied by the nonlinear system are imposed by projecting a long simulation of the estimated system onto a hermite series representation of the constrained transition density for observables the constrained transition density can be used to study the response of the system to its observable history we utilize the technique to assess the dynamics of several diffusion models for the short term interest rate that have been proposed and to compare them to a ne

some aspects of the history of bayesian information processing for many years traditional bayesian tb and information theoretic it procedures for learning from data were viewed as distinctly different approaches derivations of the tb and it learning models are reviewed and compared then the synthesis by zellner in 1988 of the tb and it learning models and generalizations of them are described along with descriptions of selected applications included are learning procedures that do not require use of likelihood functions and or priors works by leading bayesians and information theorists are cited and related to tb it issues

principal components analysis of sampled functions this paper describes a technique for principal components analysis of data consisting ofn functions each observed atp argument values this problem arises particularly in the analysis of longitudinal data in which some behavior of a number of subjects is measured at a number of points in time in such cases information about the behavior of one or more derivatives of the function being sampled can often be very useful as for example in the analysis of growth or learning curves it is shown that the use of derivative information is equivalent to a change of metric for the row space in classical principal components analysis the reproducing kernel for the hilbert space of functions plays a central role and defines the best interpolating functions which are generalized spline functions an example is offered of how sensitivity to derivative information can reveal interesting aspects of the data

adapting to unknown smoothness via wavelet shrinkage abstract we attempt to recover a function of unknown smoothness from noisy sampled data we introduce a procedure sureshrink that suppresses noise by thresholding the empirical wavelet coefficients the thresholding is adaptive a threshold level is assigned to each dyadic resolution level by the principle of minimizing the stein unbiased estimate of risk sure for threshold estimates the computational effort of the overall procedure is order n log n as a function of the sample size n sureshrink is smoothness adaptive if the unknown function contains jumps then the reconstruction essentially does also if the unknown function has a smooth piece then the reconstruction is essentially as smooth as the mother wavelet will allow the procedure is in a sense optimally smoothness adaptive it is near minimax simultaneously over a whole interval of the besov scale the size of this interval depends on the choice of mother wavelet we know from a previous paper by the authors that traditional smoot

a structure theory for linear dynamic errors in variables models we deal with problems connected with the identification of linear dynamic systems in situations when inputs and outputs may be contaminated by noise the case of uncorrelated noise components and the bounded noise case is considered if also the inputs may be contaminated by noise a number of additional complications in identification arise in particular the underlying system is not uniquely determined from the population second moments of the observations a description of classes of observationally equivalent systems is given continuity properties of mappings relating classes of observationally equivalent systems to the spectral densities of the observations are derived and the classes of spectral densities corresponding to a given maximum number of outputs are studied

a perturbation based proportional integral extremum seeking control approach this paper proposes a proportional integral extremum seeking control technique the technique is a generalization of the standard perturbation based techniques which provides fast transient performance of the closed loop system to the optimum equilibrium of a measured cost function under the assumption that the equations describing the dynamics of the nonlinear system and the cost function to be minimized are unknown it is shown that for a class of minimum phase nonlinear system with a cost function of relative degree one the closed loop esc can practically stabilize the unknown equilibrium optimum the stabilization result provides an alternative approach to output feedback control of nonlinear systems the effectiveness of the proposed approach is demonstrated using simulation examples

frequently hypercyclic operators we investigate the subject of linear dynamics by studying the notion of frequent hypercyclicity for bounded operators t on separable complex f spaces t is frequently hypercyclic if there exists a vector x such that for every nonempty open subset u of x the set of integers n such that t n x belongs to u has positive lower density we give several criteria for frequent hypercyclicity and this leads us in particular to study linear transformations from the point of view of ergodic theory several other topics which are classical in hypercyclicity theory are also investigated in the frequent hypercyclicity setting

strong mixing measures for linear operators and frequent hypercyclicity abstract we construct strongly mixing invariant measures with full support for operators on f spaces which satisfy the frequent hypercyclicity criterion for unilateral backward shifts on sequence spaces a slight modification shows that one can even obtain exact invariant measures

stochastic system identification with noisy input using cumulant statistics addresses the problem of estimating the parameters of stochastic linear systems when the measurements of the system input as well as the system output are noise contaminated it is assumed that the input is non gaussian and the noises are gaussian the square root of the magnitude of the fourth cumulant of a generalized error signal is proposed as a performance criterion for parameter estimation an optimization algorithm is presented strong consistency of the proposed parameter estimators is proved under certain sufficient conditions both single input single output and multiple input multiple output cases are investigated finally simulation results are presented to illustrate the proposed approach