layered dynamic textures a novel video representation the layered dynamic texture ldt is proposed the ldt is a generative model which represents a video as a collection of stochastic layers of different appearance and dynamics each layer is modeled as a temporal texture sampled from a different linear dynamical system the ldt model includes these systems a collection of hidden layer assignment variables which control the assignment of pixels to layers and a markov random field prior on these variables which encourages smooth segmentations an em algorithm is derived for maximum likelihood estimation of the model parameters from a training video it is shown that exact inference is intractable a problem which is addressed by the introduction of two approximate inference procedures a gibbs sampler and a computationally efficient variational approximation the trade off between the quality of the two approximations and their complexity is studied experimentally the ability of the ldt to segment videos into layers of coherent appearance and dynamics is also evaluated on both synthetic and natural videos these experiments show that the model possesses an ability to group regions of globally homogeneous but locally heterogeneous stochastic dynamics currently unparalleled in the literature

a sharp lower bound for the hausdorff dimension of the global attractors of the 2d navier stokes equations for a special class of the navier stokes equations on the two dimensional torus we give a lower bound in the formg2 3 whereg is the grashof number for the hausdorff dimension of its global attractor which is optimal up to a logarithmic term

convergence analysis of a finite volume method for maxwell s equations in nonhomogeneous media in this paper we analyze a recently developed finite volume method for the time dependent maxwell s equations in a three dimensional polyhedral domain composed of two dielectric materials with different parameter values for the electric permittivity and the magnetic permeability convergence and error estimates of the numerical scheme are established for general nonuniform tetrahedral triangulations of the physical domain in the case of nonuniform rectangular grids the scheme converges with second order accuracy in the discrete l2 norm despite the low regularity of the true solution over the entire domain in particular the finite volume method is shown to be superconvergent in the discrete h curl norm in addition the explicit dependence of the error estimates on the material parameters is given

number of degrees of freedom of two dimensional turbulence we derive upper bounds for the number of degrees of freedom of two dimensional navier stokes turbulence freely decaying from a smooth initial vorticity field omega x y 0 omega0 this number denoted by n is defined as the minimum dimension such that for n or n arbitrary n dimensional balls in phase space centered on the solution trajectory omega x y t for t 0 contract under the dynamics of the system linearized about omega x y t in other words n is the minimum number of greatest lyapunov exponents whose sum becomes negative it is found that n or c1re when the phase space is endowed with the energy norm and n or c2re 1 ln re 1 3 when the phase space is endowed with the enstrophy norm here c1 and c2 are constant and re is the reynolds number defined in terms of omega0 the system length scale and the viscosity nu the linear or nearly linear dependence of n on re is consistent with the estimate for the number of active modes deduced from a recent mathematical bound for the viscous dissipation wave number this result is in a sharp contrast to the forced case for which well known estimates for the hausdorff dimension dh of the global attractor scale highly superlinearly with nu 1 we argue that the extra dependence of dh on nu 1 is not an intrinsic property of the turbulent dynamics rather it is a removable artifact brought about by the use of a time independent forcing as a model for energy and enstrophy injection that drives the turbulence

parametrising the attractor of the two dimensional navier stokes equations with a finite number of nodal values abstract we consider the solutions lying on the global attractor of the two dimensional navier stokes equations with periodic boundary conditions and analytic forcing we show that in this case the value of a solution at a finite number of nodes determines elements of the attractor uniquely proving a conjecture due to foias and temam our results also hold for the complex ginzburg landau equation the kuramoto sivashinsky equation and reaction diffusion equations with analytic nonlinearities

extensivity of two dimensional turbulence this study is concerned with how the attractor dimension of the two dimensional navier stokes equations depends on characteristic length scales including the system integral length scale the forcing length scale and the dissipation length scale upper bounds on the attractor dimension derived by constantin foias and temam are analysed it is shown that the optimal attractor dimension estimate grows linearly with the domain area suggestive of extensive chaos for a sufficiently large domain if the kinematic viscosity and the amplitude and length scale of the forcing are held fixed for sufficiently small domain area a slightly super extensive estimate becomes optimal in the extensive regime the attractor dimension estimate is given by the ratio of the domain area to the square of the dissipation length scale defined on physical grounds in terms of the average rate of shear this dissipation length scale which is not necessarily the scale at which the energy or enstrophy dissipation takes place can be identified with the dimension correlation length scale the square of which is interpreted according to the concept of extensive chaos as the area of a subsystem with one degree of freedom furthermore these length scales can be identified with a minimum length scale of the flow which is rigorously deduced from the concept of determining nodes

joint state and parameter estimation for a target directed nonlinear dynamic system model we present a new approach to joint state and parameter estimation for a target directed nonlinear dynamic system model with switching states the model recently proposed for representing speech dynamics is called the hidden dynamic model hdm the model parameters subject to statistical estimation consist of the target vector and the system matrix also called time constants as well as parameters characterizing the nonlinear mapping from the hidden state to the observation we implement these parameters as the weights of a three layer feedforward multilayer perceptron mlp network the new estimation approach is based on the extended kalman filter ekf and its performance is compared with the traditional expectation maximization em based approach extensive simulation results are presented using both approaches and under typical hdm speech modeling conditions the ekf based algorithm demonstrates superior convergence performance compared with the em algorithm but the former suffers from excessive computational loads when adopted for training the mlp weights in all cases the simulated model output converges to the given observation sequence however only in the case where the mlp weights or the target vector are assumed known do the time constant parameters converge to their true values we also show that the mlp weights never converge to their true values thus demonstrating the many to one mapping property of the feedforward mlp we conclude that for the system to be identifiable restrictions on the parameter space are needed

universality of probability distributions among two dimensional turbulent flows we study statistical properties of two dimensional turbulent flows three systems are considered the navier stokes equation surface quasigeostrophic flow and a model equation for thermal convection in the earth s mantle direct numerical simulations are used to determine one point fluctuation properties comparative study shows universality of probability density functions pdfs across different types of flow for instance the pdfs for derivatives of the advected quantity are the same for the three flows once normalized by the average size of fluctuations the single point statistics is surprisingly robust with respect to the nature of the nonlinearity

online bayesian estimation of transition probabilities for markovian jump systems markovian jump systems mjss evolve in a jump wise manner by switching among simpler models according to a finite markov chain whose parameters are commonly assumed known this paper addresses the problem of state estimation of mjs with unknown transition probability matrix tpm of the embedded markov chain governing the jumps under the assumption of a time invariant but random tpm an approximate recursion for the tpms posterior probability density function pdf within the bayesian framework is obtained based on this recursion four algorithms for online minimum mean square error mmse estimation of the tpm are derived the first algorithm for the case of a two state markov chain computes the mmse estimate exactly if the likelihood of the tpm is linear in the transition probabilities its computational load is however increasing with the data length to limit the computational cost three alternative algorithms are further developed based on different approximation techniques truncation of high order moments quasi bayesian approximation and numerical integration respectively the proposed tpm estimation is naturally incorporable into a typical online bayesian estimation scheme for mjs e g generalized pseudo bayesian gpb or interacting multiple model imm thus adaptive versions of mjs state estimators with unknown tpm are provided simulation results of tpm adaptive imm algorithms for a system with failures and maneuvering target tracking are presented

discrete and continuous time probabilistic models and algorithms for inferring neuronal up and down states up and down states the periodic fluctuations between increased and decreased spiking activity of a neuronal population are a fundamental feature of cortical circuits understanding up down state dynamics is important for understanding how these circuits represent and transmit information in the brain to date limited work has been done on characterizing the stochastic properties of up down state dynamics we present a set of markov and semi markov discrete and continuous time probability models for estimating up and down states from multiunit neural spiking activity we model multiunit neural spiking activity as a stochastic point process modulated by the hidden up and down states and the ensemble spiking history we estimate jointly the hidden states and the model parameters by maximum likelihood using an expectation maximization em algorithm and a monte carlo em algorithm that uses reversible jump markov chain monte carlo sampling in the e step we apply our models and algorithms in the analysis of both simulated multiunit spiking activity and actual multi unit spiking activity recorded from primary somatosensory cortex in a behaving rat during slow wave sleep our approach provides a statistical characterization of up down state dynamics that can serve as a basis for verifying and refining mechanistic descriptions of this process

nonlinear transfer and spectral distribution of energy in α turbulence abstract two dimensional turbulence governed by the so called turbulence equations which include the surface quasi geostrophic equation 1 the navier stokes system 2 and the governing equation for a shallow flow on a rotating domain driven by a uniform internal heating 3 is studied here in both the unbounded and doubly periodic domains this family of equations conserves two inviscid invariants energy and enstrophy in the navier stokes case the dynamics of which are believed to undergo a dual cascade it is shown that an inverse cascade can exist in the absence of a direct cascade and that the latter is possible only when the inverse transfer rate of the inverse cascading quantity approaches its own injection rate constraints on the spectral exponents in the wavenumber ranges lower and higher than the injection range are derived for navier stokes turbulence with moderate reynolds numbers the realization of an inverse energy cascade in the complete absence of a direct enstrophy cascade is confirmed by numerical simulations

a switching multi scale dynamical network model of eeg meg we introduce a new generative model of the encephalography eeg meg data the inversion of which allows for inferring the locations and temporal evolution of the underlying sources as well as their dynamical interactions the proposed switching mesostate space model smsm builds on the multi scale generative model for eeg meg by daunizeau and friston 2007 smsm inherits the assumptions that 1 bioelectromagnetic activity is generated by a set of distributed sources 2 the dynamics of these sources can be modelled as random fluctuations about a small number of mesostates and 3 the number of mesostates engaged by a cognitive task is small additionally four generalising assumptions are now included 4 the mesostates interact according to a full dynamical causal network dcn that can be estimated 5 the dynamics of the mesostates can switch between multiple approximately linear operating regimes 6 each operating regime remains stable over finite periods of time temporal clusters and 7 the total number of times the mesostates dynamics can switch is small the proposed model adds therefore a level of flexibility by accommodating complex brain processes that cannot be characterised by purely linear and stationary gaussian dynamics importantly the smsm furnishes a new interpretation of the eeg meg data in which the source activity may have multiple discrete modes of behaviour each with approximately linear dynamics this is modelled by assuming that the connection strengths of the underlying mesoscopic dcn are time dependent but piecewise constant i e they can undergo discrete changes over time a variational bayes inversion scheme is derived to estimate all the parameters of the model by maximising a negative free energy lower bound on the model evidence this bound is used to select among different model choices that are defined by the number of mesostates as well as by the number of stationary linear regimes the full model is compared to a simplified version that uses no dynamical assumptions as well as to a standard eeg inversion technique the comparison is carried out using an extensive set of simulations and the application of smsm to a real data set is also demonstrated our results show that for experimental situations in which we have some a priori belief that there are multiple approximately linear dynamical regimes the proposed smsm provides a natural modelling tool

on gibbs sampling for state space models summary we show how to use the gibbs sampler to carry out bayesian inference on a linear state space model with errors that are a mixture of normals and coefficients that can switch over time our approach simultaneously generates the whole of the state vector given the mixture and coefficient indicator variables and simultaneously generates all the indicator variables conditional on the state vectors the states are generated efficiently using the kalman filter we illustrate our approach by several examples and empirically compare its performance to another gibbs sampler where the states are generated one at a time the empirical results suggest that our approach is both practical to implement and dominates the gibbs sampler that generates the states one at a time

variational bayesian approach to canonical correlation analysis as a dimension reduction algorithm canonical correlation analysis cca encounters the issue of selecting the number of canonical correlations in this letter we present a bayesian model selection algorithm for cca based on a probabilistic interpretation a hierarchical bayesian model is applied to probabilistic cca and learned by variational approximation this method not only estimates the model parameters but also automatically determines the number of canonical correlations and avoids overfitting experiments show that it performs better compared with maximum likelihood and some other model selection methods

hierarchical mixtures of experts and the em algorithm we present a tree structured architecture for supervised learning the statistical model underlying the architecture is a hierarchical mixture model in which both the mixture coefficients and the mixture components are generalized linear models glim s learning is treated as a maximum likelihood problem in particular we present an expectation maximization em algorithm for adjusting the parameters of the architecture we also develop an on line learning algorithm in which the parameters are updated incrementally comparative simulation results are presented in the robot dynamics domain

kirchhoff s laws as a finite volume method for the planar maxwell equations beginning with maxwell s equations for the h1 h2 e polarized mode in an inhomogeneous planar medium we derive a finite volume method that we recognize as kirchhoff s laws for a corresponding circuit consisting of inductors capacitors and resistors this association automatically gives local charge and energy conservation the method is implemented and used to find the steady state solution for two test problems by comparison with the exact solution for the homogeneous medium problem the method is shown to be linearly convergent

maximum likelihood estimation of transition probabilities of jump markov linear systems this paper describes an online maximum likelihood estimator for the transition probabilities associated with a jump markov linear system jmls the maximum likelihood estimator is derived using the reference probability method which exploits an hypothetical probability measure to find recursions for complex expectations expectation maximization em procedure is utilized for maximizing the likelihood function in order to avoid the exponential increase in the number of statistics of the optimal em algorithm we make interacting multiple model imm type approximations the resulting method needs the mode weights of an imm filter with n3 components where n is the number of models in the jmls the algorithm can also supply base state estimates and covariances as a by product the performance of the estimator is illustrated on two simulated examples and compared to a recently proposed alternative

diffraction on the two dimensional square lattice we solve the thin slit diffraction problem for two dimensional lattice waves more precisely for the discrete helmholtz equation on the semi infinite square lattice with data prescribed on the left boundary the aperture we use lattice green s functions and a discrete sommerfeld outgoing radiation condition to derive the exact solution everywhere in the lattice the solution is a discrete convolution that can be evaluated in closed form for the wave number k 2 for other wave numbers we give a recursive algorithm for computing the convolution kernel

hybrid time domain solvers for the maxwell equations in 2d numerical simulation is an important tool in understanding the electromagnetic field and how it interacts with the environment different topics for time domain finite difference fdtd and finite element fetd methods for maxwell s equations are treated in this thesis subcell models are of vital importance for the efficient modeling of small objects that are not resolved by the grid a novel model for thin sheets using shell elements is proposed this approach has the advantage of taking into account discontinuities in the normal component of the electric field unlike previous models based on impedance boundary conditions ibcs several results are presented to illustrate the capabilities of the shell element approach waveguides are of fundamental importance in many microwave applications for example in antenna feeds the key issues of excitation and truncation of waveguides are addressed a complex frequency shifted form of the uniaxial perfectly matched layer upml absorbing boundary condition abc in fetd is developed prism elements are used to promote automatic grid generation and enhance the performance results are presented where reflection errors below 70db are obtained for different types of waveguides including inhomogeneous cases excitation and analysis via the scattering parameters are achieved using waveguide modes computed by a general frequency domain mode solver for the vector helmholtz equation huygens surfaces are used in both fdtd and fetd for excitation in waveguide ports inverse problems have received an increased interest due to the availability of powerful computers an important application is non destructive evaluation of material a time domain minimization approach is presented where exact gradients are computed using the adjoint problem the approach is applied to a general form of maxwell s equations including dispersive media and upml successful reconstruction examples are presented both using synthetic and experimental measurement data parameter reduction of complex geometries using simplified models is an interesting topic that leads to an inverse problem gradients for subcell parameters are derived and a successful reconstruction example is presented for a combined dielectric sheet and slot geometry

method of speech recognition using multimodal variational inference with switching state space models a method of efficiently setting posterior probability parameters for a switching state space model begins by defining a window containing at least two but fewer than all of the frames a separate posterior probability parameter is determined for each frame in the window the window is then shifted sequentially from left to right in time so that it includes one or more subsequent frames in the sequence of frames a separate posterior probability parameter is then determined for each frame in the shifted window this method closely approximates a more rigorous solution but saves computational cost by two to three orders of magnitude further a method of determining the optimal discrete state sequence in the switching state space model is invented that directly exploits the observation vector on a frame by frame basis and operates from left to right in time

an unsupervised ensemble learning method for nonlinear dynamic state space models a bayesian ensemble learning method is introduced for unsupervised extraction of dynamic processes from noisy data the data are assumed to be generated by an unknown nonlinear mapping from unknown factors the dynamics of the factors are modeled using a nonlinear state space model the nonlinear mappings in the model are represented using multilayer perceptron networks the proposed method is computationally demanding but it allows the use of higher dimensional nonlinear latent variable models than other existing approaches experiments with chaotic data show that the new method is able to blindly estimate the factors and the dynamic process that generated the data it clearly outperforms currently available nonlinear prediction techniques in this very difficult test problem

low dimensional attractors arise from forcing at small scales abstract standard estimates of the dimension of the attractor of the 2d navier stokes equations are given in terms of a dimensionless grashof number that measures the total amount of energy injected into the flow however this result takes no account of whether the forcing is concentrated at large or small scales by a simple modification of the usual argument this paper provides a bound that is linear in a modified grashof number that reflects the spatial structure of the forcing in particular as a fixed amount of energy is injected at progressively smaller scales then the dimension of the attractor decreases

note on the constantin foias temam attractor dimension estimate for two dimensional turbulence abstract we present a simplified version of the 2d incompressible navier stokes equations attractor dimension estimate originally derived by constantin foias and temam physica d 30 1988 284 296 by considering the attractor for the vorticity in the hilbert space l2

on the dual cascade in two dimensional turbulence we study the dual cascade scenario for two dimensional turbulence driven by a spectrally localized forcing applied over a finite wavenumber range kmin kmax with kmin 0 such that the respective energy and enstrophy injection rates and satisfy k 2 k 2 the classical kraichnan leith batchelor paradigm based on the simultaneous conservation of energy and enstrophy and the scaleselectivity of the molecular viscosity requires that the domain be unbounded in both directions for two dimensional turbulence either in a doubly periodic domain or in an unbounded channel with a periodic boundary condition in the acrosschannel direction a direct enstrophy cascade is not possible in the usual case where the forcing wavenumber is no greater than the geometric mean of the integral and dissipation wavenumbers constant spectral slopes must satisfy 5 and 8 where is the asymptotic slope of the range of wavenumbers lower higher than the forcing wavenumber the influence of a large scale dissipation on the realizability of a dual cascade is analyzed we discuss the consequences for numerical simulations attempting to mimic the classical unbounded picture in a bounded domain

a unifying review of linear gaussian models factor analysis principal component analysis mixtures of gaussian clusters vector quantization kalman filter models and hidden markov models can all be unified as variations of unsupervised learning under a single basic generative model this is achieved by collecting together disparate observations and derivations made by many previous authors and introducing a new way of linking discrete and continuous state models using a simple nonlinearity through the use of other nonlinearities we show how independent component analysis is also a variation of the same basic generative model we show that factor analysis and mixtures of gaussians can be implemented in autoencoder neural networks and learned using squared error plus the same regularization term we introduce a new model for static data known as sensible principal component analysis as well as a novel concept of spatially adaptive observation noise we also review some of the literature involving global and local mixtures of the basic models and provide pseudocode for inference and learning for all the basic models

variational learning for switching state space models we introduce a new statistical model for time series that iteratively segments data into regimes with approximately linear dynamics and learns the parameters of each of these linear regimes this model combines and generalizes two of the most widely used stochastic time series models hidden markov models and linear dynamical systems and is closely related to models that are widely used in the control and econometrics literatures it can also be derived by extending the mixture of experts neural network jacobs jordan nowlan hinton 1991 to its fully dynamical version in which both expert and gating networks are recurrent inferring the posterior probabilities of the hidden states of this model is computationally intractable and therefore the exact expectation maximization em algorithm cannot be applied however we present a variational approximation that maximizes a lower bound on the log likelihood and makes use of both the forward and backward recursions for hidden markov models and the kalman filter recursions for linear dynamical systems we tested the algorithm on artificial data sets and a natural data set of respiration force from a patient with sleep apnea the results suggest that variational approximations are a viable method for inference and learning in switching state space models

a finite volume method for the approximation of maxwell s equations in two space dimensions on arbitrary meshes a new finite volume method is presented for discretizing the two dimensional maxwell equations this method may be seen as an extension of the covolume type methods to arbitrary possibly non conforming or even non convex n sided polygonal meshes thanks to an appropriate choice of degrees of freedom an equivalent formulation of the scheme is given in terms of discrete differential operators obeying discrete duality principles the main properties of the scheme are its energy conservation its stability under a cfl like condition and the fact that it preserves gauss law and divergence free magnetic fields second order convergence is demonstrated numerically on non conforming and distorted meshes

determining finite volume elements for the 2d navier stokes equations we consider the 2d navier stokes equations on a square with periodic boundary conditions dividing the square into n equal subsquares we show that if the asymptotic behavior of the average of solutions on these subsquares finite volume elements is known then the large time behavior of the solution itself is completely determined provided n is large enough we also establish a rigorous upper bound for n needed to determine the solutions to the navier stokes equation in terms of the physical parameters of the problem

chaotic attractors of an infinite dimensional dynamical system abstract we study the chaotic attractors of a delay differential equation the dimension of several attractors computed directly from the definition agrees to experimental resolution with the dimension computed from the spectrum of lyapunov exponents according to a conjecture of kaplan and yorke assuming this conjecture to be valid as the delay parameter is varied from computations of the spectrum of lyapunov exponents we observe a roughly linear increase from two to twenty in the dimension while the metric entropy remains roughly constant these results are compared to a linear analysis and the asymptotic behavior of the lyapunov exponents is derived

dynamic linear models with switching abstract the problem of modeling change in a vector time series is studied using a dynamic linear model with measurement matrices that switch according to a time varying independent random process we derive filtered estimators for the usual state vectors and also for the state occupancy probabilities of the underlying nonstationary measurement process a maximum likelihood estimation procedure is given that uses a pseudo expectation maximization algorithm in the initial stages and nonlinear optimization we relate the models to those considered previously in the literature and give an application involving the tracking of multiple targets

detection of hidden structures in nonstationary spike trains we propose an algorithm for simultaneously estimating state transitions among neural states and nonstationary firing rates using a switching state space model sssm this algorithm enables us to detect state transitions on the basis of not only discontinuous changes in mean firing rates but also discontinuous changes in the temporal profiles of firing rates e g temporal correlation we construct estimation and learning algorithms for a nongaussian sssm whose nongaussian property is caused by binary spike events local variational methods can transform the binary observation process into a quadratic form the transformed observation process enables us to construct a variational bayes algorithm that can determine the number of neural states based on automatic relevance determination additionally our algorithm can estimate model parameters from single trial data using a priori knowledge about state transitions and firing rates synthetic data analysis reveals that our algorithm has higher performance for estimating nonstationary firing rates than previous methods the analysis also confirms that our algorithm can detect state transitions on the basis of discontinuous changes in temporal correlation which are transitions that previous hidden markov models could not detect we also analyze neural data recorded from the medial temporal area the statistically detected neural states probably coincide with transient and sustained states that have been detected heuristically estimated parameters suggest that our algorithm detects the state transitions on the basis of discontinuous changes in the temporal correlation of firing rates these results suggest that our algorithm is advantageous in real data analysis

convergence analysis of fully discrete finite volume methods for maxwell s equations in nonhomogeneous media we will consider both explicit and implicit fully discrete finite volume schemes for solving three dimensional maxwell s equations with discontinuous physical coefficients on general polyhedral domains stability and convergence for both schemes are analyzed we prove that the schemes are second order accurate in time both schemes are proved to be first order accurate in space for the voronoi delaunay grids and second order accurate for nonuniform rectangular grids we also derive explicit expressions for the dependence on the physical parameters in all estimates

modeling the manifolds of images of handwritten digits this paper describes two new methods for modeling the manifolds of digitized images of handwritten digits the models allow a priori information about the structure of the manifolds to be combined with empirical data accurate modeling of the manifolds allows digits to be discriminated using the relative probability densities under the alternative models one of the methods is grounded in principal components analysis the other in factor analysis both methods are based on locally linear low dimensional approximations to the underlying data manifold links with other methods that model the manifold are discussed

bayesian nonparametric inference of switching dynamic linear models many complex dynamical phenomena can be effectively modeled by a system that switches among a set of conditionally linear dynamical modes we consider two such models the switching linear dynamical system slds and the switching vector autoregressive var process our bayesian nonparametric approach utilizes a hierarchical dirichlet process prior to learn an unknown number of persistent smooth dynamical modes we additionally employ automatic relevance determination to infer a sparse set of dynamic dependencies allowing us to learn slds with varying state dimension or switching var processes with varying autoregressive order we develop a sampling algorithm that combines a truncated approximation to the dirichlet process with efficient joint sampling of the mode and state sequences the utility and flexibility of our model are demonstrated on synthetic data sequences of dancing honey bees the ibovespa stock index and a maneuvering target tracking application

variational bayesian learning of probabilistic discriminative models with latent softmax variables this paper presents new variational bayes vb approximations for learning probabilistic discriminative models with latent softmax variables such as subclass based multimodal softmax and mixture of experts models the vb approximations derived here lead to closed form approximate parameter posteriors and suitable metrics for model selection unlike other bayesian methods for this challenging class of models the proposed vb methods require neither restrictive structural assumptions nor sampling approximations to cope with the problematic softmax function as such the proposed vb methods are also easily extendable to more complex softmax based hierarchical discriminative models and regression models for continuous outputs the proposed vb methods are evaluated on benchmark classification data and a decision modeling application demonstrating good results

dynamic linear models with markov switching abstract in this paper hamilton s 1988 1989 markov switching model is extended to a general state space model this paper also complements shumway and stoffer s 1991 dynamic linear models with switching by introducing dependence in the switching process and by allowing switching in both measurement and transition equations building upon ideas in hamilton 1989 cosslett and lee 1985 and harrison and stevens 1976 a basic filtering and smoothing algorithm is presented the algorithm and the maximum likelihood estimation procedure is applied in estimating lam s 1990 generalized hamilton model with a general autoregressive component the estimation results show that the approximation employed in this paper performs an excellent job with a considerable advantage in computation time a state space representation is a very flexible form and the approach taken in this paper therefore allows a broad class of models to be estimated that could not be handled before in addition the algorithm for calculating smoothed inferences on the unobserved states is a vastly more efficient one than that in the literature

method of speech recognition using variational inference with switching state space models a method is developed which includes 1 defining a switching state space model for a continuous valued hidden production related parameter and the observed speech acoustics and 2 approximating a posterior probability that provides the likelihood of a sequence of the hidden production related parameters and a sequence of speech units based on a sequence of observed input values in approximating the posterior probability the boundaries of the speech units are not fixed but are optimally determined under one embodiment a mixture of gaussian approximation is used in another embodiment an hmm posterior approximation is used

online learning of hierarchical pitman yor process mixture of generalized dirichlet distributions with feature selection in this paper a novel statistical generative model based on hierarchical pitman yor process and generalized dirichlet distributions gds is presented the proposed model allows us to perform joint clustering and feature selection thanks to the interesting properties of the gd distribution we develop an online variational inference algorithm formulated in terms of the minimization of a kullback leibler divergence of our resulting model that tackles the problem of learning from high dimensional examples this variational bayes formulation allows simultaneously estimating the parameters determining the model s complexity and selecting the appropriate relevant features for the clustering structure moreover the proposed online learning algorithm allows data instances to be processed in a sequential manner which is critical for large scale and real time applications experiments conducted using challenging applications namely scene recognition and video segmentation where our approach is viewed as an unsupervised technique for visual learning in high dimensional spaces showed that the proposed approach is suitable and promising

learning and inferring motion patterns using parametric segmental switching linear dynamic systems switching linear dynamic system slds models are a popular technique for modeling complex nonlinear dynamic systems an slds can describe complex temporal patterns more concisely and accurately than an hmm by using continuous hidden states however the use of slds models in practical applications is challenging for three reasons first exact inference in slds models is computationally intractable second the geometric duration model induced in standard sldss limits their representational power third standard sldss do not provide a principled way to interpret systematic variations governed by higher order parameters the contributions in this paper address all of these three challenges first we present a data driven mcmc dd mcmc sampling method for approximate inference in sldss we show dd mcmc provides an efficient method for estimation and learning in slds models second we present segmental sldss s slds where the geometric distributions of the switching state durations are replaced with arbitrary duration models third we extend the standard slds model with additional global parameters that can capture systematic temporal and spatial variations the resulting parametric slds model p slds uses em to robustly interpret parametrized motions by incorporating additional global parameters that underly systematic variations of the overall motion the overall development of the extensions for sldss provide a principled framework to interpret complex motions the framework is applied to the honey bee dance interpretation task in the context of the on going biotracking project at the georgia institute of technology the experimental results suggest that the enhanced models provide an effective framework for a wide range of motion analysis applications

robustness of the inverse cascade in two dimensional turbulence we study quasisteady inverse cascades in unbounded and bounded two dimensional turbulence driven by time independent injection and dissipated by molecular viscosity it is shown that an inverse cascade that carries only a fraction r of the energy input to the largest scales requires the enstrophy range energy spectrum to be steeper than k ensuremath 5 ruling out a direct cascade unless 1 ensuremath r ensuremath ll 1 a direct cascade requires the presence of an inverse cascade that carries virtually all energy to the largest scales 1 ensuremath r ensuremath ll 1 these facts underlie the robustness of the kolmogorov kraichnan k ensuremath 5 3 inverse cascade which is readily observable in numerical simulations without an accompanying direct enstrophy cascade we numerically demonstrate an instance where the k ensuremath 5 3 inverse cascading range is realizable with 79 of the energy injection dissipated within the energy range and virtually all of the enstrophy dissipated in the vicinity of the forcing region as equilibrium is approached the respective logarithmic slopes ensuremath alpha and ensuremath beta of the ranges of wave numbers lower and higher than the forcing wave number satisfy ensuremath alpha ensuremath beta ensuremath approx 8 these results are consistent with recent theoretical predictions

energy spectra of steady two dimensional turbulent flows more than 30 years ago the energy spectrum of twodimensional turbulence was theoretically predicted yet the evidence for this spectrum has remained weak in some cases classical theory 1 3 predicts the energy spectrum based on a cascade of either energy or enstrophy once the relevant physical quantities have been identified simple dimensional analysis leads to the power spectrum as a function of wave number k however the spectra measured in many numerical simulations decay substantially faster with k than predicted 4 the range of wave numbers larger than the inverse system size and up to the dissipative damping is called inertial range in this paper we assume power laws for the inertial range as a working hypothesis since possible deviations from a power behavior cannot be determined from numerical data with confidence a general review of two dimensional turbulence and the slope of the energy spectrum can be found in refs 5 6 following we summarize recent results on the inertial range spectrum

asymptotic analysis of the navier stokes equations abstract new bounds are established on the number of modes which determine the solutions of the navier stokes equations in two dimensions the best bound available at present is nearly proportional to the generalized grashof number defined in the paper and less than logarithmically dependent on the spatial structure or the shape of the force driving the flow to the extent than for the case of 2 dimensional rayleigh benard convection the generalized grashof number may be identified with the usual grashof number the resulting bound on the number of modes is found to differ only slightly from a bound obtained earlier on heuristic grounds

approximate inference in state space models with heavy tailed noise state space models have been successfully applied across a wide range of problems ranging from system control to target tracking and autonomous navigation their ubiquity stems from their modeling flexibility as well as the development of a battery of powerful algorithms for estimating the state variables for multivariate models the gaussian noise assumption is predominant due its convenient computational properties in some cases anyhow this assumption breaks down and no longer holds we propose a novel approach to extending the applicability of this class of models to a wider range of noise distributions without losing the computational advantages of the associated algorithms the estimation methods we develop parallel the kalman filter and thus are readily implemented and inherit the same order of complexity we derive all of the equations and algorithms from first principles in order to validate the performance of our approach we present specific instances of non gaussian state space models and test their performance on experiments with synthetic and real data

on the dimension of the attractors in two dimensional turbulence abstract using a new version of the sobolev lieb thirring inequality we derive an upper bound for the dimension of the universal attractor for two dimensional space periodic navier stokes equations this estimate is optimal up to a logarithmic correction the relevance of this estimate to turbulence and related results are also briefly discussed

variational bayesian identification and prediction of stochastic nonlinear dynamic causal models in this paper we describe a general variational bayesian approach for approximate inference on nonlinear stochastic dynamic models this scheme extends established approximate inference on hidden states to cover i nonlinear evolution and observation functions ii unknown parameters and precision hyperparameters and iii model comparison and prediction under uncertainty model identification or inversion entails the estimation of the marginal likelihood or evidence of a model this difficult integration problem can be finessed by optimising a free energy bound on the evidence using results from variational calculus this yields a deterministic update scheme that optimises an approximation to the posterior density on the unknown model variables we derive such a variational bayesian scheme in the context of nonlinear stochastic dynamic hierarchical models for both model identification and time series prediction the computational complexity of the scheme is comparable to that of an extended kalman filter which is critical when inverting high dimensional models or long time series using monte carlo simulations we assess the estimation efficiency of this variational bayesian approach using three stochastic variants of chaotic dynamic systems we also demonstrate the model comparison capabilities of the method its self consistency and its predictive power

geometric statistics in turbulence the author presents results regarding certain average properties of incompressible fluids derived from the equations of motion the author estimates the average dissipation rate the average dimension of level sets the role played by the field of direction of vorticity in the three dimensional euler and navier stokes equations is discussed and a class of two dimensional equations that are useful models of incompressible dynamics is described the author presents results concerning scaling exponents in turbulence

sparse kalman filtering approaches to realized covariance estimation from high frequency financial data estimation of the covariance matrix of asset returns from high frequency data is complicated by asynchronous returns market microstructure noise and jumps one technique for addressing both asynchronous returns and market microstructure is the kalman expectation maximization kem algorithm however the kem approach assumes log normal prices and does not address jumps in the return process which can corrupt estimation of the covariance matrix in this paper we extend the kem algorithm to price models that include jumps we propose a sparse kalman filtering approach to this problem in particular we develop a kalman expectation conditional maximization algorithm to determine the unknown covariance as well as detecting the jumps in order to promote a sparse estimate of the jumps we consider both laplace and the spike and slab jump priors numerical results using simulated data show that each of these approaches provide for improved covariance estimation relative to the kem method in a variety of settings where jumps occur