recursive maximum likelihood identification of jump markov nonlinear systems we present an online method for joint state and parameter estimation in jump markov non linear systems jmnls state inference is enabled via the use of particle filters which makes the method applicable to a wide range of non linear models to exploit the inherent structure of jmnls we design a rao blackwellized particle filter rbpf where the discrete mode is marginalized out analytically this results in an efficient implementation of the algorithm and reduces the estimation error variance the proposed rbpf is then used to compute recursively in time smoothed estimates of complete data sufficient statistics together with the online expectation maximization algorithm this enables recursive identification of unknown model parameters including the transition probability matrix the method is also applicable to online identification of jump markov linear systems jmls the performance of the method is illustrated in simulations and on a localization problem in wireless networks using real data

partial non gaussian state space summary in this paper we suggest the use of simulation techniques to extend the applicability of the usual gaussian state space filtering and smoothing techniques to a class of nongaussian time series models this allows a fully bayesian or maximum likelihood analysis of some interesting models including outlier models discrete markov chain components multiplicative models and stochastic variance models finally we discuss at some length the use of a non gaussian model to seasonally adjust the published money supply figures

state estimation in stochastic hybrid systems with sparse observations in this note we study the problem of state estimation for a class of sampled measurement stochastic hybrid systems where the continuous state x satisfies a linear stochastic differential equation and noisy measurements y are taken at assigned discrete time instants the parameters of both the state and measurement equation depend on the discrete state q of a continuous time finite markov chain even in the fault detection setting we consider at most one transition for q is admissible the switch may occur between two observations whence it turns out that the optimal estimates cannot be expressed in parametric form and time integrations are unavoidable so that the known estimation techniques cannot be applied we derive and implement an algorithm for the estimation of the states x q and of the discrete state switching time that is convenient for both recursive update and the eventual numerical quadrature numerical simulations are illustrated

extended object tracking using monte carlo methods this correspondence addresses the problem of tracking extended objects such as ships or a convoy of vehicles moving in urban environment two monte carlo techniques for extended object tracking are proposed an interacting multiple model data augmentation imm da algorithm and a modified version of the mixture kalman filter mkf of chen and liu called the mixture kalman filter modified mkfm the data augmentation da technique with finite mixtures estimates the object extent parameters whereas an interacting multiple model imm filter estimates the kinematic states position and speed of the manoeuvring object next the system model is formulated in a partially conditional dynamic linear pcdl form this affords us to propose two latent indicator variables characterizing respectively the motion mode and object size then an mkfm is developed with the pcdl model the imm da and the mkfm performance is compared with a combined imm particle filter imm pf algorithm with respect to accuracy and computational complexity the most accurate parameter estimates are obtained by the da algorithm followed by the mkfm and pf

quantum mechanical potentials related to the prime numbers and riemann zeros prime numbers are the building blocks of our arithmetic however their distribution still poses fundamental questions riemann showed that the distribution of primes could be given explicitly if one knew the distribution of the nontrivial zeros of the riemann ensuremath zeta s function according to the hilbert p olya conjecture there exists a hermitian operator of which the eigenvalues coincide with the real parts of the nontrivial zeros of ensuremath zeta s this idea has encouraged physicists to examine the properties of such possible operators and they have found interesting connections between the distribution of zeros and the distribution of energy eigenvalues of quantum systems we apply the mar ifmmode check c else v c fi henko approach to construct potentials with energy eigenvalues equal to the prime numbers and to the zeros of the ensuremath zeta s function we demonstrate the multifractal nature of these potentials by measuring the r enyi dimension of their graphs our results offer hope for further analytical progress

partial linear gaussian models for tracking in image sequences using sequential monte carlo methods the recent development of sequential monte carlo methods also called particle filters has enabled the definition of efficient algorithms for tracking applications in image sequences the efficiency of these approaches depends on the quality of the state space exploration which may be inefficient due to a crude choice of the function used to sample in the associated probability space a careful study of this issue led us to consider the modeling of the tracked dynamic system with partial linear gaussian models such models are characterized by a non linear dynamic equation a linear measurement equation and additive gaussian noises they allow inferring an analytic expression of the optimal importance function used in the diffusion process of the particle filter and enable building a relevant approximation of a validation gate despite of these potential advantages partial linear gaussian models have not been investigated the aim of this paper is therefore to demonstrate that such models can be of real interest facing difficult usual issues such as occlusions ambiguities due to cluttered backgrounds and large state space three instances of these models are proposed after a theoretical analysis their significance is demonstrated by their performance for tracking points and planar objects in challenging real world image sequences

the statistical mechanics of constructive algorithms the storage capacity of multilayer networks with overlapping receptive fields is studied for constructive algorithms using boolean perceptrons as their basic building block which have been investigated within a replica framework the assumption of weak coupling between subsequently constructed perceptrons is verified within a replica symmetric rs ansatz and shown to be negligible in most cases in comparison with correction due to replica symmetry breaking rsb in individual perceptrons the capacities of a tiling like and variants of the upstart algorithm are then calculated within rs and one step rsb with the quenched average taken over the individual units separately for networks with up to k 4000 and k 600 units respectively within this treatment the storage capacity seems to exhibit a power law behaviour in with an exponent n that may depend on the algorithm and the stability however due to finite size effects in k reliable estimates of n could not be extracted nevertheless the results strongly indicate that n should be strictly smaller than 1 within one step rsb whereas within rs the mitchison durbin bound is violated for finite k and n 1 may hold asymptotically

central limit theorem for sequential monte carlo methods and its application to bayesian inference the term sequential monte carlo methods or equivalently particle filters refers to a general class of iterative algorithms that performs monte carlo approximations of a given sequence of distributions of interest t we establish in this paper a central limit theorem for the monte carlo estimates produced by these computational methods this result holds under minimal assumptions on the distributions t and applies in a general framework which encompasses most of the sequential monte carlo methods that have been considered in the literature including the resample move algorithm of gilks and berzuini j r stat soc ser b stat methodol 63 2001 127 146 and the residual resampling scheme the corresponding asymptotic variances provide a convenient measurement of the precision of a given particle filter we study in particular in some typical examples of bayesian applications whether and at which rate these asymptotic variances diverge in time in order to assess the long term reliability of the considered algorithm

state estimation for polyhedral hybrid systems and applications to the godunov scheme for highway traffic estimation this paper investigates the problem of estimating the state of discretized hyperbolic scalar partial differential equations it uses a godunov scheme to discretize the so called lighthill whitham richards equation with a triangular flux function and proves that the resulting nonlinear dynamical system can be decomposed in a piecewise affine manner using this explicit representation the system is written as a switching dynamical system with a state space partitioned into an exponential number of polyhedra in which one mode is active we propose a feasible approach based on the interactive multiple model imm which is a widely used algorithm for estimation of hybrid systems in the scientific community the number of modes is reduced based on the geometric properties of the polyhedral partition the k means algorithm is also applied on historical data to partition modes into clusters the performance of these algorithms are compared to the extended kalman filter and the ensemble kalman filter in the context of highway traffic state estimation in particular we use sparse measurements from loop detectors along a section of the i 880 to estimate the state density for our numerical experiments

learning and retrieval in attractor neural networks above saturation the authors investigate neural networks in the range of parameters when the ground state energy is positive namely when a synaptic matrix which satisfies all the desired constraints cannot be found by the learning algorithm in particular they calculate the typical distribution functions of local stabilities obtained for a number of algorithms in this region these functions are used to investigate the retrieval properties as reflected by the size of the basins of attraction this is done analytically in sparsely connected networks and numerically in fully connected networks the main conclusion is that the retrieval behaviour of attractor neural networks can be improved by learning above saturation

a monte carlo approach to nonnormal and nonlinear state space modeling abstract a solution to multivariate state space modeling forecasting and smoothing is discussed we allow for the possibilities of nonnormal errors and nonlinear functionals in the state equation the observational equation or both an adaptive monte carlo integration technique known as the gibbs sampler is proposed as a mechanism for implementing a conceptually and computationally simple solution in such a framework the methodology is a general strategy for obtaining marginal posterior densities of coefficients in the model or of any of the unknown elements of the state space missing data problems including the k step ahead prediction problem also are easily incorporated into this framework we illustrate the broad applicability of our approach with two examples a problem involving nonnormal error distributions in a linear model setting and a one step ahead prediction problem in a situation where both the state and observational equations are nonlinear and involve unknown parameters

rao blackwellized gaussian smoothing in this paper we consider rao blackwellization of linear substructures in sigma point based gaussian assumed density smoothers we derive marginalized prediction smoothing and update steps for the mixed linear nonlinear gaussian state space model as well as for a hierarchical model for both conventional and iterated posterior linearization gaussian smoothers the proposed method is evaluated in a numerical example and it is shown that the computational complexity is reduced considerably compared to non rao blackwellized gaussian smoothers for systems with high dimensional linear subspaces

a class of fast exact bayesian filters in dynamical models with jumps we address the statistical filtering problem in dynamical models with jumps when a particular application is adequately modeled by linear and gaussian probability density functions with jumps a usual method consists in approximating the optimal bayesian estimate in the sense of the minimum mean square error mmse in a linear and gaussian jump markov state space system jmss practical solutions include algorithms based on numerical approximations or on sequential monte carlo smc methods in this paper we propose a class of alternative methods which consists in building statistical models which locally similarly model the problem of interest but in which the computation of the mmse estimate can be be computed exactly without numerical nor smc approximations and at a computational cost which is linear in the number of observations

dynamically orthogonal field equations for continuous stochastic dynamical systems abstract in this work we derive an exact closed set of evolution equations for general continuous stochastic fields described by a stochastic partial differential equation spde by hypothesizing a decomposition of the solution field into a mean and stochastic dynamical component we derive a system of field equations consisting of a partial differential equation pde for the mean field a family of pdes for the orthonormal basis that describe the stochastic subspace where the stochasticity lives as well as a system of stochastic differential equations that defines how the stochasticity evolves in the time varying stochastic subspace these new evolution equations are derived directly from the original spde using nothing more than a dynamically orthogonal condition on the representation of the solution if additional restrictions are assumed on the form of the representation we recover both the proper orthogonal decomposition equations and the generalized polynomial chaos equations we apply this novel methodology to two cases of two dimensional viscous fluid flows described by the navier stokes equations and we compare our results with monte carlo simulations

replica symmetry breaking in noise optimal neural networks recent studies of optimization in neural networks trained with noisy data have shown that replica symmetric solutions are unstable in the low noise region of parameter space we calculate the 1 step replica symmetry broken solution in this region which joins the replica symmetric solution continuously at the de almeida thouless line these solutions yield satisfactory agreement with simulations for the aligning field distribution better than those given by the replica symmetric ansatz

adaptive approximation of higher order posterior statistics filtering is an approach for incorporating observed data into time evolving systems instead of a family of dirac delta masses that is widely used in monte carlo methods we here use the wiener chaos expansion for the parametrization of the conditioned probability distribution to solve the nonlinear filtering problem the wiener chaos expansion is not the best method for uncertainty propagation without observations nevertheless the projection of the system variables in a fixed polynomial basis spanning the probability space might be a competitive representation in the presence of relatively frequent observations because the wiener chaos approach not only leads to an accurate and efficient prediction for short time uncertainty quantification but it also allows to apply several data assimilation methods that can be used to yield a better approximate filtering solution the aim of the present paper is to investigate this hypothesis we answer in the affirmative for the stochastic lorenz 63 system based on numerical simulations in which the uncertainty quantification method and the data assimilation method are adaptively selected by whether the dynamics is driven by brownian motion and the near gaussianity of the measure to be updated respectively

learning and retrieval in attractor neural networks with noise abstract a recent study on noiseless learning and retrieval in attractor neural networks above saturation by griniasty and gutfreund is extended to take account for imperfect learning by means of a temperature 1 t violations of the constraint imposed on the local stabilities are taken into account by various cost functions the distribution of local stabilities and the fraction of errors during the learning stage are analysed the retrieval dynamics for the sparsely connected network is studied showing high retrieval overlap in reduced retrieval regions for finite t

on line inference for multiple changepoint problems we propose an on line algorithm for exact filtering of multiple changepoint problems this algorithm enables simulation from the true joint posterior distribution of the number and position of the changepoints for a class of changepoint models the computational cost of this exact algorithm is quadratic in the number of observations we further show how resampling ideas from particle filters can be used to reduce the computational cost to linear in the number of observations at the expense of introducing small errors and propose two new optimum resampling algorithms for this problem one a version of rejection control allows the particle filter to automatically choose the number of particles required at each time step the new resampling algorithms substantially out perform standard resampling algorithms on examples we consider and we demonstrate how the resulting particle filter is practicable for segmentation of human gc content

adaptive gaussian sum filter for nonlinear bayesian estimation a nonlinear filter is developed by representing the state probability density function by a finite sum of gaussian density kernels whose mean and covariance are propagated from one time step to the next using linear system theory methods such as extended kalman filter or unscented kalman filter the novelty in the proposed method is that the weights of the gaussian kernels are updated at every time step by solving a convex optimization problem posed by requiring the gaussian sum approximation to satisfy the fokker planck kolmogorov equation for continuous time dynamical systems and the chapman kolmogorov equation for discrete time dynamical systems the numerical simulation results show that updating the weights of different mixture components during propagation mode of the filter not only provides us with better state estimates but also with a more accurate state probability density function

bayesian forecasting of an inhomogeneous poisson process with applications to call center data a call center is a centralized hub where customer and other telephone calls are handled by an organization in today s economy call centers have become the primary points of contact between customers and businesses thus accurate predictions of call arrival rates are indispensable to help call center practitioners staff their call centers efficiently and cost effectively this article proposes a multiplicative model for modeling and forecasting within day arrival rates to a u s commercial bank s call center markov chain monte carlo sampling methods are used to estimate both latent states and model parameters one day ahead density forecasts for the rates and counts are provided the calibration of these predictive distributions is evaluated through probability integral transforms furthermore 1 day ahead forecasts comparisons with classical statistical models are given our predictions show significant improvements of up to 25 over these standards a sequential monte carlo algorithm is also proposed

on gibbs sampling for state space models summary we show how to use the gibbs sampler to carry out bayesian inference on a linear state space model with errors that are a mixture of normals and coefficients that can switch over time our approach simultaneously generates the whole of the state vector given the mixture and coefficient indicator variables and simultaneously generates all the indicator variables conditional on the state vectors the states are generated efficiently using the kalman filter we illustrate our approach by several examples and empirically compare its performance to another gibbs sampler where the states are generated one at a time the empirical results suggest that our approach is both practical to implement and dominates the gibbs sampler that generates the states one at a time

wavelet based sequential monte carlo blind receivers in fading channels with unknown channel statistics recently an adaptive bayesian receiver for blind detection in flat fading channels was developed by the present authors based on the sequential monte carlo methodology that work is built on a parametric modeling of the fading process in the form of a state space model and assumes the knowledge of the second order statistics of the fading channel in this paper we develop a nonparametric approach to the problem of blind detection in fading channels without assuming any knowledge of the channel statistics the basic idea is to decompose the fading process using a wavelet basis and to use the sequential monte carlo technique to track both the wavelet coefficients and the transmitted symbols a novel resampling based wavelet shrinkage technique is proposed to dynamically choose the number of wavelet coefficients to best fit the fading process under such a framework blind detectors for both flat fading channels and frequency selective fading channels are developed simulation results are provided to demonstrate the excellent performance of the proposed blind adaptive receivers

optimally adapted attractor neural networks in the presence of noise by adapting an attractor neural network to an appropriate training overlap the authors optimize its attractor overlap and subsequently the storage capacity when retrieval noise temperature is present in the system the training overlap is determined self consistently by the optimal attractor overlap the phase diagram of the optimal attractor overlap in the temperature storage space is found a novel co existence phase of strong and weak retrievers is present the maximum storage capacity deviates from the storage capacity of the maximally stable network on increasing temperature and in the high temperature regime t or 0 38 for gaussian noise the hopfield network yields the maximum storage capacity this analysis demonstrates the principles of specialization and adaptation in neural networks

particle filters for state estimation of jump markov linear systems jump markov linear systems jmls are linear systems whose parameters evolve with time according to a finite state markov chain in this paper our aim is to recursively compute optimal state estimates for this class of systems we present efficient simulation based algorithms called particle filters to solve the optimal filtering problem as well as the optimal fixed lag smoothing problem our algorithms combine sequential importance sampling a selection scheme and markov chain monte carlo methods they use several variance reduction methods to make the most of the statistical structure of jmls computer simulations are carried out to evaluate the performance of the proposed algorithms the problems of on line deconvolution of impulsive processes and of tracking a maneuvering target are considered it is shown that our algorithms outperform the current methods

brief paper detection and estimation for abruptly changing systems the problem of state estimation and system structure detection for discrete stochastic dynamical systems with parameters which may switch among a finite set of values is considered the switchings are modelled by a markov chain with known transition probabilities a brief survey and a unified treatment of the existing suboptimal algorithms are provided the optimal algorithms require exponentially increasing memory and computations with time simulation results comparing the various suboptimal algorithms are presented

large systems of random linear equations with nonnegative solutions characterizing the solvable and the unsolvable phase large systems of linear equations are ubiquitous in science quite often e g when considering population dynamics or chemical networks the solutions must be nonnegative recently it has been shown that large systems of random linear equations exhibit a sharp transition from a phase where a nonnegative solution exists with probability one to one where typically no such solution may be found the critical line separating the two phases was determined by combining farkas lemma with the replica method here we show that the same methods remain viable to characterize the two phases away from criticality to this end we analytically determine the residual norm of the system in the unsolvable phase and a suitable measure of robustness of solutions in the solvable one our results are in very good agreement with numerical simulations

replica symmetry breaking in perceptrons in the problem of optimization of pattern stabilization in perceptrons the replica symmetric ansatz is known to be mathematically unstable for storage capacities greater than some in this paper we demonstrate that for greater than the one step replica symmetry broken rsb solution is also unstable we further show that in this region full rsb is necessary for an exact solution direct evaluation of the two step rsb solution yields a minimum storage error which is only slightly greater than the one step rsb which itself is greater than that given by the unstable replica symmetric ansatz by a much larger amount

efficient sequential monte carlo with multiple proposals and control variates sequential monte carlo is a useful simulation based method for online filtering of state space models for certain complex state space models a single proposal distribution is usually not satisfactory and using multiple proposal distributions is a general approach to address various aspects of the filtering problem this article proposes an efficient method of using multiple proposals in combination with control variates the likelihood approach of tan 2004 is used in both resampling and estimation the new algorithm is shown to be asymptotically more efficient than the direct use of multiple proposals and control variates the guidance for selecting multiple proposals and control variates is also given numerical examples are used to demonstrate that the new algorithm can significantly improve over the bootstrap filter and auxiliary particle filter

beyond storage capacity in a single model neuron continuous replica symmetry breaking a single mcculloch pitts neuron that is the simple perceptron is studied with focus on the region beyond storage capacity it is shown that parisi s hierarchical ansatz for the overlap matrix of the synaptic couplings with so called continuous replica symmetry breaking is a solution and as we propose it is the exact one to the equilibrium problem we describe some of the most salient features of the theory and give results about the low temperature region in particular the basics of the parisi technique and the way to calculate thermodynamical expectation values is explained we have numerically extremized the replica free energy functional for some parameter settings and thus obtained the order parameter function i e the probability distribution of overlaps that enabled us to evaluate the probability density of the local stability parameter we also performed a simulation and found a local stability density closer to the theoretical curve than previous numerical results were

marginalized particle filters for mixed linear nonlinear state space models the particle filter offers a general numerical tool to approximate the posterior density function for the state in nonlinear and non gaussian filtering problems while the particle filter is fairly easy to implement and tune its main drawback is that it is quite computer intensive with the computational complexity increasing quickly with the state dimension one remedy to this problem is to marginalize out the states appearing linearly in the dynamics the result is that one kalman filter is associated with each particle the main contribution in this paper is the derivation of the details for the marginalized particle filter for a general nonlinear state space model several important special cases occurring in typical signal processing applications will also be discussed the marginalized particle filter is applied to an integrated navigation system for aircraft it is demonstrated that the complete high dimensional system can be based on a particle filter using marginalization for all but three states excellent performance on real flight data is reported

independent particle filters sequential monte carlo methods especially the particle filter pf and its various modifications have been used effectively in dealing with stochastic dynamic systems the standard pf samples the current state through the underlying state dynamics then uses the current observation to evaluate the sample s importance weight however there is a set of problems in which the current observation provides significant information about the current state but the state dynamics are weak and thus sampling using the current observation often produces more efficient samples than sampling using the state dynamics in this article we propose a new variant of the pf the independent particle filter ipf to deal with these problems the ipf generates exchangeable samples of the current state from a sampling distribution that is conditionally independent of the previous states a special case of which uses only the current observation each sample can then be matched with multiple samples of the previous states in

asymptotic behaviour in learning from stochastic examples one step rsb calculation of the learning curve the asymptotic learning curve of a n dimensional stochastic learning model is calculated in the statistical mechanical framework for the n 1 case kabashima and shinomoto 1992 gave a power law with exponent however this is inconsistent with the rs solution with the exponent obtained by gyorgyi and tishby 1990 we show that the one step rsb solution is consistent with kabashima and shinomoto s result up to a logarithmic correction

storage capacity of a constructive learning algorithm upper and lower bounds for the typical storage capacity of a constructive algorithm the tilinglike learning algorithm for the parity machine biehl m and opper m 1991 phys rev a 44 6888 are determined in the asymptotic limit of large training set sizes the properties of a perceptron with threshold learning a training set of patterns having a biased distribution of targets needed as an intermediate step in the capacity calculation are determined analytically the lower bound for the capacity determined with a cavity method is proportional to the number of hidden units the upper bound obtained with the hypothesis of replica symmetry is close to the one predicted by mitchinson and durbin 1989 biol cybern 60 345

approximate conditional mean particle filtering for linear nonlinear dynamic state space models we consider linear systems whose state parameters are separable into linear and nonlinear sets and evolve according to some known transition distribution and whose measurement noise is distributed according to a mixture of gaussians in doing so we propose a novel particle filter that addresses the optimal state estimation problem for the aforementioned class of systems the proposed filter referred to as the approximate conditional mean particle filter acm pf is a combination of the approximate conditional mean filter and the sequential importance sampling particle filter the algorithm development depends on approximating a mixture of gaussians distribution with a moment matched gaussian in the weight update recursion a condition indicating when this approximation is valid is given in order to evaluate the performance of the proposed algorithm we address the blind signal detection problem for an impulsive flat fading channel and the tracking of a maneuvering target in the presence of glint noise extensive computer simulations were carried out for computationally intensive implementations large number of particles the proposed algorithm offers performance that is comparable to other state of the art particle filtering algorithms in the scenario where computational horsepower is heavily constrained it is shown that the proposed algorithm offers the best performance amongst the considered algorithms for these specific examples

weight space structure and generalization in the reversed wedge perceptron the generalization ability of the reversed wedge perceptron serving as a toy model for multilayer neural networks is investigated we analyse the decomposition of the version space into disjoint cells belonging to different internal representations defined by the signs of the aligning fields the version space is characterized by the number and size of these cells and their typical overlap with the teacher network for a small training set the system is unable to detect the structure of the patterns induced by the teacher accordingly it performs as if storing random input output patterns with very low generalization ability and a large misfit in the internal representation with increasing training set size cells with large misfit are eliminated at a much higher rate than those with internal representation similar to that of the teacher this results eventually in the discontinuous phase transition to good generalization typical for multilayer neural networks

perceptrons above saturation we study the storage of random patterns by a perceptron above its storage capacity alpha c i e in the region where perfect storage becomes impossible we determine the minimal fraction of learning errors and the distribution of stabilities for different learning rules in one step replica symmetry breaking thereby we not only extend the known replica symmetric results to values of the storage capacity beyond the at line but also show that depending on the learning rule replica symmetry may be globally unstable already well below the at line as an example for possible implications we compare the results for the typical basins of attraction of an extremely diluted attractor neural network as given by replica symmetry and one step replica symmetry breaking

maximal stability in unsupervised learning a perceptron can classify a set of random patterns into two groups the maximal gap between the two groups is calculated using replica theory the replica symmetric solution is shown to be unstable and gives results which are qualitatively different from the one step replica symmetry breaking solution rsb1 the results of a calculation without the replica method are given yielding an exact upper bound for the gap which almost coincides with the rsb1 solution finding the classification of a set of patterns with a maximal gap is a difficult combinatorial optimization problem an algorithm is developed which gives large gaps

maximum likelihood sequence estimation of cpm signals transmitted over rayleigh flat fading channels a method for the sequential updating of log likelihood functions for maximum likelihood sequence estimation is presented it is shown that in a general case this method can be implemented using kalman filtering techniques for the special case of rayleigh flat fading and continuous phase modulation cpm signaling this approach is shown to lead to an attractive receiver structure this type of receiver called the linear predictive receiver can be implemented in the form of the viterbi algorithm with the trellis updates being computed using a bank of finite pulse response fir filter and square operations simulation results are presented that demonstrate the superiority of the linear predictive receiver over receivers employing differential detection in the presence of fast fading it is shown that the optimal linear predictive receiver does not possess an irreducible error rate for a class of rayleigh fading channels used to model terrestrial mobile channels

particle filtering for large dimensional state spaces with multimodal observation likelihoods we study efficient importance sampling techniques for particle filtering pf when either a the observation likelihood ol is frequently multimodal or heavy tailed or b the state space dimension is large or both when the ol is multimodal but the state transition pdf stp is narrow enough the optimal importance density is usually unimodal under this assumption many techniques have been proposed but when the stp is broad this assumption does not hold we study how existing techniques can be generalized to situations where the optimal importance density is multimodal but is unimodal conditioned on a part of the state vector sufficient conditions to test for the unimodality of this conditional posterior are derived our result is directly extendable to testing for unimodality of any posterior the number of particles n to accurately track using a pf increases with state space dimension thus making any regular pf impractical for large dimensional tracking problems but in most such problems most of the state change occurs in only a few dimensions while the change in the rest of the dimensions is small using this property we propose to replace importance sampling from a large part of the state space whose conditional posterior is narrow enough by just tracking the mode of the conditional posterior this introduces some extra error but it also greatly reduces the importance sampling dimension the net effect is much smaller error for a given n especially when the available n is small an important class of large dimensional problems with multimodal ol is tracking spatially varying physical quantities such as temperature or pressure in a large area using a network of sensors which may be nonlinear and or may have nonnegligible failure probabilities improved performance of our proposed algorithms over existing pfs is demonstrated for this problem

the shifted rayleigh mixture filter for bearings only tracking of maneuvering targets this paper introduces the shifted rayleigh mixture filter srmf which is based on jump markov linear systems the formulation permits the presence of clutter for bearings only tracking problems involving maneuvering targets the conditional density of the target state given the available measurements evolves as a growing mixture of probability density functions associated with a history of manoeuvre modes similar to other mixture algorithms the srmf approximates this conditional density by a gaussian mixture of fixed order unlike the extended or unscented kalman filters the shifted rayleigh filter incorporates an exact calculation of the posterior density when the prior is assumed to be gaussian given the latest bearings measurement computer simulations are provided to demonstrate the performance of the algorithm

optimal capacity of graded response perceptrons a replica symmetry breaking solution optimal capacities of perceptrons with graded input output relations are studied within the first step replica symmetry breaking gardner approach input data errors and a limited output precision are allowed in particular the role of non monotonicity in the input output relations on the breaking and on the overall performance is determined

dynamic conditional independence models and markov chain monte carlo methods abstract in dynamic statistical modeling situations observations arise sequentially causing the model to expand by progressive incorporation of new data items and new unknown parameters for example in clinical monitoring patients and data arrive sequentially and new patient specific parameters are introduced with each new patient markov chain monte carlo mcmc might be used for continuous updating of the evolving posterior distribution but would need to be restarted from scratch at each expansion stage thus mcmc methods are often too slow for real time inference in dynamic contexts by combining mcmc with importance resampling we show how real time sequential updating of posterior distributions can be effected the proposed dynamic sampling algorithms use posterior samples from previous updating stages and exploit conditional independence between groups of parameters to allow samples of parameters no longer of interest to be discarded such as when a patient dies or is discharged we apply the

sequential imputations and bayesian missing data problems abstract for missing data problems tanner and wong have described a data augmentation procedure that approximates the actual posterior distribution of the parameter vector by a mixture of complete data posteriors their method of constructing the complete data sets is closely related to the gibbs sampler both required iterations and similar to the em algorithm convergence can be slow we introduce in this article an alternative procedure that involves imputing the missing data sequentially and computing appropriate importance sampling weights in many applications this new procedure works very well without the need for iterations sensitivity analysis influence analysis and updating with new data can be performed cheaply bayesian prediction and model selection can also be incorporated examples taken from a wide range of applications are used for illustration

an ell _ 1 laplace robust kalman smoother robustness is a major problem in kalman filtering and smoothing that can be solved using heavy tailed distributions e g l1 laplace this paper describes an algorithm for finding the maximum a posteriori map estimate of the kalman smoother for a nonlinear model with gaussian process noise and l1 laplace observation noise the algorithm uses the convex composite extension of the gauss newton method this yields convex programming subproblems to which an interior point path following method is applied the number of arithmetic operations required by the algorithm grows linearly with the number of time points because the algorithm preserves the underlying block tridiagonal structure of the kalman smoother problem excellent fits are obtained with and without outliers even though the outliers are simulated from distributions that are not l1 laplace it is also tested on actual data with a nonlinear measurement model for an underwater tracking experiment the l1 laplace smoother is able to construct a smoothed fit without data removal from data with very large outliers

on line inference for hidden markov models via particle filters we consider the on line bayesian analysis of data by using a hidden markov model where inference is tractable conditional on the history of the state of the hidden component a new particle filter algorithm is introduced and shown to produce promising results when analysing data of this type the algorithm is similar to the mixture kalman filter but uses a different resampling algorithm we prove that this resampling algorithm is computationally efficient and optimal among unbiased resampling algorithms in terms of minimizing a squared error loss function in a practical example that of estimating break points from well log data our new particle filter outperforms two other particle filters one of which is the mixture kalman filter by between one and two orders of magnitude

mixture kalman filters in treating dynamic systems sequential monte carlo methods use discrete samples to represent a complicated probability distribution and use rejection sampling importance sampling and weighted resampling to complete the on line filtering task we propose a special sequential monte carlo method the mixture kalman filter which uses a random mixture of the gaussian distributions to approximate a target distribution it is designed for on line estimation and prediction of conditional and partial conditional dynamic linear models which are themselves a class of widely used non linear systems and also serve to approximate many others compared with a few available filtering methods including monte carlo methods the gain in efficiency that is provided by the mixture kalman filter can be very substantial another contribution of the paper is the formulation of many non linear systems into conditional or partial conditional linear form to which the mixture kalman filter can be applied examples in target tracking and digital communications are given to demonstrate the procedures proposed

particle filtering with dependent noise processes modeling physical systems often leads to discrete time state space models with dependent process and measurement noises for linear gaussian models the kalman filter handles this case as is well described in literature however for nonlinear or non gaussian models the particle filter as described in literature provides a general solution only for the case of independent noise here we present an extended theory of the particle filter for dependent noises with the following key contributions i the optimal proposal distribution is derived ii the special case of gaussian noise in nonlinear models is treated in detail leading to a concrete algorithm that is as easy to implement as the corresponding kalman filter iii the marginalized rao blackwellized particle filter handling linear gaussian substructures in the model in an efficient way is extended to dependent noise and finally iv the parameters of a joint gaussian distribution of the noise processes are estimated jointly with the state in a recursive way

gaussian sum particle filtering we use the gaussian particle filter to build several types of gaussian sum particle filters these filters approximate the filtering and predictive distributions by weighted gaussian mixtures and are basically banks of gaussian particle filters then we extend the use of gaussian particle filters and gaussian sum particle filters to dynamic state space dss models with non gaussian noise with non gaussian noise approximated by gaussian mixtures the non gaussian noise models are approximated by banks of gaussian noise models and gaussian mixture filters are developed using algorithms developed for gaussian noise dss models as a result problems involving heavy tailed densities can be conveniently addressed simulations are presented to exhibit the application of the framework developed herein and the performance of the algorithms is examined

adaptive joint detection and estimation in mimo systems a hybrid systems approach an adaptive receiver based on hybrid system theory is developed for a multiuser multiple input multiple output mimo fading code division multiple access cdma system the basic idea is to treat the transmitted symbols and channel gains as unknown states discrete and continuous within a hybrid systems framework the bayesian inference based state estimation is derived using multiple model theory resulting in an optimal joint sequence estimator which is shown to be intractable in its computational complexity a suboptimal receiver imm sic is then derived based on the well known interacting multiple model imm algorithm and successive interference cancellation sic scheme this paper shows the specific approximations made to the probability densities of the optimal receiver in deriving the imm sic receiver with complexity linear in number of users this receiver design is well suited for online recursive processing of space time coded cdma system where the decoding stage is incorporated within the multiple model framework

a blind particle filtering detector of signals transmitted over flat fading channels a new particle filtering detector pfd is proposed for blind signal detection over flat rayleigh fading channels whose model coefficients are unknown the detector employs a hybrid importance function and a mixture kalman filter it also incorporates an auxiliary particle filtering strategy with a smoothing kernel in the resampling step further by considering practical information of communication systems and the physical interpretation of the adopted second order autoregressive ar channel model a fully blind particle filtering implementation is developed the structure of the proposed pfd can be easily adapted to other system requirements simulations are provided that demonstrate the performance of the new pfd

particle filtering for partially observed gaussian state space models summary solving bayesian estimation problems where the posterior distribution evolves over time through the accumulation of data has many applications for dynamic models a large number of algorithms based on particle filtering methods also known as sequential monte carlo algorithms have recently been proposed to solve these problems we propose a special particle filtering method which uses random mixtures of normal distributions to represent the posterior distributions of partially observed gaussian state space models this algorithm is based on a marginalization idea for improving efficiency and can lead to substantial gains over standard algorithms it differs from previous algorithms which were only applicable to conditionally linear gaussian state space models computer simulations are carried out to evaluate the performance of the proposed algorithm for dynamic tobit and probit models

on sequential data assimilation for scalar macroscopic traffic flow models a b s t r a c t we consider the problem of sequential data assimilation for transportation networks using optimal filtering with a scalar macroscopic traffic flow model properties of the distribution of the uncertainty on the true state related to the specific nonlinearity and non differentiability inherent to macroscopic traffic flow models are investigated derived analytically and analyzed we show that nonlinear dynamics by creating discontinuities in the traffic state affect the performances of classical filters and in particular that the distribution of the uncertainty on the traffic state at shock waves is a mixture distribution the non differentiability of traffic dynamics around stationary shock waves is also proved and the resulting optimality loss of the estimates is quantified numerically the properties of the estimates are explicitly studied for the godunov scheme and thus the cell transmission model leading to specific conclusions about their use in the context of filtering which is a significant contribution of this article analytical proofs and numerical tests are introduced to support the results presented a java implementation of the classical filters used in this work is available on line at http traffic berkeley edu for facilitating further efforts on this topic and fostering reproducible research

data assimilation by conditioning of driving noise on future observations this work was supported by king abdullah university of science and technology kaust award no kuk c1 013 04

quantum chaos in one dimension in this work we investigate the inverse of the celebrated bohigas giannoni schmit conjecture using two inversion methods we compute a one dimensional potential whose lowest n eigenvalues obey random matrix statistics our numerical results indicate that in the asymptotic limit n yields infinity the solution is nowhere differentiable and most probably nowhere continuous thus such a counterexample does not exist

bayesian conditional monte carlo algorithms for nonlinear time series state estimation bayesian filtering aims at estimating sequentially a hidden process from an observed one in particular sequential monte carlo smc techniques propagate in time weighted trajectories which represent the posterior probability density function pdf of the hidden process given the available observations on the other hand conditional monte carlo cmc is a variance reduction technique which replaces the estimator of a moment of interest by its conditional expectation given another variable in this paper we show that up to some adaptations one can make use of the time recursive nature of smc algorithms in order to propose natural temporal cmc estimators of some point estimates of the hidden process which outperform the associated crude monte carlo mc estimator whatever the number of samples we next show that our bayesian cmc estimators can be computed exactly or approximated efficiently in some hidden markov chain hmc models in some jump markov state space systems jmss as well as in multitarget filtering finally our algorithms are validated via simulations

gardner optimal capacity of the diluted blume emery griffiths neural network abstract the optimal capacity of a diluted blume emery griffiths neural network is studied as a function of the pattern activity and the embedding stability using the gardner entropy approach annealed dilution is considered cutting some of the couplings referring to the ternary patterns themselves and some of the couplings related to the active patterns both simultaneously synchronous dilution or independently asynchronous dilution through the de almeida thouless criterion it is found that the replica symmetric solution is locally unstable as soon as there is dilution the distribution of the couplings shows the typical gap with a width depending on the amount of dilution but this gap persists even in cases where a particular type of coupling plays no role in the learning process

optimal storage properties of neural network models the authors calculate the number p alpha n of random n bit patterns that an optimal neural network can store allowing a given fraction f of bit errors and with the condition that each right bit is stabilised by a local field at least equal to a parameter k for each value of alpha and k there is a minimum fraction fmin of wrong bits they find a critical line alpha c k with alpha c 0 2 the minimum fraction of wrong bits vanishes for alpha alpha c k the calculations are done using a saddle point method and the order parameters at the saddle point are assumed to be replica symmetric this solution is locally stable in a finite region of the k alpha plane including the line alpha c k but there is a line above which the solution becomes unstable and replica symmetry must be broken

following a moving target monte carlo inference for dynamic bayesian models markov chain monte carlo mcmc sampling is a numerically intensive simulation technique which has greatly improved the practicality of bayesian inference and prediction however mcmc sampling is too slow to be of practical use in problems involving a large number of posterior target distributions as in dynamic modelling and predictive model selection alternative simulation techniques for tracking moving target distributions known as particle filters which combine importance sampling importance resampling and mcmc sampling tend to suffer from a progressive degeneration as the target sequence evolves we propose a new technique based on these same simulation methodologies which does not suffer from this progressive degeneration

filtering via simulation auxiliary particle filters this article analyses the recently suggested particle approach to filtering time series we suggest that the algorithm is not robust to outliers for two reasons the design of the simulators and the use of the discrete support to represent the sequentially updating prior distribution here we tackle the first of these problems

riemann zeros and a fractal potential the nontrivial riemann zeroes are reproduced using a one dimensional local potential model a close look at the potential suggests that it has a fractal structure of dimension d 1 5

the calculation of posterior distributions by data augmentation comment a noniterative sampling importance resampling alternative to the data augmentation algorithm for creating a few imputations when fractions of missing information are modest the sir algorithm

nyquist 3 pulse shaping in continuous phase modulation continuous phase modulation cpm schemes using nyquist 3 pulse shaping and correlative codes are popular candidates for mobile radio applications this paper presents a detailed study of the effects of nyquist 3 pulse shaping on the spectral characteristics of cpm schemes with a modulation index of 1 2 the influence of pulse parameters such as its shape bandwidth coding polynomial and truncation length on spectral characteristics such as main lobe fractional power bandwidths and out of band power levels is thoroughly investigated error performance is also briefly analyzed among uncoded schemes a new scheme using a pulse satisfying both nyquist 3 and nyquist 2 criteria is pointed out as a promising candidate and as a good compromise between spectral and power efficiencies

threshold induced phase transitions in perceptrons error rates of a boolean perceptron with threshold and either spherical or ising constraint on the weight vector are calculated for storing patterns from biased input and output distributions derived within a one step replica symmetry breaking rsb treatment for unbiased output distribution and non zero stability of the patterns we find a critical load p above which two solutions to the saddlepoint equations appear one with higher free energy and zero threshold and a dominant solution with non zero threshold we examine this second order phase transition and the dependence of p on the required pattern stability for both one step rsb and replica symmetry rs in the spherical case and for one step rsb in the ising case

particle filter with a mode tracker for visual tracking across illumination changes in this correspondence our goal is to develop a visual tracking algorithm that is able to track moving objects in the presence of illumination variations in the scene and that is robust to occlusions we treat the illumination and motion x y translation and scale parameters as the unknown state sequence the observation is the entire image and the observation model allows for occasional occlusions modeled as outliers the nonlinearity and multimodality of the observation model necessitate the use of a particle filter pf due to the inclusion of illumination parameters the state dimension increases thus making regular pfs impractically expensive we show that the recently proposed approach using a pf with a mode tracker can be used here since even in most occlusion cases the posterior of illumination conditioned on motion and the previous state is unimodal and quite narrow the key idea is to importance sample on the motion states while approximating importance sampling by posterior mode tracking for estimating illumination experiments demonstrate the advantage of the proposed algorithm over existing pf based approaches for various face and vehicle tracking we are also able to detect illumination model changes e g those due to transition from shadow to sunlight or vice versa by using the generalized expected log likelihood statistics and successfully compensate for it without ever loosing track

numerical schemes for dynamically orthogonal equations of stochastic fluid and ocean flows the quantification of uncertainties is critical when systems are nonlinear and have uncertain terms in their governing equations or are constrained by limited knowledge of initial and boundary conditions such situations are common in multiscale intermittent and non homogeneous fluid and ocean flows the dynamically orthogonal do field equations provide an adaptive methodology to predict the probability density functions of such flows the present work derives efficient computational schemes for the do methodology applied to unsteady stochastic navier stokes and boussinesq equations and illustrates and studies the numerical aspects of these schemes semi implicit projection methods are developed for the mean and for the do modes and time marching schemes of first to fourth order are used for the stochastic coefficients conservative second order finite volumes are employed in physical space with new advection schemes based on total variation diminishing methods other results include i the definition of pseudo stochastic pressures to obtain a number of pressure equations that is linear in the subspace size instead of quadratic ii symmetric advection schemes for the stochastic velocities iii the use of generalized inversion to deal with singular subspace covariances or deterministic modes and iv schemes to maintain orthonormal modes at the numerical level to verify our implementation and study the properties of our schemes and their variations a set of stochastic flow benchmarks are defined including asymmetric dirac and symmetric lock exchange flows lid driven cavity flows and flows past objects in a confined channel different reynolds number and grashof number regimes are employed to illustrate robustness optimal convergence under both time and space refinements is shown as well as the convergence of the probability density functions with the number of stochastic realizations

adaptive bayesian multiuser detection for synchronous cdma with gaussian and impulsive noise we consider the problem of simultaneous parameter estimation and data restoration in a synchronous cdma system in the presence of either additive gaussian or additive impulsive white noise with unknown parameters the impulsive noise is modeled by a two term gaussian mixture distribution bayesian inference of all unknown quantities is made from the superimposed and noisy received signals the gibbs sampler a markov chain monte carlo procedure is employed to calculate the bayesian estimates the basic idea is to generate ergodic random samples from the joint posterior distribution of all unknown and then to average the appropriate samples to obtain the estimates of the unknown quantities adaptive bayesian multiuser detectors based on the gibbs sampler are derived for both the gaussian noise synchronous cdma channel and the impulsive noise synchronous cdma channel a salient feature of the proposed adaptive bayesian multiuser detectors is that they can incorporate the a priori symbol probabilities and they produce as output the a posteriori symbol probabilities that is they are soft input soft output algorithms hence these methods are well suited for iterative processing in a coded system which allows the adaptive bayesian multiuser detector to refine its processing based on the information from the decoding stage and vice versa a receiver structure termed the adaptive turbo multiuser detector

blind deconvolution via sequential imputations abstract the sequential imputation procedure is applied to adaptively and sequentially reconstruct discrete input signals that are blurred by an unknown linear moving average channel and contaminated by additive gaussian noises a problem known as blind deconvolution in digital communication a rejuvenation procedure for improving the efficiency of sequential imputation is introduced and theoretically justified the proposed method does not require the channel to be nonminimum phase and can be used in real time signal restoration two simulated systems are studied to illustrate the proposed method our result shows that the ideas of multiple imputations and flexible simulation techniques are as powerful in engineering as in survey sampling

covariance structure of the gibbs sampler with applications to the comparisons of estimators and augmentation schemes summary we study the covariance structure of a markov chain generated by the gibbs sampler with emphasis on data augmentation when applied to a bayesian missing data problem the gibbs sampler produces two natural approximations for the posterior distribution of the parameter vector the empirical distribution based on the sampled values of the parameter vector and a mixture of complete data posteriors we prove that rao blackwellization causes a one lag delay for the autocovariances among dependent samples obtained from data augmentation and consequently the mixture approximation produces estimates with smaller variances than the empirical approximation the covariance structure results are used to compare different augmentation schemes it is shown that collapsing and grouping random components in a gibbs sampler with two or three components usually result in more efficient sampling schemes

inference and model choice for sequentially ordered hidden markov models summary the system equation of a hidden markov model is rewritten to label the components by order of appearance and to make explicit the random behaviour of the number of components mt we argue that this reformulation is often a good way to achieve identifiability as it facilitates the interpretation of the posterior density and the estimation of the number of components that have appeared in a given sample we develop a sequential monte carlo algorithm for estimating the reformulated model which relies on particle filtering and gibbs sampling our algorithm has a computational cost that is similar to that of a markov chain monte carlo sampler and is much less likely to be affected by label switching i e the possibility of becoming trapped in a local mode of the posterior density the extension to transdimensional priors is also considered the approach is illustrated by two real data examples

local momentum autoregression and the modeling of interest rate term structure abstract a parsimonious autoregressive model that is globally mean reverting but locally driven by momentum is proposed the local momentum autoregression lm ar model carries one extra parameter and depending on the sign of this extra parameter it can be either local momentum preserving or momentum building the lm ar model is motivated by observing us interest rate movement over many decades which over a long time span seems to mean revert but over a period of several months or years can actually exhibit a momentum like behavior we use the lm ar model with a stochastic central tendency factor as the dominant global risk factor in interest rates and add a local variation component of the standard mean reverting type to create a 3 factor risk environment we then derive its corresponding term structure model and empirically implement the model on us interest rates of seven maturities from january 1954 to december 2013 on a weekly frequency to establish the presence of local momentum building

fuzzy particle filtering for uncertain systems in this paper we propose a novel fuzzy particle filtering method for online estimation of nonlinear dynamic systems with fuzzy uncertainties this approach uses a sequential fuzzy simulation to approximate the possibilities of the state intervals in the state space and estimates the state by fuzzy expected value operator to solve the degeneracy problem of the fuzzy particle filter one corresponding resampling technique is introduced in addition we compare the fuzzy particle filter with ordinary particle filter in both aspects of the theoretical basis and algorithm design and demonstrate that the proposed filter outperforms standard particle filters especially when the number of the particles is small the numerical simulations of two continuous state nonlinear systems and a jump markov system are employed to show the effectiveness and robustness of the proposed fuzzy particle filter

a low complexity sequential monte carlo algorithm for blind detection in mimo systems in statistical signal processing the sequential monte carlo smc method is powerful and can approach the theoretical optima however its computational complexity is usually very high especially in multiple input multiple output mimo systems this paper presents a new low complexity smc lc smc algorithm for blind detection in mimo systems the main idea of which is to shrink the sampling space via channel estimation which is initialized using the first differentially modulated symbol and then updated using the monte carlo samples since the a posteriori probability of the transmitted symbols can be calculated separately by each transmit antenna the proposed lc smc algorithm is not only computationally efficient as compared to the original smc whose complexity grows exponentially with the number of transmit antennas but also makes blind turbo receiver more feasible for multilevel phase modulations simulation results are presented to demonstrate the effectiveness of the lc smc algorithm

learning of non monotonic rules by simple perceptrons in this paper we study the generalization ability of a simple perceptron which learns an unrealizable boolean function represented by a perceptron with a non monotonic transfer function of reversed wedge type this type of non monotonic perceptron is considered as a variant of multilayer perceptron and is parametrized by a single wedge parameter a reflecting the non monotonic nature of the target function a discontinuous transition from the poor generalization phase to the good generalization phase is observed in the learning curve for intermediate values of a we also find that asymptotic learning curves are classified into the following two categories depending on a for large a the learning curve obeys a power law with exponent 1 on the other hand a power law with exponent is obtained for small a although these two exponents are obtained from unstable replica symmetric solutions by using the replica method they are consistent with the results obtainable without using the replica method in a low dimensional version of this learning problem this suggests that our results are good approximations even if they are not exact

particle filtering for demodulation in fading channels with non gaussian additive noise an efficient particle filtering algorithm is developed to solve the problem of demodulation of m ary modulated signals under conditions of fading channels in the presence of non gaussian additive noise simulations for mdpsk signals are presented the results show that the algorithm outperforms the current methods