ideal spatial adaptation by wavelet shrinkage summary with ideal spatial adaptation an oracle furnishes information about how best to adapt a spatially variable estimator whether piecewise constant piecewise polynomial variable knot spline or variable bandwidth kernel to the unknown function estimation with the aid of an oracle offers dramatic advantages over traditional linear estimation by nonadaptive kernels however it is a priori unclear whether such performance can be obtained by a procedure relying on the data alone we describe a new principle for spatially adaptive estimation selective wavelet reconstruction we show that variable knot spline fits and piecewise polynomial fits when equipped with an oracle to select the knots are not dramatically more powerful than selective wavelet reconstruction with an oracle we develop a practical spatially adaptive method riskshrink which works by shrinkage of empirical wavelet coefficients riskshrink mimics the performance of an oracle for selective wavelet reconstruction as well as it is possible to do so a new inequality in multivariate normal decision theory which we call the oracle inequality shows that attained performance differs from ideal performance by at most a factor of approximately 2 log n where n is the sample size moreover no estimator can give a better guarantee than this within the class of spatially adaptive procedures riskshrink is essentially optimal relying only on the data it comes within a factor log 2 n of the performance of piecewise polynomial and variableknot spline methods equipped with an oracle in contrast it is unknown how or if piecewise polynomial methods could be made to function this well when denied access to an oracle and forced to rely on data alone

a sequential particle filter method for static models summary particle filter methods are complex inference procedures which combine importance sampling and monte carlo schemes in order to explore consistently a sequence of multiple distributions of interest we show that such methods can also offer an efficient estimation

on line inference for multiple changepoint problems we propose an on line algorithm for exact filtering of multiple changepoint problems this algorithm enables simulation from the true joint posterior distribution of the number and position of the changepoints for a class of changepoint models the computational cost of this exact algorithm is quadratic in the number of observations we further show how resampling ideas from particle filters can be used to reduce the computational cost to linear in the number of observations at the expense of introducing small errors and propose two new optimum resampling algorithms for this problem one a version of rejection control allows the particle filter to automatically choose the number of particles required at each time step the new resampling algorithms substantially out perform standard resampling algorithms on examples we consider and we demonstrate how the resulting particle filter is practicable for segmentation of human gc content

statistical inference in hidden markov models using k segment constraints hidden markov models hmms are one of the most widely used statistical methods for analyzing sequence data however the reporting of output from hmms has largely been restricted to the presentation of the most probable map hidden state sequence found via the viterbi algorithm or the sequence of most probable marginals using the forward backward algorithm in this article we expand the amount of information we could obtain from the posterior distribution of an hmm by introducing linear time dynamic programming recursions that conditional on a user specified constraint in the number of segments allow us to i find map sequences ii compute posterior probabilities and iii simulate sample paths we collectively call these recursions k segment algorithms and illustrate their utility using simulated and real examples we also highlight the prospective and retrospective use of k segment constraints for fitting hmms or exploring existing model fits supplementary materials for this article are available online

particle markov chain monte carlo methods summary markov chain monte carlo and sequential monte carlo methods have emerged as the two main tools to sample from high dimensional probability distributions although asymptotic convergence of markov chain monte carlo algorithms is ensured under weak assumptions the performance of these algorithms is unreliable when the proposal distributions that are used to explore the space are poorly chosen and or if highly correlated variables are updated independently we show here how it is possible to build efficient high dimensional proposal distributions by using sequential monte carlo methods this allows us not only to improve over standard markov chain monte carlo schemes but also to make bayesian inference feasible for a large class of statistical models where this was not previously so we demonstrate these algorithms on a non linear state space model and a levy driven stochastic volatility model

multiple changepoint fitting via quasilikelihood with application to dna sequence segmentation summary we consider situations where a step function with a variable number of steps provides an adequate model for a regression relationship while the variance of the observations depends on their mean this model provides for discontinuous jumps at changepoints and for constant means and error variances in between changepoints the basic statistical problem consists of identification of the number of changepoints their locations and the levels the function assumes in between we embed this problem into a quasilikelihood formulation and utilise the minimum deviance criterion to fit the model for the choice of the number of changepoints we discuss a modified schwarz criterion a dynamic programming algorithm makes the segmentation feasible for sequences of moderate length the performance of the segmentation method is demonstrated in an application to the segmentation of the bacteriophage l sequence

exact bayesian curve fitting and signal segmentation we consider regression models where the underlying functional relationship between the response and the explanatory variable is modeled as independent linear regressions on disjoint segments we present an algorithm for perfect simulation from the posterior distribution of such a model even allowing for an unknown number of segments and an unknown model order for the linear regressions within each segment the algorithm is simple can scale well to large data sets and avoids the problem of diagnosing convergence that is present with monte carlo markov chain mcmc approaches to this problem we demonstrate our algorithm on standard denoising problems on a piecewise constant ar model and on a speech segmentation problem

model change detection with the mdl principle we are concerned with the issue of detecting model changes in probability distributions we specifically consider the strategies based on the minimum description length mdl principle we theoretically analyze their basic performance from the two aspects data compression and hypothesis testing from the view of data compression we derive a new bound on the minimax regret for model changes here the mini max regret is defined as the minimum of the worst case code length relative to the least normalized maximum likelihood code length over all model changes from the view of hypothesis testing we reduce the model change detection into a simple hypothesis testing problem we thereby derive upper bounds on error probabilities for the mdl based model change test the error probabilities are valid for finite sample size and are related to the information theoretic complexity as well as the discrepancy measure of the hypotheses to be tested

the repeated insertion model for rankings missing link between two subset choice models several probabilistic models for subset choice have been proposed in the literature for example to explain approval voting data we show that marley et al s latent scale model is subsumed by falmagne and regenwetter s size independent model in the sense that every choice probability distribution generated by the former can also be explained by the latter our proof relies on the construction of a probabilistic ranking model which we label the repeated insertion model this model is a special case of marden s orthogonal contrast model class and in turn includes the classical mallows model as a special case we explore its basic properties as well as its relationship to fligner and verducci s multistage ranking model

bayesian analysis of isochores statistical identification of isochore structure the variation in large scale gc composition proportion of dna bases that are g or c as opposed to a or t of mammalian genomes is a necessary requirement for understanding both the evolution of base composition and the many genomic features such as mutation and recombination rates which covary with base composition we have developed a bayesian method for isochore analysis that we demonstrate to be more accurate than the commonly used binary segmentation approach implemented within the program isofinder the method accounts for both fine scale and large scale structure we adapt direct simulation methods to allow for iid samples from the posterior distribution of our model and provide an accurate approximation to this that can analyze data from a chromosome in a matter of seconds we apply our method to human chromosome 1 the resulting estimate of how gc content varies across this region is shown to be a better predictor of local recombination rates t

on line inference for hidden markov models via particle filters we consider the on line bayesian analysis of data by using a hidden markov model where inference is tractable conditional on the history of the state of the hidden component a new particle filter algorithm is introduced and shown to produce promising results when analysing data of this type the algorithm is similar to the mixture kalman filter but uses a different resampling algorithm we prove that this resampling algorithm is computationally efficient and optimal among unbiased resampling algorithms in terms of minimizing a squared error loss function in a practical example that of estimating break points from well log data our new particle filter outperforms two other particle filters one of which is the mixture kalman filter by between one and two orders of magnitude

sequential monte carlo samplers summary we propose a methodology to sample sequentially from a sequence of probability distributions that are defined on a common space each distribution being known up to a normalizing constant these probability distributions are approximated by a cloud of weighted random samples which are propagated over time by using sequential monte carlo methods this methodology allows us to derive simple algorithms to make parallel markov chain monte carlo algorithms interact to perform global optimization and sequential bayesian estimation and to compute ratios of normalizing constants we illustrate these algorithms for various integration tasks arising in the context of bayesian inference

estimation and comparison of multiple change point models this paper provides a new bayesian approach for models with multiple change points the centerpiece of the approach is a formulation of the change point model in terms of a latent discrete state variable that indicates the regime from which a particular observation has been drawn this state variable is specified to evolve according to a discrete time discrete state markov process with the transition probabilities constrained so that the state variable can either stay at the current value or jump to the next higher value this parameterization exactly reproduces the change point model the model is estimated by markov chain monte carlo methods using an approach that is based on chib 1996 this methodology is quite valuable since it allows for the fitting of more complex change point models than was possible before methods for the computation of bayes factors are also developed all the techniques are illustrated using simulated and real data sets

infinitely many constrained inequalities for the von neumann entropy we exhibit infinitely many new constrained inequalities for the von neumann entropy and show that they are independent of each other and the known inequalities obeyed by the von neumann entropy basically strong subadditivity the new inequalities were proved originally by makarychev for the shannon entropy using properties of probability distributions our approach extends the proof of the inequalities to the quantum domain and includes their independence for the quantum and also the classical cases

multiscale change point inference summary we introduce a new estimator the simultaneous multiscale change point estimator smuce for the change point problem in exponential family regression an unknown step function is estimated by minimizing the number of change points over the acceptance region of a multiscale test at a level the probability of overestimating the true number of change points k is controlled by the asymptotic null distribution of the multiscale test statistic further we derive exponential bounds for the probability of underestimating k by balancing these quantities will be chosen such that the probability of correctly estimating k is maximized all results are even non asymptotic for the normal case on the basis of these bounds we construct asymptotically honest confidence sets for the unknown step function and its change points at the same time we obtain exponential bounds for estimating the change point locations which for example yield the minimax rate up to a log term finally the simultaneous multiscale change point estimator achieves the optimal detection rate of vanishing signals as n even for an unbounded number of change points we illustrate how dynamic programming techniques can be employed for efficient computation of estimators and confidence regions the performance of the multiscale approach proposed is illustrated by simulations and in two cutting edge applications from genetic engineering and photoemission spectroscopy

rejection control and sequential importance sampling abstract we discuss ways of combining rejection sampling and importance sampling methods in monte carlo computations and demonstrate their usefulness in updating dynamic systems specifically we propose the rejection controlled sequential importance sampling rc sis algorithm which is designed to simultaneously reduce monte carlo variation and retain independent samples in sequential importance sampling the proposed method is demonstrated by three examples taken from econometrics hierarchical bayes analysis and digital telecommunications they all show significant improvements over previous results

reversible jump markov chain monte carlo computation and bayesian model determination markov chain monte carlo methods for bayesian computation have until recently been restricted to problems where the joint distribution of all variables has a density with respect to some fixed standard underlying measure they have therefore not been available for application to bayesian model determination where the dimensionality of the parameter vector is typically not fixed this paper proposes a new framework for the construction of reversible markov chain samplers that jump between parameter subspaces of differing dimensionality which is flexible and entirely constructive it should therefore have wide applicability in model determination problems the methodology is illustrated with applications to multiple change point analysis in one and two dimensions and to a bayesian comparison of binomial experiments

bayesian curve fitting using mcmc with applications to signal segmentation we propose some bayesian methods to address the problem of fitting a signal modeled by a sequence of piecewise constant linear in the parameters regression models for example autoregressive or volterra models a joint prior distribution is set up over the number of the changepoints knots their positions and over the orders of the linear regression models within each segment if these are unknown hierarchical priors are developed and as the resulting posterior probability distributions and bayesian estimators do not admit closed form analytical expressions reversible jump markov chain monte carlo mcmc methods are derived to estimate these quantities results are obtained for standard denoising and segmentation of speech data problems that have already been examined in the literature these results demonstrate the performance of our methods

mixture kalman filters in treating dynamic systems sequential monte carlo methods use discrete samples to represent a complicated probability distribution and use rejection sampling importance sampling and weighted resampling to complete the on line filtering task we propose a special sequential monte carlo method the mixture kalman filter which uses a random mixture of the gaussian distributions to approximate a target distribution it is designed for on line estimation and prediction of conditional and partial conditional dynamic linear models which are themselves a class of widely used non linear systems and also serve to approximate many others compared with a few available filtering methods including monte carlo methods the gain in efficiency that is provided by the mixture kalman filter can be very substantial another contribution of the paper is the formulation of many non linear systems into conditional or partial conditional linear form to which the mixture kalman filter can be applied examples in target tracking and digital communications are given to demonstrate the procedures proposed

product partition models for change point problems product partition models assume that observations in different components of a random partition of the data are independent if the probability distribution of random partitions is in a certain product form prior to making the observations it is also in product form given the observations the product model thus provides a convenient machinery for allowing the data to weight the partitions likely to hold and inference about particular future observations may then be made by first conditioning on the partition and then averaging over all partitions these models apply with special computational simplicity to change point problems where the partitions divide the sequence of observations into components within which different regimes hold we show with appropriate selection of prior product models that the observations can eventually determine approximately the true partition

a pairwise likelihood based approach for changepoint detection in multivariate time series models this paper develops a composite likelihood based approach for multiple changepoint estimation in multivariate time series we derive a criterion based on pairwise likelihood and minimum description length for estimating the number and locations of changepoints and for performing model selection in each segment the number and locations of the changepoints can be consistently estimated under mild conditions and the computation can be conducted efficiently with a pruned dynamic programming algorithm simulation studies and real data examples demonstrate the statistical and computational efficiency of the proposed method