adaptive bayesian wavelet shrinkage abstract when fitting wavelet based models shrinkage of the empirical wavelet coefficients is an effective tool for denoising the data this article outlines a bayesian approach to shrinkage obtained by placing priors on the wavelet coefficients the prior for each coefficient consists of a mixture of two normal distributions with different standard deviations the simple and intuitive form of prior allows us to propose automatic choices of prior parameters these parameters are chosen adaptively according to the resolution level of the coefficients typically shrinking high resolution frequency coefficients more heavily assuming a good estimate of the background noise level we obtain closed form expressions for the posterior means and variances of the unknown wavelet coefficients the latter may be used to assess uncertainty in the reconstruction several examples are used to illustrate the method and comparisons are made with other shrinkage methods

image denoising in steerable pyramid domain based on a local laplace prior this paper presents a new image denoising algorithm based on the modeling of coefficients in each subband of steerable pyramid employing a laplacian probability density function pdf with local variance this pdf is able to model the heavy tailed nature of steerable pyramid coefficients and the empirically observed correlation between the coefficient amplitudes within this framework we describe a novel method for image denoising based on designing both maximum a posteriori map and minimum mean squared error mmse estimators which relies on the zero mean laplacian random variables with high local correlation despite the simplicity of our spatially adaptive denoising method both in its concern and implementation our denoising results achieves better performance than several published methods such as bayes least squared gaussian scale mixture bls gsm technique that is a state of the art denoising technique

a stable normalized least mean fourth algorithm with improved transient and tracking behaviors the stability problems of the least mean fourth lmf algorithm put a limitation on its tracking capability the paper investigates the possibility of solving this problem via stabilization of the algorithm the analysis is done for a markov plant it is found that the available stable normalized lmf nlmf algorithm has a tracking limitation for high signal to noise ratio the paper presents a new stable nlmf algorithm that is free of this limitation mean square stability of the algorithm is proved expressions are derived for the minimum steady state mean square deviation msd and the corresponding convergence time the new algorithm outperforms the available stable nlmf algorithm in both the transient and steady states the new algorithm is also compared with the nlms algorithm when the adaptation parameter of each algorithm is set to the value that minimizes its steady state msd for large initial msd the algorithm outperforms the nlms algorithm even for gaussian noise for small initial msd the algorithm outperforms the nlms algorithm for sub gaussian noise while the situation is opposite for gaussian noise analytical results are supported by simulations

stability bound of the initial mean square deviation of high order stochastic gradient adaptive filtering algorithms the paper derives the stability bound of the initial mean square deviation of an adaptive filtering algorithm based on minimizing the 2 l th moment of the estimation error with l being an integer greater than 1 the analysis is done for a time invariant plant with even input probability density function dependence of the stability bound on the algorithm step size type of the noise distribution signal to noise ratio snr and l is studied it is shown that the stability bound is decreasing in the step size the stability bound decreases as the tail of the noise probability density function becomes lighter for the same steady state misadjustment this result is surprising since for example it is expected that the stability of the least mean 2 l th algorithm for binary noise distribution is better than that for uniform distribution the stability bound is proportional to the reciprocal of the snr another surprising result finally the stability bound is decreasing in l which implies a limitation on the ability to improve the performance of the adaptive filter by increasing the order of the cost function theoretical results are supported by simulations

efficient empirical bayes variable selection and estimation in linear models we propose an empirical bayes method for variable selection and coefficient estimation in linear regression models the method is based on a particular hierarchical bayes formulation and the empirical bayes estimator is shown to be closely related to the lasso estimator such a connection allows us to take advantage of the recently developed quick lasso algorithm to compute the empirical bayes estimate and provides a new way to select the tuning parameter in the lasso method unlike previous empirical bayes variable selection methods which in most practical situations can be implemented only through a greedy stepwise algorithm our method gives a global solution efficiently simulations and real examples show that the proposed method is very competitive in terms of variable selection estimation accuracy and computation speed compared with other variable selection and estimation methods

analysis in a finite time frequency plane we revise our conjecture concerning signals realizing the lower bound on compaction in the phase plane to include a forgotten case of periodization and indicate that this case can be used to develop an orthonormal basis for data lengths n that are the square of an integer

adapting to unknown sparsity by controlling the false discovery rate we attempt to recover an n dimensional vector observed in white noise where n is large and the vector is known to be sparse but the degree of sparsity is unknown we consider three different ways of defining sparsity of a vector using the fraction of nonzero terms imposing power law decay bounds on the ordered entries and controlling the lp norm for p small we obtain a procedure which is asymptotically minimax for l r loss simultaneously throughout a range of such sparsity classes the optimal procedure is a data adaptive thresholding scheme driven by control of the false discovery rate fdr fdr control is a relatively recent innovation in simultaneous testing ensuring that at most a certain fraction of the rejected null hypotheses will correspond to false rejections in our treatment the fdr control parameter qn also plays a determining role in asymptotic minimaxity if q lim qn 0 1 2 and also qn log n we get sharp asymptotic minimaxity simultaneously over a wide range of sparse parameter spaces and loss functions on the other hand q lim qn 1 2 1 forces the risk to exceed the minimax risk by a factor growing with q to our knowledge this relation between ideas in simultaneous inference and asymptotic decision theory is new our work provides a new perspective on a class of model selection rules which has been introduced recently by several authors these new rules impose complexity penalization of the form 2 log potential model size actual model size we exhibit a close connection with fdr controlling procedures under stringent control of the false discovery rate

eigenvalues invariant factors highest weights and schubert calculus we describe recent work of klyachko totaro knutson and tao that characterizes eigenvalues of sums of hermitian matrices and decomposition of tensor products of representations of gln c we explain related applications to invariant factors of products of matrices intersections in grassmann varieties and singular values of sums and products of arbitrary matrices recent breakthroughs primarily by a klyachko b totaro a knutson and t tao with contributions by p belkale and c woodward have led to complete solutions of several old problems involving the various notions in the title our aim here is to describe this work and especially to show how these solutions are derived from it along the way we will see that these problems are also related to other areas of mathematics including geometric invariant theory symplectic geometry and combinatorics in addition we present some related applications to singular values of arbitrary matrices although many of the theorems we state here have not appeared elsewhere their proofs are mostly soft algebra based on the hard geometric or combinatorial work of others indeed this paper emphasizes concrete elementary arguments we do give some new examples and counterexamples and raise some new open questions we have attempted to point to the sources and to some of the key partial results that had been conjectured or proved before however there is a very large literature particularly for linear algebra problems about eigenvalues singular values and invariant factors we have listed only a few of these articles from whose bibliographies we hope an interested reader can trace the history we apologize to the many whose work is not cited directly we begin in the first five sections by describing each of the problems together with some of their early histories and we state as theorems the new solutions to these problems in section 6 we describe the steps toward these solutions that were carried out before the recent breakthroughs then we discuss the recent solutions and explain how these theorems follow from the work of the above mathematicians sections 7 8 9 and 10 also contain variations and generalizations of some of the theorems stated in the first five sections as well as attributions of the theorems to their authors one of our fascinations with this subject even now that we have proofs of the theorems is the challenge to understand in a deeper way why all these subjects are received by the editors in july 1999 and in revised form january 3 2000 2000 mathematics subject classification primary 15a42 22e46 14m15 secondary 05e15 13f10 14c17 15a18 47b07 the author was partly supported by nsf grant dms9970435 c 2000 american mathematical society

a wavelet or lifting scheme based imputation method summary the paper proposes a new approach to imputation using the expected sparse representation of a surface in a wavelet or lifting scheme basis our method incorporates a bayesian mixture prior for these wavelet coefficients into a gibbs sampler to generate a complete posterior distribution for the variable of interest intuitively the estimator operates by borrowing strength from those observed neighbouring values to impute at the unobserved sites we demonstrate the strong performance of our estimator in both one and two dimensional imputation problems where we also compare its application with the standard imputation techniques of kriging and thin plate splines

frequentist optimality of bayesian wavelet shrinkage rules for gaussian and non gaussian noise the present paper investigates theoretical performance of various bayesian wavelet shrinkage rules in a nonparametric regression model with i i d errors which are not necessarily normally distributed the main purpose is comparison of various bayesian models in terms of their frequentist asymptotic optimality in sobolev and besov spaces we establish a relationship between hyperparameters verify that the majority of bayesian models studied so far achieve theoretical optimality state which bayesian models cannot achieve optimal convergence rate and explain why it happens

reply to comments on resolution in time frequency the authors reply that the first sentence of the conjecture numbered 5 as stated by dr knockaert see ibid vol 48 no 12 p 3585 86 2000 in his comments was proven first by dembo et al 1991 this they discovered in march 1999 furthermore dr m doroslovacki of george washington university pointed out in march 1999 that a time sequence used by donoho et al 1989 was a counter example to conjecture 5 in that paper they correspondingly revised their conjecture to the improved conjecture presented

mean square stability analysis of a normalized least mean fourth algorithm for a markov plant recently it has been shown that the stability of the least mean fourth lmf algorithm depends on the nonstationarity of the plant the present paper investigates the possibility of overcoming this problem by normalization of the weight vector update term a rigorous mean square stability analysis is provided for a recent normalized lmf algorithm which is normalized by a term that is second order in the estimation error and fourth order in the regressor the analysis is done for a markov plant with a stationary white input with even probability density function and a stationary zero mean white noise it is proved that the mean square deviation msd of the algorithm is bounded for all finite values of the input variance noise variance initial msd and mean square plant parameter increment analytical results are supported by simulations

wavelet threshold estimators for data with correlated noise wavelet threshold estimators for data with stationary correlated noise are constructed by applying a level dependent soft threshold to the coefficients in the wavelet transform a variety of threshold choices is proposed including one based on an unbiased estimate of mean squared error the practical performance of the method is demonstrated on examples including data from a neurophysiological context the theoretical properties of the estimators are investigated by comparing them with an ideal but unattainable bench mark that can be considered in the wavelet context as the risk obtained by ideal spatial adaptivity and more generally is obtained by the use of an oracle that provides information that is not actually available in the data it is shown that the level dependent threshold estimator performs well relative to the bench mark risk and that its minimax behaviour cannot be improved on in order of magnitude by any other estimator the wavelet domain structure of both short and long range dependent noise is considered and in both cases it is shown that the estimators have near optimal behaviour simultaneously in a wide range of function classes adapting automatically to the regularity properties of the underlying model the proofs of the main results are obtained by considering a more general multivariate normal decision theoretic problem

prediction via orthogonalized model mixing abstract we introduce an approach and algorithms for model mixing in large prediction problems with correlated predictors we focus on the choice of predictors in linear models and mix over possible subsets of candidate predictors our approach is based on expressing the space of models in terms of an orthogonalization of the design matrix advantages are both statistical and computational statistically orthogonalization often leads to a reduction in the number of competing models by eliminating correlations computationally large model spaces cannot be enumerated recent approaches are based on sampling models with high posterior probability via markov chains based on orthogonalization of the space of candidate predictors we can approximate the posterior probabilities of models by products of predictor specific terms this leads to an importance sampling function for sampling directly from the joint distribution over the model space without resorting to markov chains compared to the latter ortho

empirical bayes selection of wavelet thresholds this paper explores a class of empirical bayes methods for level dependent threshold selection in wavelet shrinkage the prior considered for each wavelet coefficient is a mixture of an atom of probability at zero and a heavy tailed density the mixing weight or sparsity parameter for each level of the transform is chosen by marginal maximum likelihood if estimation is carried out using the posterior median this is a random thresholding procedure the estimation can also be carried out using other thresholding rules with the same threshold details of the calculations needed for implementing the procedure are included in practice the estimates are quick to compute and there is software available simulations on the standard model functions show excellent performance and applications to data drawn from various fields of application are used to explore the practical performance of the approach by using a general result on the risk of the corresponding marginal maximum likelihood approach for a single sequence overall bounds on the risk of the method are found subject to membership of the unknown function in one of a wide range of besov classes covering also the case of f of bounded variation the rates obtained are optimal for any value of the parameter p in 0 simultaneously for a wide range of loss functions each dominating the lq norm of the th derivative with 0 and 0 q 2 attention is paid to the distinction between sampling the unknown function within white noise and sampling at discrete points and between placing constraints on the function itself and on the discrete wavelet transform of its sequence of values at the observation points results for all relevant combinations of these scenarios are obtained in some cases a key feature of the theory is a particular boundary corrected wavelet basis details of which are discussed overall the approach described seems so far unique in combining the properties of fast computation good theoretical properties and good performance in simulations and in practice a key feature appears to be that the estimate of sparsity adapts to three different zones of estimation first where the signal is not sparse enough for thresholding to be of benefit second where an appropriately chosen threshold results in substantially improved estimation and third where the signal is so sparse that the zero estimate gives the optimum accuracy rate

dynamic denoising of tracking sequences in this paper we describe an approach to the problem of simultaneously enhancing image sequences and tracking the objects of interest represented by the latter the enhancement part of the algorithm is based on bayesian wavelet denoising which has been chosen due to its exceptional ability to incorporate diverse a priori information into the process of image recovery in particular we demonstrate that in dynamic settings useful statistical priors can come both from some reasonable assumptions on the properties of the image to be enhanced as well as from the images that have already been observed before the current scene using such priors forms the main contribution of the present paper which is the proposal of the dynamic denoising as a tool for simultaneously enhancing and tracking image sequences within the proposed framework the previous observations of a dynamic scene are employed to enhance its present observation the mechanism that allows the fusion of the information within successive image frames is bayesian estimation while transferring the useful information between the images is governed by a kalman filter that is used for both prediction and estimation of the dynamics of tracked objects therefore in this methodology the processes of target tracking and image enhancement collaborate in an interlacing manner rather than being applied separately the dynamic denoising is demonstrated on several examples of sar imagery the results demonstrated in this paper indicate a number of advantages of the proposed dynamic denoising over static approaches in which the tracking images are enhanced independently of each other

bayesian model selection using test statistics existing bayesian model selection procedures require the specification of prior distributions on the parameters appearing in every model in the selection set in practice this requirement limits the application of bayesian model selection methodology to overcome this limitation we propose a new approach towards bayesian model selection that uses classical test statistics to compute bayes factors between possible models in several test cases our approach produces results that are similar to previously proposed bayesian model selection and model averaging techniques in which prior distributions were carefully chosen in addition to eliminating the requirement to specify complicated prior distributions this method offers important computational and algorithmic advantages over existing simulation based methods because it is easy to evaluate the operating characteristics of this procedure for a given sample size and specified number of covariates our method facilitates the selection of hyperparameter values through prior predictive simulation

needles and straw in haystacks empirical bayes estimates of possibly sparse sequences an empirical bayes approach to the estimation of possibly sparse sequences observed in gaussian white noise is set out and investigated the prior considered is a mixture of an atom of probability at zero and a heavy tailed density y with the mixing weight chosen by marginal maximum likelihood in the hope of adapting between sparse and dense sequences if estimation is then carried out using the posterior median this is a random thresholding procedure other thresholding rules employing the same threshold can also be used probability bounds on the threshold chosen by the marginal maximum likelihood approach lead to overall risk bounds over classes of signal sequences of length n allowing for sparsity of various kinds and degrees the signal classes considered are nearly black sequences where only a proportion is allowed to be nonzero and sequences with normalized p norm bounded by for 0 and 0 1 simulations show excellent performance for appropriately chosen functions y the method is computationally tractable and software is available the extension to a modified thresholding method relevant to the estimation of very sparse sequences is also considered

computing bayes factors by combining simulation and asymptotic approximations abstract the bayes factor is a ratio of two posterior normalizing constants which may be difficult to compute we compare several methods of estimating bayes factors when it is possible to simulate observations from the posterior distributions via markov chain monte carlo or other techniques the methods that we study are all easily applied without consideration of special features of the problem provided that each posterior distribution is well behaved in the sense of having a single dominant mode we consider a simulated version of laplace s method a simulated version of bartlett correction importance sampling and a reciprocal importance sampling technique we also introduce local volume corrections for each of these in addition we apply the bridge sampling method of meng and wong we find that a simulated version of laplace s method with local volume correction furnishes an accurate approximation that is especially useful when likelihood function evaluations are costly a simple bridge sampli

general empirical bayes wavelet methods and exactly adaptive minimax estimation in many statistical problems stochastic signals can be represented as a sequence of noisy wavelet coefficients in this paper we develop general empirical bayes methods for the estimation of true signal our estimators approximate certain oracle separable rules and achieve adaptation to ideal risks and exact minimax risks in broad collections of classes of signals in particular our estimators are uniformly adaptive to the minimum risk of separable estimators and the exact minimax risks simultaneously in besov balls of all smoothness and shape indices and they are uniformly superefficient in convergence rates in all compact sets in besov spaces with a finite secondary shape parameter furthermore in classes nested between besov balls of the same smoothness index our estimators dominate threshold and james stein estimators within an infinitesimal fraction of the minimax risks more general block empirical bayes estimators are developed both white noise with drift and nonparametric regression are considered

seeking efficient data augmentation schemes via conditional and marginal augmentation data augmentation sometimes known as the method of auxiliary variables is a powerful tool for constructing optimisation and simulation algorithms in the context of optimisation meng van dyk 1997 1998 reported several successes of the working parameter approach for constructing efficient data augmentation schemes for fast and simple em type algorithms this paper investigates the use of working parameters in the context of markov chain monte carlo in particular in the context of tanner wong s 1987 data augmentation algorithm via a theoretical study of two working parameter approaches the conditional augmentation approach and the marginal augmentation approach posterior sampling under the univariate t model is used as a running example which particularly illustrates how the marginal augmentation approach obtains a fast mixing positive recurrent markov chain by first constructing a nonpositive recurrent markov chain in a larger space

sampling based approaches to calculating marginal densities abstract stochastic substitution the gibbs sampler and the sampling importance resampling algorithm can be viewed as three alternative sampling or monte carlo based approaches to the calculation of numerical estimates of marginal probability distributions the three approaches will be reviewed compared and contrasted in relation to various joint probability structures frequently encountered in applications in particular the relevance of the approaches to calculating bayesian posterior densities for a variety of structured models will be discussed and illustrated

on bayesian testimation and its application to wavelet thresholding summary we consider the problem of estimating the unknown response function in the gaussian white noise model we first utilize the recently developed bayesian maximum a posteriori testimation procedure of abramovich et al 2007 for recovering an unknown high dimensional gaussian mean vector the existing results for its upper error bounds over various sparse lp balls are extended to more general cases we show that for a properly chosen prior on the number of nonzero entries of the mean vector the corresponding adaptive estimator is asymptotically minimax in a wide range of sparse and dense lp balls the proposed procedure is then applied in a wavelet context to derive adaptive global and level wise wavelet estimators of the unknown response function in the gaussian white noise model these estimators are then proven to be respectively asymptotically near minimax and minimax in a wide range of besov balls these results are also extended to the estimation of derivatives of the response function simulated examples are conducted to illustrate the performance of the proposed level wise wavelet estimator in finite sample situations and to compare it with several existing counterparts

image denoising using scale mixtures of gaussians in the wavelet domain we describe a method for removing noise from digital images based on a statistical model of the coefficients of an overcomplete multiscale oriented basis neighborhoods of coefficients at adjacent positions and scales are modeled as the product of two independent random variables a gaussian vector and a hidden positive scalar multiplier the latter modulates the local variance of the coefficients in the neighborhood and is thus able to account for the empirically observed correlation between the coefficient amplitudes under this model the bayesian least squares estimate of each coefficient reduces to a weighted average of the local linear estimates over all possible values of the hidden multiplier variable we demonstrate through simulations with images contaminated by additive white gaussian noise that the performance of this method substantially surpasses that of previously published methods both visually and in terms of mean squared error

wavelet shrinkage using cross validation summary wavelets are orthonormal basis functions with special properties that show potential in many areas of mathematics and statistics this paper concentrates on the estimation of functions and images from noisy data by using wavelet shrinkage a modified form of twofold cross validation is introduced to choose a threshold for wavelet shrinkage estimators operating on data sets of length a power of 2 the cross validation algorithm is then extended to data sets of any length and to multidimensional data sets the algorithms are compared with established threshold choosers by using simulation an application to a real data set arising from anaesthesia is presented

bayesian model averaging for linear regression models abstract we consider the problem of accounting for model uncertainty in linear regression models conditioning on a single selected model ignores model uncertainty and thus leads to the underestimation of uncertainty when making inferences about quantities of interest a bayesian solution to this problem involves averaging over all possible models i e combinations of predictors when making inferences about quantities of interest this approach is often not practical in this article we offer two alternative approaches first we describe an ad hoc procedure occam s window which indicates a small set of models over which a model average can be computed second we describe a markov chain monte carlo approach that directly approximates the exact solution in the presence of model uncertainty both of these model averaging procedures provide better predictive performance than any single model that might reasonably have been selected in the extreme case where there are many candidate predictors but

flexible parsimonious smoothing and additive modeling a simple method is presented for fitting regression models that are nonlinear in the explanatory variables despite its simplicity or perhaps because of it the method has some powerful characteristics that cause it to be competitive with and often superior to more sophisticated techniques especially for small data sets in the presence of high noise

scalable bayesian model averaging through local information propagation this article shows that a probabilistic version of the classical forward stepwise variable inclusion procedure can serve as a general data augmentation scheme for model space distributions in generalized linear models this latent variable representation takes the form of a markov process thereby allowing information propagation algorithms to be applied for sampling from model space posteriors in particular we propose a sequential monte carlo method for achieving effective unbiased bayesian model averaging in high dimensional problems using proposal distributions constructed using local information propagation the method called lips for local information propagation based sampling is illustrated using real and simulated examples with dimensionality ranging from 15 to 1000 and its performance in estimating posterior inclusion probabilities and in out of sample prediction is compared to those of several other methods namely mcmc bas ibma and lasso in addition it is shown that the latent variab

stabilization of high order stochastic gradient adaptive filtering algorithms the paper is concerned with stabilizing the family of adaptive filtering algorithms based on minimizing the 2lth moment of the estimation error with l being an integer greater than 1 stabilization is attained via a proposed normalization of the algorithm mean square stability of the normalized algorithm is proved for a markov plant for all l 1 transient and steady state performances of the algorithm are analyzed for a time invariant plant expressions are derived for the steady state misadjustment and convergence time tradeoff between the transient and steady state performances is evaluated dependence of this tradeoff on the value of l is studied this tradeoff is compared with the tradeoff of the nlms algorithm the proposed algorithm is dramatically superior to the nlms algorithm for sub gaussian noise the superiority increases with l initial mean square deviation and signal to noise ratio analytical results are supported by simulations

quasi linear wavelet estimation abstract the main paradigm of the modern wavelet theory of spatial adaptation formulated by donoho and johnstone is that there is a divergence between the linear minimax adaptation theory and the heuristic guiding algorithm development that leads to the necessity of using strongly nonlinear adaptive thresholded methods on the other hand it is well known that linear adaptive estimates are the best whenever an estimated function is smooth is it possible to suggest a quasi linear wavelet estimate by adding to a linear adaptive estimate a minimal number of nonlinear terms on finest scales that offers advantages of linear adaptive estimates and at the same time matches asymptotic properties of strongly nonlinear procedures like the benchmark sureshrink the answer is yes and we discuss quasi linear estimation both theoretically and via a monte carlo study in particular i show that asymptotically a quasi linear procedure not only matches properties of sureshrink over the besov scale but also allows

on comparing restricted least squares estimators abstract this paper examines the conditions under which it is possible to make comparisons between restricted least squares estimators under a generalized mean square error criterion our results demonstrate the rather extreme limitations on such comparisons in many cases it is not possible to compare two restricted least squares estimators using the generalized mean square error criterion

posterior odds ratios for regression hypotheses general considerations and some specific results abstract much work is econometrics and statistics has been concerned with comparing bayesian and non bayesian estimation results while much less has involved comparisons of bayesian and non bayesian analyses of hypotheses some issues arising in this latter area that are mentioned and discussed in the paper are 1 is it meaningful to associate probabilities with hypotheses 2 what concept of probability is to be employed in analyzing hypotheses 3 is a separate theory of testing needed 4 must a theory of testing be capable of treating both sharp and non sharp hypotheses 5 how is prior information incorporated in testing 6 does the use of power functions in practice necessitate the use of prior information 7 how are significance levels determined when sample sizes are large and what are the interpretations of p values and tail areas 8 how are conflicting results provided by asymptotically equivalent testing procedures to be reconciled 9 what is the rationale for the 5 accept reject syndrome that afflicts econometrics and applied statistics 10 does it make sense to test a null hypothesis with no alternative hypothesis present and 11 how are the results of analyses of hypotheses to be combined with estimation and prediction procedures brief discussions of these issues with references to the literature are provided since there is much controversy concerning how hypotheses are actually analyzed in applied work the results of a small survey relating to 22 articles employing empirical data published in leading economic and econometric journals in 1978 are presented the major results of this survey indicate that there is wide spread use of the 1 and 5 levels of significance in non bayesian testing with no systematic relation between choice of significance level and sample size also power considerations are not generally discussed in empirical studies in fact there was a discussion of power in only one of the articles surveyed further there was very little formal or informal use of prior information employed in testing hypotheses and practically no attention was given to the effects of tests or pre tests on the properties of subsequent tests or estimation results these results indicate that there is much room for improvement in applied analyses of hypotheses given the findings of the survey of applied studies it is suggested that bayesian procedures for analyzing hypotheses may be helpful in improving applied analyses in this connection the paper presents a review of some bayesian procedures and results for analyzing sharp and non sharp hypotheses with explicit use of prior information in general bayesian procedures have good sampling properties and enable investigators to compute posterior probabilities and posterior odds ratios associated with alternative hypotheses quite readily the relationships of several posterior odds ratios to usual non bayesian testing procedures is clearly demonstrated also a relation between the p value or tail area and a posterior odds ratio is described in detail in the important case of hypotheses about a mean of a normal distribution other examples covered in the paper include posterior odds ratios for the hypotheses that 1 i and i where i is a regression coefficient 2 data are drawn from either of two alternative distributions 3 0 and where is the mean of a normal distribution 4 0 and 0 where is a vector of regression coefficients 5 2 0 vs 2 0 where 1 2 is a vector regression coefficients and 1 s value is unrestricted in several cases is a vector of regression coefficients and 1 s value is unrestricted in several cases tabulations of odds ratios are provided bayesian versions of the chow test for equality of regression coefficients and of the goldfeld quandt test for equality of disturbance variances are given also an application of bayesian posterior odds ratios to a regression model selection problem utilizing the hald data is reported in summary the results reported in the paper indicate that operational bayesian procedures for analyzing many hypotheses encountered in model selection problems are available these procedures yield posterior odds ratios and posterior probabilities for competing hypotheses these posterior odds ratios represent the weight of the evidence supporting one model or hypothesis relative to another given a loss structure as is well known one can choose among hypotheses so as to minimize expected loss also with posterior probabilities available and an estimation or prediction loss function it is possible to choose a point estimate or prediction that minimizes expected loss by averaging over alternative hypotheses or models thus it is seen that the bayesian approach for analyzing competing models or hypotheses provides a unified framework that is extremely useful in solving a number of model selection problems

bayesian wavelet based image denoising using the gauss hermite expansion the probability density functions pdfs of the wavelet coefficients play a key role in many wavelet based image processing algorithms such as denoising the conventional pdfs usually have a limited number of parameters that are calculated from the first few moments only consequently such pdfs cannot be made to fit very well with the empirical pdf of the wavelet coefficients of an image as a result the shrinkage function utilizing any of these density functions provides a substandard denoising performance in order for the probabilistic model of the image wavelet coefficients to be able to incorporate an appropriate number of parameters that are dependent on the higher order moments a pdf using a series expansion in terms of the hermite polynomials that are orthogonal with respect to the standard gaussian weight function is introduced a modification in the series function is introduced so that only a finite number of terms can be used to model the image wavelet coefficients ensuring at the same time the resulting pdf to be non negative it is shown that the proposed pdf matches the empirical one better than some of the standard ones such as the generalized gaussian or bessel k form pdf a bayesian image denoising technique is then proposed wherein the new pdf is exploited to statistically model the subband as well as the local neighboring image wavelet coefficients experimental results on several test images demonstrate that the proposed denoising method both in the subband adaptive and locally adaptive conditions provides a performance better than that of most of the methods that use pdfs with limited number of parameters

testing a point null hypothesis the irreconcilability of p values and evidence abstract the problem of testing a point null hypothesis or a small interval null hypothesis is considered of interest is the relationship between the p value or observed significance level and conditional and bayesian measures of evidence against the null hypothesis although one might presume that a small p value indicates the presence of strong evidence against the null such is not necessarily the case expanding on earlier work especially edwards lindman and savage 1963 and dickey 1977 it is shown that actual evidence against a null as measured say by posterior probability or comparative likelihood can differ by an order of magnitude from the p value for instance data that yield a p value of 05 when testing a normal mean result in a posterior probability of the null of at least 30 for any objective prior distribution objective here means that equal prior weight is given the two hypotheses and that the prior is symmetric and nonincreasing away from the null other definiti

global stabilization of the least mean fourth algorithm the least mean fourth algorithm has several stability problems its stability depends on the variance and distribution type of the adaptive filter input the noise variance and the initialization of the filter weights the present correspondence provides a global solution to all these stability problems this is achieved by normalizing the weight vector update term by a term that is fourth order in the regressor and second order in the estimation error the former property stabilizes the algorithm against the variance and distribution type of the filter input while the latter stabilizes the algorithm against the noise variance and the weight initialization the obtained algorithm is stable for all values of the step size between 0 and 2 the stability of the algorithm is supported by simulations

bayesian analysis of binary and polychotomous response data abstract a vast literature in statistics biometrics and econometrics is concerned with the analysis of binary and polychotomous response data the classical approach fits a categorical response regression model using maximum likelihood and inferences about the model are based on the associated asymptotic theory the accuracy of classical confidence statements is questionable for small sample sizes in this article exact bayesian methods for modeling categorical response data are developed using the idea of data augmentation the general approach can be summarized as follows the probit regression model for binary outcomes is seen to have an underlying normal regression structure on latent continuous data values of the latent data can be simulated from suitable truncated normal distributions if the latent data are known then the posterior distribution of the parameters can be computed using standard results for normal linear models draws from this posterior are used to sample new latent data and t

calibration and empirical bayes variable selection for the problem of variable selection for the normal linear model selection criteria such as aic c p bic and ric have fixed dimensionality penalties such criteria are shown to correspond to selection of maximum posterior models under implicit hyperparameter choices for a particular hierarchical bayes formulation based on this calibration we propose empirical bayes selection criteria that use hyperparameter estimates instead of fixed choices for obtaining these estimates both marginal and conditional maximum likelihood methods are considered as opposed to traditional fixed penalty criteria these empirical bayes criteria have dimensionality penalties that depend on the data their performance is seen to approximate adaptively the performance of the best fixed penalty criterion across a variety of orthogonal and nonorthogonal set ups including wavelet regression empirical bayes shrinkage estimators of the selected coefficients are also proposed

parameter expansion for data augmentation abstract viewing the observed data of a statistical model as incomplete and augmenting its missing parts are useful for clarifying concepts and central to the invention of two well known statistical algorithms expectation maximization em and data augmentation recently liu rubin and wu demonstrated that expanding the parameter space along with augmenting the missing data is useful for accelerating iterative computation in an em algorithm the main purpose of this article is to rigorously define a parameter expanded data augmentation px da algorithm and to study its theoretical properties the px da is a special way of using auxiliary variables to accelerate gibbs sampling algorithms and is closely related to reparameterization techniques we obtain theoretical results concerning the convergence rate of the px da algorithm and the choice of prior for the expansion parameter to understand the role of the expansion parameter we establish a new theory for iterative conditional sampling under the tra

mixtures of g priors for bayesian variable selection zellner s g prior remains a popular conventional prior for use in bayesian variable selection despite several undesirable consistency issues in this article we study mixtures of g priors as an alternative to default g priors that resolve many of the problems with the original formulation while maintaining the computational tractability that has made the g prior so popular we present theoretical properties of the mixture g priors and provide real and simulated examples to compare the mixture formulation with fixed g priors empirical bayes approaches and other default procedures please see arnold zellner s letter and the author s response

benchmark priors for bayesian model averaging in contrast to a posterior analysis given a particular sampling model posterior model probabilities in the context of model uncertainty are typically rather sensitive to the speciflcation of the prior in particular difiuse priors on model speciflc parameters can lead to quite unexpected consequences here we focus on the practically relevant situation where we need to entertain a large number of sampling models and we have or wish to use little or no subjective prior information we aim at providing an automatic or benchmark prior structure that can be used in such cases we focus on the normal linear regression model with uncertainty in the choice of re gressors we propose a partly noninformative prior structure related to a natural conjugate g prior speciflcation where the amount of subjective information requested from the user is limited to the choice of a single scalar hyperparameter g0j the consequences of difierent choices for g0j are examined we investigate theoretical properties such as consistency of the implied bayesian procedure links with classical information criteria are provided in addition we examine the flnite sample implications of several choices of g0j in a simulation study the use of the mc 3 algorithm of madigan and york 1995 combined with e cient coding in fortran makes it feasible to conduct large simulations in addition to posterior criteria we shall also compare the predictive performance of difierent priors a classic exam ple concerning the economics of crime will also be provided and contrasted with results in the literature the main flndings of the paper will lead us to propose a benchmark prior speciflcation in a linear regression context with model uncertainty

the collapsed gibbs sampler in bayesian computations with applications to a gene regulation problem abstract this article describes a method of grouping and collapsing in using the gibbs sampler and proves from an operator theory viewpoint that the method is in general beneficial the norms of the forward operators associated with the corresponding nonreversible markov chains are used to discriminate among different simulation schemes when applied to bayesian missing data problems the idea of collapsing suggests skipping the steps of sampling parameter s values in standard data augmentation by doing this we obtain a predictive update version of the gibbs sampler a procedure of calculating the posterior odds ratio via the collapsed gibbs sampler when incomplete observations are involved is presented as an illustration of possible applications three examples along with a bayesian treatment for identifying common protein binding sites in unaligned dna sequences are provided

spike and slab variable selection frequentist and bayesian strategies variable selection in the linear regression model takes many apparent faces from both frequentist and bayesian standpoints in this paper we introduce a variable selection method referred to as a rescaled spike and slab model we study the importance of prior hierarchical specifications and draw connections to frequentist generalized ridge regression estimation specifically we study the usefulness of continuous bimodal priors to model hypervariance parameters and the effect scaling has on the posterior mean through its relationship to penalization several model selection strategies some frequentist and some bayesian in nature are developed and studied theoretically we demonstrate the importance of selective shrinkage for effective variable selection in terms of risk misclassification and show this is achieved using the posterior from a rescaled spike and slab model we also show how to verify a procedure s ability to reduce model uncertainty in finite samples using a specialized forward selection strategy using this tool we illustrate the effectiveness of rescaled spike and slab models in reducing model uncertainty

adaptive sampling for bayesian variable selection our paper proposes adaptive monte carlo sampling schemes for bayesian variable selection in linear regression that improve on standard markov chain methods we do so by considering metropolis hastings proposals that make use of accumulated information about the posterior distribution obtained during sampling adaptation needs to be done carefully to ensure that sampling is from the correct ergodic distribution we give conditions for the validity of an adaptive sampling scheme in this problem and for simulating from a distribution on a finite state space in general and suggest a class of adaptive proposal densities which uses best linear prediction to approximate the gibbs sampler our sampling scheme is computationally much faster per iteration than the gibbs sampler and when this is taken into account the efficiency gains when using our sampling scheme compared to alternative approaches are substantial in terms of precision of estimation of posterior quantities of interest for a given amount of computation time we compare our method with other sampling schemes for examples involving both real and simulated data the methodology developed in the paper can be extended to variable selection in more general problems

a theoretical comparison of the data augmentation marginal augmentation and px da algorithms the data augmentation da algorithm is a widely used markov chain monte carlo mcmc algorithm that is based on a markov transition density of the form p x x fy f x y x y f y x y x dy where f x y and f y x are conditional densities the px da and marginal augmentation algorithms of liu and wu j amer statist assoc 94 1999 1264 1274 and meng and van dyk biometrika 86 1999 301 320 are alternatives to da that often converge much faster and are only slightly more computationally demanding the transition densities of these alternative algorithms can be written in the form p r x x f y y f x y x y r y dy f y x y x dy where r is a markov transition function on y we prove that when r satisfies certain conditions the mcmc algorithm driven by pr is at least as good as that driven by p in terms of performance in the central limit theorem and in the operator norm sense these results are brought to bear on a theoretical comparison of the da px da and marginal augmentation algorithms our focus is on situations where the group structure exploited by liu and wu is available we show that the px da algorithm based on haar measure is at least as good as any px da algorithm constructed using a proper prior on the group

adapting to unknown smoothness via wavelet shrinkage abstract we attempt to recover a function of unknown smoothness from noisy sampled data we introduce a procedure sureshrink that suppresses noise by thresholding the empirical wavelet coefficients the thresholding is adaptive a threshold level is assigned to each dyadic resolution level by the principle of minimizing the stein unbiased estimate of risk sure for threshold estimates the computational effort of the overall procedure is order n log n as a function of the sample size n sureshrink is smoothness adaptive if the unknown function contains jumps then the reconstruction essentially does also if the unknown function has a smooth piece then the reconstruction is essentially as smooth as the mother wavelet will allow the procedure is in a sense optimally smoothness adaptive it is near minimax simultaneously over a whole interval of the besov scale the size of this interval depends on the choice of mother wavelet we know from a previous paper by the authors that traditional smoot

alternative tests of rational expectations models the case of the term structure a linearized version of the rational expectations models of the term structure is put forth in terms of a complete vector of equally spaced observations along the yield curve a data series on intermediate maturity yields which meets the specifications of the model is presented the model is tested against a specific and easily interpreted alternative earlier studies of rational expectations models which used volatility tests or likelihood ratio tests are discussed this abstract was borrowed from another version of this item

a bayesian approach to testing the arbitrage pricing theory abstract we consider new methods of testing the restrictions imposed on asset returns implied by an equilibrium version of the arbitrage pricing theory in the competitive equilibrium version of the apt see for example connor and korajczyk 1988 excess returns on a collection of portfolios or individual assets are related to measured factors in a multivariate regression model without intercepts the traditional significance testing approach uses estimates of these intercept terms and a likelihood ratio procedure for statistical testing in our approach we quantify differences between the restricted and unrestricted models by computing posterior model probabilities these posterior model probabilities are combined to form posterior odds ratios which represent the weight of prior and sample evidence for or against the pricing restrictions a dramatic simplification of the odds ratio calculation is achieved by using the savage density ratio method the special structure of the apt model with factors extracted by principal components is exploited to produce a simple prior the sensitivity of the odds ratio to different prior specifications is explored we apply these new methodologies to tests of the apt with a collection of ten size based portfolios the factors are extracted by a principal components technique developed by connor and korajczyk using all listed amex nyse firms over the period 1964 1983 the odds ratios show that the sample evidence is weak and does not favor the apt pricing restrictions we are able to gauge the strength of the evidence in the data by examining the sensitivity of the odds ratio to the prior

stochastic analysis of a stable normalized least mean fourth algorithm for adaptive noise canceling with a white gaussian reference the least mean fourth lmf algorithm has several stability problems its stability depends on the variance and distribution type of the adaptive filter input the noise variance and the initialization of the filter weights a global solution to these stability problems was presented recently for a normalized lmf nlmf algorithm here a stochastic analysis of the mean square deviation msd of the globally stable nlmf algorithm is provided the analysis is done in the context of adaptive noise canceling with a white gaussian reference input and gaussian binary and uniform desired signals the analytical model is shown to accurately predict the results of monte carlo simulations comparisons of the nlmf and nlms algorithms are then made for various parameter selections it is then shown under what conditions the nlmf algorithm is superior to nlms algorithm for adaptive noise canceling

the least mean fourth lmf adaptive algorithm and its family new steepest descent algorithms for adaptive filtering and have been devised which allow error minimization in the mean fourth and mean sixth etc sense during adaptation the weights undergo exponential relaxation toward their optimal solutions time constants have been derived and surprisingly they turn out to be proportional to the time constants that would have been obtained if the steepest descent least mean square lms algorithm of widrow and hoff had been used the new gradient algorithms are insignificantly more complicated to program and to compute than the lms algorithm their general form is w j 1 w j 2 mu k epsilon j 2k 1 x j where w j is the present weight vector w j 1 is the next weight vector epsilon j is the present error x j is the present input vector mu is a constant controlling stability and rate of convergence and 2k is the exponent of the error being minimized conditions have been derived for weight vector convergence of the mean and of the variance for the new gradient algorithms the behavior of the least mean fourth lmf algorithm is of special interest in comparing this algorithm to the lms algorithm when both are set to have exactly the same time constants for the weight relaxation process the lmf algorithm under some circumstances will have a substantially lower weight noise than the lms algorithm it is possible therefore that a minimum mean fourth error algorithm can do a better job of least squares estimation than a mean square error algorithm this intriguing concept has implications for all forms of adaptive algorithms whether they are based on steepest descent or otherwise

multiple shrinkage and subset selection in wavelets this paper discusses bayesian methods for multiple shrinkage estimation in wavelets wavelets are used in applications for data denoising via shrinkage of the coefficients towards zero and for data compression by shrinkage and setting small coefficients to zero we approach wavelet shrinkage by using bayesian hierarchical models assigning a positive prior probability to the wavelet coefficients being zero the resulting estimator for the wavelet coefficients is a multiple shrinkage estimator that exhibits a wide variety of nonlinear patterns we discuss fast computational implementations with a focus on easy to compute analytic approximations as well as importance sampling and markov chain monte carlo methods multiple shrinkage estimators prove to have excellent mean squared error performance in reconstructing standard test functions we demonstrate this in simulated test examples comparing various implementations of multiple shrinkage to commonly used shrinkage rules finally we illustrate our approach with an application to the so called glint data

a mean square stability analysis of the least mean fourth adaptive algorithm this paper presents a new convergence analysis of the least mean fourth lmf adaptive algorithm in the mean square sense the analysis improves previous results in that it is valid for non gaussian noise distributions and explicitly shows the dependence of algorithm stability on the initial conditions of the weights analytical expressions are derived presenting the relationship between the step size the initial weight error vector and mean square stability the analysis assumes a white zero mean gaussian reference signal and an independent identically distributed i i d measurement noise with any even probability density function pdf it has been shown by nascimento and bermudez probability of divergence for the least mean fourth lmf algorithm ieee transactions on signal processing vol 54 no 4 pp 1376 1385 apr 2006 that the lmf algorithm is not mean square stable for reference signals whose pdfs have infinite support however the probability of divergence as a function of the step size value tends to rise abruptly only when it moves past a given threshold our analysis provides a simple and yet precise estimate of the region of quick rise in the probability of divergence hence the present analysis is useful for predicting algorithm instability in most practical applications

the control of the false discovery rate in multiple testing under dependency benjamini and hochberg suggest that the false discovery rate may be the appropriate error rate to control in many applied multiple testing problems a simple procedure was given there as an fdr controlling procedure for independent test statistics and was shown to be much more powerful than comparable procedures which control the traditional familywise error rate we prove that this same procedure also controls the false discovery rate when the test statistics have positive regression dependency on each of the test statistics corresponding to the true null hypotheses this condition for positive dependency is general enough to cover many problems of practical interest including the comparisons of many treatments with a single control multivariate normal test statistics with positive correlation matrix and multivariate t furthermore the test statistics may be discrete and the tested hypotheses composite without posing special difficulties for all other forms of dependency a simple conservative modification of the procedure controls the false discovery rate thus the range of problems for which a procedure with proven fdr control can be offered is greatly increased 1 1 simultaneous hypotheses testing the control of the increased type i error when testing simultaneously a family of hypotheses is a central issue in the area of multiple comparisons rarely are we interested only in whether all hypotheses are jointly true or not which is the test of the intersection null hypothesis in most applications we infer about the individual hypotheses realizing that some of the tested hypotheses are usually true we hope not all and some are not we wish to decide which ones are not true indicating statistical discoveries an important such problem is that of multiple endpoints in a clinical trial a new treatment is compared with an existing one in terms of a large number of potential benefits endpoints

rao blackwellization for bayesian variable selection and model averaging in linear and binary regression a novel data augmentation approach choosing the subset of covariates to use in regression or generalized linear models is a ubiquitous problem the bayesian paradigm addresses the problem of model uncertainty by considering models corresponding to all possible subsets of the covariates where the posterior distribution over models is used to select models or combine them via bayesian model averaging bma although conceptually straightforward bma is often difficult to implement in practice since either the number of covariates is too large for enumeration of all subsets calculations cannot be done analytically or both for orthogonal designs with the appropriate choice of prior the posterior probability of any model can be calculated without having to enumerate the entire model space and scales linearly with the number of predictors p in this article we extend this idea to a much broader class of nonorthogonal design matrices we propose a novel method which augments the observed nonorthogonal design by at most p new rows to obtain

adaptive weights smoothing with applications to image restoration we propose a new method of nonparametric estimation which is based on locally constant smoothing with an adaptive choice of weights for every pair of data points some theoretical properties of the procedure are investigated then we demonstrate the performance of the method on some simulated univariate and bivariate examples and compare it with other nonparametric methods finally we discuss applications of this procedure to magnetic resonance and satellite imaging

lower bounds on bayes factors for interval null hypotheses abstract expressions are derived for lower bounds on bayes factors for interval null hypotheses these are then studied for varying interval lengths and are related to two interesting results one for point null hypotheses and another for one sided hypotheses these results are illustrated with two of the most frequently used hypothesis tests the normal test and student s t test

control functionals for monte carlo integration summary a non parametric extension of control variates is presented these leverage gradient information on the sampling density to achieve substantial variance reduction it is not required that the sampling density be normalized the novel contribution of this work is based on two important insights a trade off between random sampling and deterministic approximation and a new gradient based function space derived from stein s identity unlike classical control variates our estimators improve rates of convergence often requiring orders of magnitude fewer simulations to achieve a fixed level of precision theoretical and empirical results are presented the latter focusing on integration problems arising in hierarchical models and models based on non linear ordinary differential equations

automatic bayesian curve fitting summary a method of estimating a variety of curves by a sequence of piecewise polynomials is proposed motivated by a bayesian model and an appropriate summary of the resulting posterior distribution a joint distribution is set up over both the number and the position of the knots defining the piecewise polynomials throughout we use reversible jump markov chain monte carlo methods to compute the posteriors the methodology has been successful in giving good estimates for smooth functions i e continuous and differentiable as well as functions which are not differentiable and perhaps not even continuous at a finite number of points the methodology is extended to deal with generalized additive models

dependence of the stability of the least mean fourth algorithm on target weights non stationarity the paper investigates a new stability problem of the least mean fourth lmf algorithm which is the dependence of the algorithm stability on the time variation of the target weights of the adaptive filter the analysis is done in the context of tracking a markov plant with a stationary white gaussian input it is found that the algorithm diverges if the mean square increment of the plant parameter vector exceeds a threshold value that depends on the step size input variance and noise moments the paper also derives a closed form of the steady state mean square deviation without the usual assumption of a strong noise comparison of the tracking capabilities of the lmf and lms algorithms is provided the comparison is done in terms of the minimum mean square deviation attained by each algorithm over the stability range of its step size gaussian uniform and binary distributions of the noise are considered conditions that make one algorithm outperform the other are determined analytical results are supported by simulations

probability of divergence for the least mean fourth algorithm in this paper it is shown that the least mean fourth lmf adaptive algorithm is not mean square stable when the regressor input is not strictly bounded as happens for example if the input has a gaussian distribution for input distributions with infinite support even for the gaussian distribution the lmf always has a nonzero probability of divergence no matter how small the step size is chosen this result is proven for a slight modification of the gaussian distribution in a one tap filter and corroborated with several simulations in addition an upper bound is given for the probability of divergence of lmf as a function of the filter length input power step size and noise variance for the case of gaussian regressors the results reported in this paper provide tools for designers to better understand the behavior of the lmf algorithm and decide on the convenience or not of its use for a given application

flexible empirical bayes estimation for wavelets summary wavelet shrinkage estimation is an increasingly popular method for signal denoising and compression although bayes estimators can provide excellent mean squared error mse properties the selection of an effective prior is a difficult task to address this problem we propose empirical bayes eb prior selection methods for various error distributions including the normal and the heavier tailed student t distributions under such eb prior distributions we obtain threshold shrinkage estimators based on model selection and multiple shrinkage estimators based on model averaging these eb estimators are seen to be computationally competitive with standard classical thresholding methods and to be robust to outliers in both the data and wavelet domains simulated and real examples are used to illustrate the flexibility and improved mse performance of these methods in a wide variety of settings

the optimal transform for the discrete hirschman uncertainty principle we determine all signals giving equality for the discrete hirschman uncertainty principle we single out the case where the entropies of the time signal and its fourier transform are equal these signals up to scalar multiples form an orthonormal basis giving an orthogonal transform that optimally packs a finite duration discrete time signal the transform may be computed via a fast algorithm due to its relationship to the discrete fourier transform

building robust wavelet estimators for multicomponent images using stein s principle multichannel imaging systems provide several observations of the same scene which are often corrupted by noise in this paper we are interested in multispectral image denoising in the wavelet domain we adopt a multivariate statistical approach in order to exploit the correlations existing between the different spectral components our main contribution is the application of stein s principle to build a new estimator for arbitrary multichannel images embedded in additive gaussian noise simulation tests carried out on optical satellite images show that the proposed method outperforms conventional wavelet shrinkage techniques

resolution in time frequency we introduce a new measure h sub p that is related to the heisenberg uncertainty principle the measure predicts the compactness of discrete time signal descriptions in the sample frequency phase plane we conjecture a lower limit on the compaction in the phase plane and show that discretized gaussians may not provide the most compact basis

variable selection via gibbs sampling abstract a crucial problem in building a multiple regression model is the selection of predictors to include the main thrust of this article is to propose and develop a procedure that uses probabilistic considerations for selecting promising subsets this procedure entails embedding the regression setup in a hierarchical normal mixture model where latent variables are used to identify subset choices in this framework the promising subsets of predictors can be identified as those with higher posterior probability the computational burden is then alleviated by using the gibbs sampler to indirectly sample from this multinomial posterior distribution on the set of possible subset choices those subsets with higher probability the promising ones can then be identified by their more frequent appearance in the gibbs sample

covariance structure of the gibbs sampler with applications to the comparisons of estimators and augmentation schemes summary we study the covariance structure of a markov chain generated by the gibbs sampler with emphasis on data augmentation when applied to a bayesian missing data problem the gibbs sampler produces two natural approximations for the posterior distribution of the parameter vector the empirical distribution based on the sampled values of the parameter vector and a mixture of complete data posteriors we prove that rao blackwellization causes a one lag delay for the autocovariances among dependent samples obtained from data augmentation and consequently the mixture approximation produces estimates with smaller variances than the empirical approximation the covariance structure results are used to compare different augmentation schemes it is shown that collapsing and grouping random components in a gibbs sampler with two or three components usually result in more efficient sampling schemes

bayes factors for nonlinear hypotheses and likelihood distributions summary new methods are proposed which allow bayes factors to be computed for hypotheses which impose nonlinear restrictions on the parameters projection methods are used to induce the prior distribution over the restricted parameter space which is required for computation of the bayes factor various distance metrics are introduced to define the projection including a utility based metric which gives kullback leibler divergence as a special case draws from the restricted and unrestricted prior distributions are used to construct marginal distributions of the likelihood which is shown to have additional diagnostic value over and above the bayes factor these methods are applied to hypotheses statisticians are frequently asked to entertain hypotheses which are nonlinear in the model parameters nonlinear hypotheses arise both from subject matter theory as well as from the process of model selection to address nonlinear hypotheses researchers use frequentist methods which are usually implemented using asymptotic approximations berger 1987 zellner 1971 1978 1984 and others have discussed the weaknesses of the classical significance testing approach and suggest bayesian alternatives such as odds ratios the odds ratio is the ratio of posterior probabilities of two competing hypotheses and is the product of the prior odds ratio and the bayes factor the odds ratio has the conceptual advantages of providing a natural probability metric for assessing the strength of data and prior beliefs because of the difficulties of assessing priors under the nonlinear restrictions and the computation of the required integrals few bayes factors have been computed for nonlinear hypotheses see lindley 1988 for an exception in the present paper we outline a general method for computing bayes factors which is flexible conceptually appealing and computationally tractable the literature on bayes factors starts with the work of jeffreys 1961 other references include berger delampady 1987 dickey 1971 good 1985 kass vaidyanathan 1992 smith spiegelhalter 1980 and zellner 1971 1978 1984 these papers deal with the testing of linear restrictions on the parameter space often in the context of linear models even the widely used schwarz 1978 model selection criterion considers only linear restrictions in models of the exponential family form the key difficulty which has kept researchers from developing bayes factors for nonlinear hypotheses is the

feature extraction across individual time series observations with spikes using wavelet principal component analysis clinical studies frequently include repeated measurements of individuals often for long periods we present a methodology for extracting common temporal features across a set of individual time series observations in particular the methodology explores extreme observations within the time series such as spikes as a possible common temporal phenomenon wavelet basis functions are attractive in this sense as they are localized in both time and frequency domains simultaneously allowing for localized feature extraction from a time varying signal we apply wavelet basis function decomposition of individual time series with corresponding wavelet shrinkage to remove noise we then extract common temporal features using linear principal component analysis on the wavelet coefficients before inverse transformation back to the time domain for clinical interpretation we demonstrate the methodology on a subset of a large fetal activity study aiming to identify temporal patterns in fetal movement fm count data in order to explore formal fm counting as a screening tool for identifying fetal compromise and thus preventing adverse birth outcomes copyright 2013 john wiley sons ltd

the horseshoe estimator for sparse signals this paper proposes a new approach to sparsity called the horseshoe estimator the horseshoe is a close cousin of other widely used bayes rules arising from for example double exponential and cauchy priors in that it is a member of the same family of multivariate scale mixtures of normals but the horseshoe enjoys a number of advantages over existing approaches including its robustness its adaptivity to dierent sparsity patterns and its analytical tractability we prove two theorems that formally characterize both the horseshoe s adeptness at large outlying signals and its super ecient rate of convergence to the correct estimate of the sampling density in sparse situations finally using a combination of real and simulated data we show that the horseshoe estimator corresponds quite closely to the answers one would get by pursuing a full bayesian model averaging approach using a discrete mixture prior to model signals and noise

controlling the false discovery rate a practical and powerful approach to multiple testing summary the common approach to the multiplicity problem calls for controlling the familywise error rate fwer this approach though has faults and we point out a few a different approach to problems of multiple significance testing is presented it calls for controlling the expected proportion of falsely rejected hypotheses the false discovery rate this error rate is equivalent to the fwer when all hypotheses are true but is smaller otherwise therefore in problems where the control of the false discovery rate rather than that of the fwer is desired there is potential for a gain in power a simple sequential bonferronitype procedure is proved to control the false discovery rate for independent test statistics and a simulation study shows that the gain in power is substantial the use of the new procedure and the appropriateness of the criterion are illustrated with examples

generalized fiducial inference for wavelet regression we apply fisher s fiducial idea to conduct statistical inference for wavelet regression we first develop a general methodology for handling model selection problems within the fiducial framework with this new methodology we then propose fiduical based methods for performing wavelet curve estimation as well as constructing both pointwise and curvewise confidence intervals it is shown that under some mild regularity conditions both the new fiducial based pointwise and curvewise confidence intervals have asymptotically correct coverage furthermore simulation results show that these new fiducial based methods especially for constructing pointwise confidence intervals also possess promising empirical properties to the best of our knowledge this is the first time that the fiducial idea has been applied to a nonparametric estimation problem

denoising methods for thermomechanical decomposition for quasi equilibrium molecular dynamics simulations in this work we applied robust denoising methods well established in the signal processing field for the thermomechanical decomposition of velocity data obtained from molecular dynamics md simulations in the decomposition the atomic velocity was assumed to be the sum of the mechanical velocity and the thermal velocity which can be linked to the stress and temperature field at the continuum scale respectively for the quasi equilibrium process with the thermal velocity treated as the gaussian distributed stationary noise with zero mean and a positive variance that is linearly proportional to temperature and mechanical velocity as the clean signal the velocity decomposition can be recasted into a denoising problem for which powerful denoising methods have been developed to estimate a clean signal from noisy data we investigated the widely used linear parametric real domain linear nonparametric fourier based and nonlinear nonparametric wavelet based denoising methods first on their theoretical properties and then made comparsion among them for denosing some synthetic noisy 1 dimensional 1d data generated from md simulations the nonlinear wavelet based thresholding estimator possessed better optimality properties than the other estimators and also outperformed the other estimators in the synthetic data test a further test comparing the various denoising methods for an adiabatic shear crack nucleation and propagation process simulated using md simulations showed better performance by the wavelet based denoisng method results from this work reveal good potential of applying wavelet based denoising method to the study of thermomechanical processes simulated using md simulations

wavelet thresholding via a bayesian approach we discuss a bayesian formalism which gives rise to a type of wavelet threshold estimation in nonparametric regression a prior distribution is imposed on the wavelet coefficients of the unknown response function designed to capture the sparseness of wavelet expansion that is common to most applications for the prior specified the posterior median yields a thresholding procedure our prior model for the underlying function can be adjusted to give functions falling in any specific besov space we establish a relationship between the hyperparameters of the prior model and the parameters of those besov spaces within which realizations from the prior will fall such a relationship gives insight into the meaning of the besov space parameters moreover the relationship established makes it possible in principle to incorporate prior knowledge about the function s regularity properties into the prior model for its wavelet coefficients however prior knowledge about a function s regularity properties might be difficult to elicit with this in mind we propose a standard choice of prior hyperparameters that works well in our examples several simulated examples are used to illustrate our method and comparisons are made with other thresholding methods we also present an application to a data set that was collected in an anaesthesiological study

wavelet decomposition approaches to statistical inverse problems summary a wide variety of scientific settings involve indirect noisy measurements where one faces a linear inverse problem in the presence of noise primary interest is in some function f t but data are accessible only about some linear transform corrupted by noise the usual linear methods for such inverse problems do not perform satisfactorily when f t is spatially inhomogeneous one existing nonlinear alternative is the wavelet vaguelette decomposition method based on the expansion of the unknown f t in wavelet series in the vaguelette wavelet decomposition method proposed here the observed data are expanded directly in wavelet series the performances of various methods are compared through exact risk calculations in the context of the estimation of the derivative of a function observed subject to noise a result is proved demonstrating that with a suitable universal threshold somewhat larger than that used for standard denoising problems both the wavelet based approaches have an ideal spatial adaptivity property

longitudinal data analysis using generalized linear models summary this paper proposes an extension of generalized linear models to the analysis of longitudinal data we introduce a class of estimating equations that give consistent estimates of the regression parameters and of their variance under mild assumptions about the time dependence the estimating equations are derived without specifying the joint distribution of a subject s observations yet they reduce to the score equations for multivariate gaussian outcomes asymptotic theory is presented for the general class of estimators specific cases in which we assume independence m dependence and exchangeable correlation structures from each subject are discussed efficiency of the proposed estimators in two simple situations is considered the approach is closely related to quasi likelih ood some key ironh estimating equation generalized linear model longitudinal data quasi likelihood repeated measures

the calculation of posterior distributions by data augmentation abstract the idea of data augmentation arises naturally in missing value problems as exemplified by the standard ways of filling in missing cells in balanced two way tables thus data augmentation refers to a scheme of augmenting the observed data so as to make it more easy to analyze this device is used to great advantage by the em algorithm dempster laird and rubin 1977 in solving maximum likelihood problems in situations when the likelihood cannot be approximated closely by the normal likelihood maximum likelihood estimates and the associated standard errors cannot be relied upon to make valid inferential statements from the bayesian point of view one must now calculate the posterior distribution of parameters of interest if data augmentation can be used in the calculation of the maximum likelihood estimate then in the same cases one ought to be able to use it in the computation of the posterior distribution it is the purpose of this article to explain how this can be done the basic idea