estimation of the mean of a multivariate normal distribution estimation of the means of independent normal random variables is considered using sum of squared errors as loss an unbiased estimate of risk is obtained for an arbitrary estimate and certain special classes of estimates are then discussed the results are applied to smoothing by use of moving averages and to trimmed analogs of the james stein estimate a suggestion is made for calculating approximate confidence sets for the mean vector centered at an arbitrary estimate

optimized wavelet denoising for self similar alpha stable processes we investigate the performance of wavelet shrinkage methods for the denoising of symmetric alpha stable text s alpha text s self similar stochastic processes corrupted by additive white gaussian noise awgn where alpha is tied to the sparsity of the process the wavelet transform is assumed to be orthonormal and the shrinkage function minimizes the mean square approximation error mmse estimator we derive the corresponding formula for the expected value of the averaged estimation error we show that the predicted mmse is a monotone function of a simple criterion that depends on the wavelet and the statistical parameters of the process using the calculus of variations we then optimize this criterion to find the best performing wavelet within the extended family of meyer wavelets which are bandlimited these are compared with the daubechies wavelets which are compactly supported in time we find that the wavelets that are shorter in time in particular the haar basis are better suited to denoise the sparser processes say alpha while the bandlimited ones including the held and shannon wavelets offer the best performance for alpha 1 6 the limit corresponding to the gaussian case fbm with alpha 2

bayesian inference for neighborhood filters with application in denoising range weighted neighborhood filters are useful and popular for their edge preserving property and simplicity but they are originally proposed as intuitive tools previous works needed to connect them to other tools or models for indirect property reasoning or parameter estimation in this paper we introduce a unified empirical bayesian framework to do both directly a neighborhood noise model is proposed to reason and infer the yaroslavsky bilateral and modified non local means filters by joint maximum a posteriori and maximum likelihood estimation then the essential parameter range variance can be estimated via model fitting to the empirical distribution of an observable chi scale mixture variable an algorithm based on expectation maximization and quasi newton optimization is devised to perform the model fitting efficiently finally we apply this framework to the problem of color image denoising a recursive fitting and filtering scheme is proposed to improve the image quality extensive experiments are performed for a variety of configurations including different kernel functions filter types and support sizes color channel numbers and noise types the results show that the proposed framework can fit noisy images well and the range variance can be estimated successfully and efficiently

dimension reduction and semiparametric estimation of survival models in this paper we propose a new dimension reduction method by introducing a nominal regression model with the hazard function as the conditional mean which naturally retrieves information from complete data and censored data as well moreover without requiring the linearity condition the new method can estimate the entire central subspace consistently and exhaustively the method also provides an alternative approach for the analysis of censored data assuming neither the link function nor the distribution hence it exhibits superior robustness properties numerical studies show that the method can indeed be readily used to efficiently estimate survival models explore the data structures and identify important variables

an unbiased risk estimator for image denoising in the presence of mixed poisson gaussian noise the behavior and performance of denoising algorithms are governed by one or several parameters whose optimal settings depend on the content of the processed image and the characteristics of the noise and are generally designed to minimize the mean squared error mse between the denoised image returned by the algorithm and a virtual ground truth in this paper we introduce a new poisson gaussian unbiased risk estimator pg ure of the mse applicable to a mixed poisson gaussian noise model that unifies the widely used gaussian and poisson noise models in fluorescence bioimaging applications we propose a stochastic methodology to evaluate this estimator in the case when little is known about the internal machinery of the considered denoising algorithm and we analyze both theoretically and empirically the characteristics of the pg ure estimator finally we evaluate the pg ure driven parametrization for three standard denoising algorithms with and without variance stabilizing transforms and different characteristics of the poisson gaussian noise mixture

random calibration with many measurements an application of stein estimation in the problem considered a vector of many imprecise measurements e g spectroscopic is used to linearly predict a quantity whose precise measurement is difficult or expensive the regression vector is estimated from a calibration experiment having both types of measurements for a random sample most previous approaches to this problem adjust for approximate multicollinearity which often results from the correlations among the imprecise measurements by inverting an approximation e g factor analytic to their covariance matrix in the approach here it is argued that the regression vector should lie in a lower dimensional suhspace determined by the principal components of the covariance matrix it is then estimated by applying a stein contraction of the least squares estimator to the principal components regression estimator examples using real data are presented in which the proposed estimator substantially improves on the ordinary least squares estimation

hyperanalytic denoising a new threshold rule for the estimation of a deterministic image immersed in noise is proposed the full estimation procedure is based on a separable wavelet decomposition of the observed image and the estimation is improved by introducing the new threshold to estimate the decomposition coefficients the observed wavelet coefficients are thresholded using the magnitudes of wavelet transforms of a small number of replicates of the image the replicates are calculated by extending the image into a vector valued hyperanalytic signal more than one hyperanalytic signal may be chosen and either the hypercomplex or riesz transforms are used to calculate this object the deterministic and stochastic properties of the observed wavelet coefficients of the hyperanalytic signal at a fixed scale and position index are determined a universal threshold is calculated for the proposed procedure an expression for the risk of an individual coefficient is derived the risk is calculated explicitly when the universal threshold is used and is shown to be less than the risk of universal hard thresholding under certain conditions the proposed method is implemented and the derived theoretical risk reductions substantiated

unbiased risk estimates for singular value thresholding and spectral estimators in an increasing number of applications it is of interest to recover an approximately low rank data matrix from noisy observations this paper develops an unbiased risk estimate holding in a gaussian model for any spectral estimator obeying some mild regularity assumptions in particular we give an unbiased risk estimate formula for singular value thresholding svt a popular estimation strategy that applies a soft thresholding rule to the singular values of the noisy observations among other things our formulas offer a principled and automated way of selecting regularization parameters in a variety of problems in particular we demonstrate the utility of the unbiased risk estimation for svt based denoising of real clinical cardiac mri series data we also give new results concerning the differentiability of certain matrix valued functions

it formulae for gamma target mutual information and relative entropy in this paper we introduce new stein identities for gamma target distribution as well as a new non linear channel specifically designed for gamma inputs from these two ingredients we derive an explicit and simple formula for the derivative of the input output mutual information of this non linear channel with respect to the channel quality parameter this relation is reminiscent of the well known link between the derivative of the input output mutual information of additive gaussian noise channel with respect to the signal to noise ratio and the minimum mean square error the proof relies on a rescaled version of de bruijn identity for gamma target distribution together with a stochastic representation for the gamma specific fisher information finally we are able to derive precise bounds and asymptotics for the input output mutual information of the non linear channel with gamma inputs

llsure local linear sure based edge preserving image filtering in this paper we propose a novel approach for performing high quality edge preserving image filtering based on a local linear model and using the principle of stein s unbiased risk estimate as an estimator for the mean squared error from the noisy image only we derive a simple explicit image filter which can filter out noise while preserving edges and fine scale details moreover this filter has a fast and exact linear time algorithm whose computational complexity is independent of the filtering kernel size thus it can be applied to real time image processing tasks the experimental results demonstrate the effectiveness of the new filter for various computer vision applications including noise reduction detail smoothing and enhancement high dynamic range compression and flash no flash denoising

wavelet threshold estimators for data with correlated noise wavelet threshold estimators for data with stationary correlated noise are constructed by applying a level dependent soft threshold to the coefficients in the wavelet transform a variety of threshold choices is proposed including one based on an unbiased estimate of mean squared error the practical performance of the method is demonstrated on examples including data from a neurophysiological context the theoretical properties of the estimators are investigated by comparing them with an ideal but unattainable bench mark that can be considered in the wavelet context as the risk obtained by ideal spatial adaptivity and more generally is obtained by the use of an oracle that provides information that is not actually available in the data it is shown that the level dependent threshold estimator performs well relative to the bench mark risk and that its minimax behaviour cannot be improved on in order of magnitude by any other estimator the wavelet domain structure of both short and long range dependent noise is considered and in both cases it is shown that the estimators have near optimal behaviour simultaneously in a wide range of function classes adapting automatically to the regularity properties of the underlying model the proofs of the main results are obtained by considering a more general multivariate normal decision theoretic problem

nonparametric empirical bayes growth curve analysis abstract nonparametric and semiparametric regression have been suggested as alternatives to parametric models for growth curve analysis in this article we demonstrate that empirical bayes estimation can be used to improve linear smoothing estimates when multiple curves are available

regularized discriminant analysis abstract linear and quadratic discriminant analysis are considered in the small sample high dimensional setting alternatives to the usual maximum likelihood plug in estimates for the covariance matrices are proposed these alternatives are characterized by two parameters the values of which are customized to individual situations by jointly minimizing a sample based estimate of future misclassification risk computationally fast implementations are presented and the efficacy of the approach is examined through simulation studies and application to data these studies indicate that in many circumstances dramatic gains in classification accuracy can be achieved

principal hessian directions for regression with measurement error summary we consider a nonlinear regression problem with predictors with measurement error we assume that the response is related to unknown linear combinations of a p dimensional predictor vector through an unknown link function instead of observing the predictors we observe a surrogate vector with the property that its expectation is linearly related to the predictor vector with constant variance we use an important linear transformation of the surrogates based on the transformed variables we develop the modified principal hessian directions method for estimating the subspace of the effective dimension reduction space we derive the asymptotic variances of the modified principal hessian directions estimators several examples are reported and comparisons are made with the sliced inverse regression method of carroll li 1992

an example in linear quadratic optimal control we construct a simple example of a quadratic optimal control problem for an infinite dimensional linear system based on a shift semigroup this system has an unbounded control operator the cost is quadratic in the input and the state and the weighting operators are bounded despite its extreme simplicity this example has all the unexpected features discovered recently by o staffans and also by m weiss and g weiss more precisely in the formula linking the optimal feedback operator to the optimal cost operator as well as in the riccati equation the weighting operator of the input has to be replaced by another operator which can be derived from the spectral factorization of the popov function

bayesian selection for the ell _2 potts model regularization parameter 1 d piecewise constant signal denoising piecewise constant denoising can be solved either by deterministic optimization approaches based on the potts model or by stochastic bayesian procedures the former lead to low computational time but require the selection of a regularization parameter whose value significantly impacts the achieved solution and whose automated selection remains an involved and challenging problem conversely fully bayesian formalisms encapsulate the regularization parameter selection into hierarchical models at the price of high computational costs this contribution proposes an operational strategy that combines hierarchical bayesian and potts model formulations with the double aim of automatically tuning the regularization parameter and maintaining computational efficiency the proposed procedure relies on formally connecting a bayesian framework to a ell 2 potts functional behaviors and performance for the proposed piecewise constant denoising and regularization parameter tuning techniques are studied qualitatively and assessed quantitatively and shown to compare favorably against those of a fully bayesian hierarchical procedure both in accuracy and computational load

misspecified nonconvex statistical optimization for sparse phase retrieval existing nonconvex statistical optimization theory and methods crucially rely on the correct specification of the underlying true statistical models to address this issue we take a first step towards taming model misspecification by studying the high dimensional sparse phase retrieval problem with misspecified link functions in particular we propose a simple variant of the thresholded wirtinger flow algorithm that given a proper initialization linearly converges to an estimator with optimal statistical accuracy for a broad family of unknown link functions we further provide extensive numerical experiments to support our theoretical findings

the sure let approach to image denoising we propose a new approach to image denoising based on the image domain minimization of an estimate of the mean squared error stein s unbiased risk estimate sure unlike most existing denoising algorithms using the sure makes it needless to hypothesize a statistical model for the noiseless image a key point of our approach is that although the nonlinear processing is performed in a transformed domain typically an undecimated discrete wavelet transform but we also address nonorthonormal transforms this minimization is performed in the image domain indeed we demonstrate that when the transform is a ldquotightrdquo frame an undecimated wavelet transform using orthonormal filters separate subband minimization yields substantially worse results in order for our approach to be viable we add another principle that the denoising process can be expressed as a linear combination of elementary denoising processes linear expansion of thresholds let armed with the sure and let principles we show that a denoising algorithm merely amounts to solving a linear system of equations which is obviously fast and efficient quite remarkably the very competitive results obtained by performing a simple threshold image domain sure optimized on the undecimated haar wavelet coefficients show that the sure let principle has a huge potential

small confidence sets for the mean of a spherically symmetric distribution summary suppose that x has a k variate spherically symmetric distribution with mean vector 0 and identity covariance matrix wepresent two spherical confidence sets for 0 both centred at a positive part stein estimator t x in the first we obtain the radius by approximating the upper a point of the sampling disiribution of iits x 9112 by the first two non zero terms of its taylor series about the origin we can analyse some of the properties of this confidence set and see that it performs well in terms of coverage probability volume and conditional behaviour in the second method we find the radius by using a parametric bootstrap procedure here even greater improvement in terms of volume over the usual confidence set is possible at the expense of having a less explicit radius function a real data example is provided and extensions to the unknown covariance matrix and elliptically symmetric cases are discussed

the estimation of prediction error having constructed a data based estimation rule perhaps a logistic regression or a classification tree the statistician would like to know its performance as a predictor of future cases there are two main theories concerning prediction error 1 penalty methods such as cp akaike s information criterion and stein s unbiased risk estimate that depend on the covariance between data points and their corresponding predictions and 2 cross validation and related nonparametric bootstrap techniques this article concerns the connection between the two theories a rao blackwell type of relation is derived in which nonparametric methods such as cross validation are seen to be randomized versions of their covariance penalty counterparts the model based penalty methods offer substantially better accuracy assuming that the model is believable

newton stein method an optimization method for glms via stein s lemma we consider the problem of efficiently computing the maximum likelihood estimator in generalized linear models glms when the number of observations is much larger than the number of coefficients n p 1 in this regime optimization algorithms can immensely benefit from approximate second order information we propose an alternative way of constructing the curvature information by formulating it as an estimation problem and applying a stein type lemma which allows further improvements through subsampling and eigenvalue thresholding our algorithm enjoys fast convergence rates resembling that of second order methods with modest per iteration cost we provide its convergence analysis for the general case where the rows of the design matrix are samples from a sub gaussian distribution we show that the convergence has two phases a quadratic phase followed by a linear phase finally we empirically demonstrate that our algorithm achieves the highest performance compared to various optimization algorithms on several data sets

bootstrapping improved estimators for linear regression models abstract since stein 1955 proved that the least squares ls estimator for the linear regression model is inadmissible and james and stein 1961 demonstrated a superior estimator under squared error loss a number of better estimators have been proposed one reason these estimators have not been used in empirical work is that it is difficult to derive their sampling distributions this study shows that nonparametric bootstrapping provides good estimates of the sampling distribution of mundlak s 1981 restricted principal components estimator and a new stein rule estimator which shrinks mundlak s towards ls a monte carlo study of the bootstrap is performed for a range of increasingly collinear designs with 100 observations and 10 regressors using a squared error loss function the stein rule estimator performed much better than ls over most of the designs the nonparametric bootstrap generated reasonable estimates of the estimator s risk as well as standard errors of the individual components

l_ 1 discretization for sampled data controller synthesis via piecewise linear approximation this paper develops a new discretization method with piecewise linear approximation for the l 1 optimal controller synthesis problem of sampled data systems which is the problem of minimizing the l infty induced norm of sampled data systems we apply fast lifting on the top of the lifting technique by which the sampling interval 0 h is divided into m subintervals with an equal width the signals on each subinterval are then approximated by linear functions by introducing two types of linearizing operators for input and output which leads to piecewise linear approximation of sampled data systems by using the arguments of preadjoint operators we provide an important inequality that forms a theoretical basis for tackling the l 1 optimal controller synthesis problem of sampled data systems more efficiently than the conventional method more precisely a mathematical basis for the piecewise linear approximation method associated with the convergence rate is shown through this inequality and this suggests that the piecewise linear approximation method may drastically outperform the conventional method in the l 1 optimal controller synthesis problem of sampled data systems we then provide a discretization procedure of sampled data systems by which the l 1 optimal controller synthesis problem is converted to the discrete time l 1 optimal controller synthesis problem finally effectiveness of the proposed method is demonstrated through a numerical example

the linear quadratic control problem for infinite dimensional systems with unbounded input and output operators this paper establishes a general semigroup framework for solving quadratic control problems with infinite dimensional state space and unbounded input and output operators

covariance identities and inequalities for functionals on wiener and poisson spaces we present covariance identities and inequalities for functionals of the wiener and the poisson processes using malliavin calculus techniques an expansion with a remainder term is obtained for the covariance of such functionals our results extend known identities and inequalities for functions of multivariate random vectors

a sure approach for digital signal image deconvolution problems in this paper we are interested in the classical problem of restoring data degraded by a convolution and the addition of a white gaussian noise the originality of the proposed approach is twofold first we formulate the restoration problem as a nonlinear estimation problem leading to the minimization of a criterion derived from stein s unbiased quadratic risk estimate secondly the deconvolution procedure is performed using any analysis and synthesis frames that can be overcomplete or not new theoretical results concerning the calculation of the variance of the stein s risk estimate are also provided in this work simulations carried out on natural images show the good performance of our method with respect to conventional wavelet based restoration methods

sensitivity analysis for box cox power transformation model contrast parameters summary sensitivity analysis is an important tool for the evaluation of scientific models we consider the box cox power transformation model and study the sensitivity for the estimated slope vector when the transformation parameter is perturbed this is a key issue in the debate among bickel doksum 1981 box cox 1982 and hinkley runger 1984 on the application of the power transformation model hinkley runger 1984 conjectured that box cox s 1964 z transformation would eliminate asymptotically the sensitivity for the estimated slope vector we establish this conjecture under appropriate symmetry conditions on the joint distribution for the regressors x when the true transformation is logarithmic the conjecture holds if the distribution for x is axially symmetric for other power transformations the conjecture holds if x is multivariate normal we also consider a weaker version of hinkley runger s conjecture namely the z transformation would achieve the best possible reduction in the sensitivity we establish this weaker conjecture under less restrictive symmetry conditions which only involve the marginal distribution for xfl both conjectures might fail when those symmetry conditions are not satisfied two examples are given to illustrate the limitations of the conjectures sensitivity analysis is an important tool for the evaluation of scientific models many scientific models include some nuisance parameters which are not known precisely it is usually desirable for the model output to be stable with respect to those parameters the performance of the model can be questionable if the model output is highly sensitive to those parameters it is therefore useful to carry out a sensitivity analysis in which those parameters are perturbed the model output is derived under the perturbed parameters and compared with the model output prior to the perturbation we will study the sensitivity analysis for the box cox power transformation model with respect to the specific transformation taken let yi xi i 1 n be the observed data where yi denotes a positive scalar outcome and xi denotes a p dimensional row vector of regressors let the transformed outcome be

risk reducing shrinkage estimation for generalized linear models to generalized linear models in a manner retaining the original spirit of shrinkage estimation which is to reduce risk the investigation identifies two classes of simple all purpose prior distributions which supplement such non informative priors as jeffreys s prior with mechanisms for risk reduction one new class of priors is motivated as optimizers of a core component of asymptotic risk the methodology is evaluated in a numerical exploration and application to an existing data set

transfer functions of regular linear systems part i characterizations of regularity we recall the main facts about the representation of regular linear systems essentially that they can be described by equations of the form x t ax t bu t y t cx t du t like finite dimensional systems but now a b and c are in general unbounded operators regular linear systems are a subclass of abstract linear systems we define transfer functions of abstract linear systems via a generalization of a theorem of foures and segal we prove a formula for the transfer function of a regular linear system which is similar to the formula in finite dimensions the main result is a simple to state but hard to prove necessary and sufficient condition for an abstract linear system to be regular in terms of its transfer function other conditions equivalent to regularity are also obtained the main result is a consequence of a new tauberian theorem which is of independent interest

on denoising and best signal representation we propose a best basis algorithm for signal enhancement in white gaussian noise the best basis search is performed in families of orthonormal bases constructed with wavelet packets or local cosine bases we base our search for the best basis on a criterion of minimal reconstruction error of the underlying signal this approach is intuitively appealing because the enhanced or estimated signal has an associated measure of performance namely the resulting mean square error previous approaches in this framework have focused on obtaining the most compact signal representations which consequently contribute to effective denoising these approaches however do not possess the inherent measure of performance which our algorithm provides we first propose an estimator of the mean square error based on a heuristic argument and subsequently compare the reconstruction performance based upon it to that based on the stein 1981 unbiased risk estimator we compare the two proposed estimators by providing both qualitative and quantitative analyses of the bias term having two estimators of the mean square error we incorporate these cost functions into the search for the best basis and subsequently provide a substantiating example to demonstrate their performance

minimax estimation with thresholding and its application to wavelet analysis many statistical practices involve choosing between a full model and reduced models where some coefficients are reduced to zero data were used to select a model with estimated coefficients is it possible to do so and still come up with an estimator always better than the traditional estimator based on the full model the james stein estimator is such an estimator having a property called minimaxity however the estimator considers only one reduced model namely the origin hence it reduces no coefficient estimator to zero or every coefficient estimator to zero in many applications including wavelet analysis what should be more desirable is to reduce to zero only the estimators smaller than a threshold called thresholding in this paper is it possible to construct this kind of estimators which are minimax in this paper we construct such minimax estimators which perform thresholding we apply our recommended estimator to the wavelet analysis and show that it performs the best among the well known estimators aiming simultaneously at estimation and model selection some of our estimators are also shown to be asymptotically optimal

adaptive wavelet estimation a block thresholding and oracle inequality approach we study wavelet function estimation via the approach of block thresh olding and ideal adaptation with oracle oracle inequalities are derived and serve as guides for the selection of smoothing parameters based on an oracle inequality and motivated by the data compression and localization properties of wavelets an adaptive wavelet estimator for nonparametric re gression is proposed and the optimality of the procedure is investigated we show that the estimator achieves simultaneously three objectives adaptiv ity spatial adaptivity and computational efficiency specifically it is proved that the estimator attains the exact optimal rates of convergence over a range of besov classes and the estimator achieves adaptive local minimax rate for estimating functions at a point the estimator is easy to imple ment at the computational cost of on simulation shows that the es timator has excellent numerical performance relative to more traditional wavelet estimators 1 introduction wavelet methods have demonstrated considerable suc cess in nonparametric function estimation in terms of spatial adaptivity com putational efficiency and asymptotic optimality in contrast to the traditional linear procedures wavelet methods achieve near optimal convergence rates over large function classes such as besov classes and enjoy excellent mean squared error properties when used to estimate functions that are spatially inhomogeneous for example as shown by donoho and johnstone 1998 wavelet methods can outperform optimal linear methods even at the level of convergence rate over certain besov classes standard wavelet methods achieve adaptivity through term by term thresholding of the empirical wavelet coefficients there each individual empirical wavelet coefficient is compared with a predetermined threshold a wavelet coefficient is retained if its magnitude is above the threshold level and is discarded otherwise a well known example of term by term thresholding is donoho and johnstone s visushrink donoho and johnstone 1994 visushrink is spatially adaptive and the estimator is within a log arithmic factor of the optimal convergence rate over a wide range of besov classes visushrink achieves a degree of tradeoff between variance and bias contributions to the mean squared error however the tradeoff is not optimal visushrink reconstruction is often over smoothed hall kerkyacharian and picard 1999 considered block thresholding for wavelet function estimation which thresholds empirical wavelet coefficients in

admissible factorizations of hankel operators induce well posed linear systems abstract one of the basic axioms of a well posed linear system says that the hankel operator of the input output map of the system factors into the product of the input map and the output map here we prove the converse every factorization of the hankel operator of a bounded causal time invariant map from l 2 to l 2 which satisfies a certain admissibility condition induces a stable well posed linear system in particular there is a one to one correspondence between the set of all minimal stable well posed realizations of a given stable causal time invariant input output map or equivalently of a given h transfer function and all minimal stable admissible factorizations of the hankel operator of this input output map

analysis of long period variable stars with nonparametric tests for trend detection in astronomy the study of variable stars that is stars characterized by showing significant variation in their brightness over time has made crucial contributions to our understanding of many phenomena from stellar birth and evolution to the calibration of the extragalactic distance scale in this article we develop a method for analyzing multiple pseudo periodic time series with the goal of detecting temporal trends in their periods we allow for nonstationary noise and for clustering among the various time series we apply this method to the long standing astronomical problem of identifying variable stars whose regular brightness fluctuations have periods that change over time the results of our analysis show that such changes can be substantial raising the possibility that astronomers estimates of galactic distances can be refined two significant contributions of our approach relative to existing methods for this problem are as follows 1 the method is nonparametric making minimal assumpt

on the robustness of shrinkage predictors in regression to differences between past and future data on etudie la robustesse des predicteurs de copas 1983 vis a vis des ecarts a la distribution

pure let image deconvolution we propose a non iterative image deconvolution algorithm for data corrupted by poisson or mixed poisson gaussian noise many applications involve such a problem ranging from astronomical to biological imaging we parameterize the deconvolution process as a linear combination of elementary functions termed as linear expansion of thresholds this parameterization is then optimized by minimizing a robust estimate of the true mean squared error the poisson unbiased risk estimate each elementary function consists of a wiener filtering followed by a pointwise thresholding of undecimated haar wavelet coefficients in contrast to existing approaches the proposed algorithm merely amounts to solving a linear system of equations which has a fast and exact solution simulation experiments over different types of convolution kernels and various noise levels indicate that the proposed method outperforms the state of the art techniques in terms of both restoration quality and computational complexity finally we present some results on real confocal fluorescence microscopy images and demonstrate the potential applicability of the proposed method for improving the quality of these images

bayesian robustness and the stein effect abstract in simultaneous estimation of normal means it is shown that through use of the stein effect surprisingly large gains of a bayesian nature can be achieved at little or no cost if the prior information is misspecified this provides a justification in terms of robustness with respect to mis specification of the prior for employing the stein effect even when combining a priori independent problems i e problems in which no empirical bayes effects are obtainable to study this issue a class of minimax estimators that closely mimic the conjugate prior bayes estimators is introduced

monte carlo sure a black box optimization of regularization parameters for general denoising algorithms we consider the problem of optimizing the parameters of a given denoising algorithm for restoration of a signal corrupted by white gaussian noise to achieve this we propose to minimize stein s unbiased risk estimate sure which provides a means of assessing the true mean squared error mse purely from the measured data without need for any knowledge about the noise free signal specifically we present a novel monte carlo technique which enables the user to calculate sure for an arbitrary denoising algorithm characterized by some specific parameter setting our method is a black box approach which solely uses the response of the denoising operator to additional input noise and does not ask for any information about its functional form this therefore permits the use of sure for optimization of a wide variety of denoising algorithms we justify our claims by presenting experimental results for sure based optimization of a series of popular image denoising algorithms such as total variation denoising wavelet soft thresholding and wiener filtering smoothing splines in the process we also compare the performance of these methods we demonstrate numerically that sure computed using the new approach accurately predicts the true mse for all the considered algorithms we also show that sure uncovers the optimal values of the parameters in all cases

adaptive wavelet thresholding for image denoising and compression the first part of this paper proposes an adaptive data driven threshold for image denoising via wavelet soft thresholding the threshold is derived in a bayesian framework and the prior used on the wavelet coefficients is the generalized gaussian distribution ggd widely used in image processing applications the proposed threshold is simple and closed form and it is adaptive to each subband because it depends on data driven estimates of the parameters experimental results show that the proposed method called bayesshrink is typically within 5 of the mse of the best soft thresholding benchmark with the image assumed known it also outperforms sureshrink donoho and johnstone 1994 1995 donoho 1995 most of the time the second part of the paper attempts to further validate claims that lossy compression can be used for denoising the bayesshrink threshold can aid in the parameter selection of a coder designed with the intention of denoising and thus achieving simultaneous denoising and compression specifically the zero zone in the quantization step of compression is analogous to the threshold value in the thresholding function the remaining coder design parameters are chosen based on a criterion derived from rissanen s minimum description length mdl principle experiments show that this compression method does indeed remove noise significantly especially for large noise power however it introduces quantization noise and should be used only if bitrate were an additional concern to denoising

quadratic optimal control of stable well posed linear systems we consider the infinite horizon quadratic cost minimization problem for a stable time invariant well posed linear system in the sense of salamon and weiss and show that it can be reduced to a spectral factorization problem in the control space more precisely we show that the optimal solution of the quadratic cost minimization problem is of static state feedback type if and only if a certain spectral factorization problem has a solution if both the system and the spectral factor are regular then the feedback operator can be expressed in terms of the riccati operator and the riccati operator is a positive self adjoint solution of an algebraic riccati equation this riccati equation is similar to the usual algebraic riccati equation but one of its coefficients varies depending on the subspace in which the equation is posed similar results are true for unstable systems as we have proved elsewhere

differential geometric least angle regression a differential geometric approach to sparse generalized linear models sparsity is an essential feature of many contemporary data problems remote sensing various forms of automated screening and other high throughput measurement devices collect a large amount of information typically about few independent statistical subjects or units in certain cases it is reasonable to assume that the underlying process generating the data is itself sparse in the sense that only a few of the measured variables are involved in the process we propose an explicit method of monotonically decreasing sparsity for outcomes that can be modelled by an exponential family in our approach we generalize the equiangular condition in a generalized linear model although the geometry involves the fisher information in a way that is not obvious in the simple regression setting the equiangular condition turns out to be equivalent with an intuitive condition imposed on the rao score test statistics in certain special cases the method can be tweaked to obtain l1 penalized generalized linear model solution paths but the method itself defines sparsity more directly although the computation of the solution paths is not trivial the method compares favourably with other path following algorithms

how to saif ly boost denoising performance spatial domain image filters e g bilateral filter non local means locally adaptive regression kernel have achieved great success in denoising their overall performance however has not generally surpassed the leading transform domain based filters such as bm3 d one important reason is that spatial domain filters lack efficiency to adaptively fine tune their denoising strength something that is relatively easy to do in transform domain method with shrinkage operators in the pixel domain the smoothing strength is usually controlled globally by for example tuning a regularization parameter in this paper we propose spatially adaptive iterative filtering saif a new strategy to control the denoising strength locally for any spatial domain method this approach is capable of filtering local image content iteratively using the given base filter and the type of iteration and the iteration number are automatically optimized with respect to estimated risk i e mean squared error in exploiting the estimated local signal to noise ratio we also present a new risk estimator that is different from the often employed sure method and exceeds its performance in many cases experiments illustrate that our strategy can significantly relax the base algorithm s sensitivity to its tuning smoothing parameters and effectively boost the performance of several existing denoising filters to generate state of the art results under both simulated and practical conditions

a note on conditional aic for linear mixed effects models the conventional model selection criterion aic has been applied to choose candidate models in mixed effects models by the consideration of marginal likelihood vaida and blanchard 2005 demonstrated that such a marginal aic and its small sample correction are inappropriate when the research focus is on clusters correspondingly these authors suggested to use conditional aic the conditional aic is derived under the assumptions of the variance covariance matrix or scaled variance covariance matrix of random effects being known we develop a general conditional aic but without these strong assumptions this allows vaida and blanchard s conditional aic to be applied in a wide range simulation studies show that the proposed method is promising

minimax risk of matrix denoising by singular value thresholding an unknown m by n matrix x 0 is to be estimated from noisy measurements y x 0 z where the noise matrix z has i i d gaussian entries a popular matrix denoising scheme solves the nuclear norm penalization problem operatorname min x y x f 2 2 lambda x where x denotes the nuclear norm sum of singular values this is the analog for matrices of ell 1 penalization in the vector case it has been empirically observed that if x 0 has low rank it may be recovered quite accurately from the noisy measurement y in a proportional growth framework where the rank r n number of rows m n and number of columns n all tend to infty proportionally to each other r n m n rightarrow rho m n n rightarrow beta we evaluate the asymptotic minimax mse mathcal m rho beta lim m n n rightarrow infty inf lambda sup operatorname rank x leq r n operatorname mse x 0 hat x lambda our formulas involve incomplete moments of the quarter and semi circle laws beta 1 square case and the marcenko pastur law beta 1 nonsquare case for finite m and n we show that mse increases as the nonzero singular values of x 0 grow larger as a result the finite n worst case mse a quantity which can be evaluated numerically is achieved when the signal x 0 is infinitely strong the nuclear norm penalization problem is solved by applying soft thresholding to the singular values of y we also derive the minimax threshold namely the value lambda rho which is the optimal place to threshold the singular values all these results are obtained for general nonsquare nonsymmetric real matrices comparable results are obtained for square symmetric nonnegative definite matrices

efficient shrinkage in parametric models this paper introduces shrinkage for general parametric models we show how to shrink maximum likelihood estimators towards parameter subspaces de ned by general nonlinear restrictions we derive the asymptotic distribution and risk of the generalized shrinkage estimator using a local asymptotic framework we show that if the shrinkage dimension exceeds two the asymptotic risk of the shrinkage estimator is strictly less than that of the mle this reduction holds globally in the parameter space we show that the reduction in asymptotic risk is substantial even for moderately large values of the parameters the formula simplify in a very convenient way in the context of high dimensional models we derive a simple bound for the asymptotic risk we also provide a new large sample minimax e ciency bound we use the concept of local asymptotic minimax bounds a generalization of the conventional asymptotic minimax bounds the

improved prediction in the presence of multicollinearity abstract in this paper monte carlo techniques are used to examine the performance of several estimators for the linear statistical model under a squared error of prediction loss measure when the data are multicollinear under this measure of performance the stein like rules that shrink toward the principal components estimator perform very well relative to other minimax estimators for alternative specifications of the characteristics root spectrum the sampling performance of a non minimax pretest rule is also considered

extending the scope of wavelet regression methods by coefficient dependent thresholding abstract various aspects of the wavelet approach to nonparametric regression are considered with the overall aim of extending the scope of wavelet techniques to irregularly spaced data to regularly spaced datasets of arbitrary size to heteroscedastic and correlated data and to data that contain outliers the core of the methodology is an algorithm for finding ail of the variances and within level covariances in the wavelet table of a sequence with given covariance structure if the original covariance matrix is band limited then the algorithm is linear in the length of the sequence the variance calculation algorithm allows data on any set of independent variable values to be treated by first interpolating to a fine regular grid of suitable length and then constructing a wavelet expansion of the gridded data various thresholding methods are discussed and investigated exact risk formulas for the mean square error of the methodology for given design are derived good performance is obtained by nois

infinite dimensional linear systems with unbounded control and observation a functional analytic approach abstract the object of these notes is to develop a unifying framework for the functional analytic representation of infinite dimensional linear systems with unbounded input and output operators on the basis of the general approach new results are derived on the wellposedness of feedback systems and on the linear quadratic control problem the implications of the theory for large classes of functional and partial differential equations are discussed in detail keywords representation of infinite dimensional systems semigroups boundary control feedback linear quadratic control

how much does stein estimation help in multiple linear regression the use of stein estimation in multiple linear regression is considered tables and graphs are presented that compare the prediction mean squared errors of positive part james stein preliminary test reduced and full model least squares estimates the appropriateness of using stein contraction on possibly extraneous variables is emphasized and a procedure is presented for evaluating the likely savings in using stein estimation on the problem at hand an example is given

riccati equations for strongly stabilizable bounded linear systems we obtain necessary and sufficient conditions for the existence of strongly stabilizing solutions to riccati equations for infinite dimensional systems that are strongly but not exponentially stabilizable we apply the theory to derive a riccati criterion for the existence of j spectral factorizations and illustrate it by proving the strict positive real and bounded real lemmas for a class of strongly stable systems with collocated actuators and sensors

on minimax estimation of a sparse normal mean vector mallows has conjectured that among distributions which are gaussian but for occasional contamination by additive noise the one having least fisher information has two sided geometric contamination a very similar problem arises in estimation of a non negative vector parameter in gaussian white noise when it is known also that most i e 1i components are zero we provide a partial asymptotic expansion of the minimax risk as 0 while the conjecture seems unlikely to be exactly true for flnite we verify it asymptotically up to the accuracy of the expansion numerical work suggests the expansion is accurate for as large as 05 the best 1 estimation rule is flrst but not second order minimax the results bear on an earlier study of maximum entropy estimation and various questions in robustness and function estimation using wavelet bases

regularization parameter selection for nonlinear iterative image restoration and mri reconstruction using gcv and sure based methods regularized iterative reconstruction algorithms for imaging inverse problems require selection of appropriate regularization parameter values we focus on the challenging problem of tuning regularization parameters for nonlinear algorithms for the case of additive possibly complex gaussian noise generalized cross validation gcv and weighted mean squared error mse approaches based on stein s unbiased risk estimate sure need the jacobian matrix of the nonlinear reconstruction operator representative of the iterative algorithm with respect to the data we derive the desired jacobian matrix for two types of nonlinear iterative algorithms a fast variant of the standard iterative reweighted least squares method and the contemporary split bregman algorithm both of which can accommodate a wide variety of analysis and synthesis type regularizers the proposed approach iteratively computes two weighted sure type measures predicted sure and projected sure which require knowledge of noise variance 2 and gcv which does not need 2 for these algorithms we apply the methods to image restoration and to magnetic resonance image mri reconstruction using total variation and an analysis type l1 regularization we demonstrate through simulations and experiments with real data that minimizing predicted sure and projected sure consistently lead to near mse optimal reconstructions we also observe that minimizing gcv yields reconstruction results that are near mse optimal for image restoration and slightly suboptimal for mri theoretical derivations in this paper related to jacobian matrix evaluations can be extended in principle to other types of regularizers and reconstruction algorithms

on the discrete and continuous time infinite dimensional algebraic riccati equations the standard state space solution of the finite dimensional continuous time quadratic cost minimization problem has a straightforward extension to infinite dimensional problems with bounded or moderately unbounded control and observation operators however if these operators are allowed to be sufficiently unbounded then a strange change takes place in one of the coefficients of the algebraic riccati equation and the continuous time riccati equation begins to resemble the discrete time riccati equation to explain why this phenomenon must occur we discuss a particular hyperbolic pde in one space dimension with boundary control and observation a transmission line that can be formulated both as a discrete time system and as a continuous time system and show that in this example the continuous time riccati equation can be recovered from the discrete time riccati equation a particular feature of this example is that the riccati operator does not map the domain of the generator into the domain of the adjoint generator as it does in the standard case

simple regression in view of elliptical models abstract for the simple linear model y 1 x where the error vector follows the elliptically contoured distribution we consider the unrestricted restricted preliminary test and shrinkage estimators for the intercept parameter when it is suspected that the slope parameter may be o the exact bias and mse expressions are derived and the mean square relative efficiency is taken to determine the relative dominance properties of the proposed estimators in comparison in the continuation the optimal level of significance of the preliminary test estimator is tabulated and some graphical result are also displayed

on the predictive risk in misspecified quantile regression abstract in the present paper we investigate the predictive risk of possibly misspecified quantile regression functions the in sample risk is well known to be an overly optimistic estimate of the predictive risk and we provide two relatively simple asymptotic characterizations of the associated bias also called expected optimism we propose estimates for the expected optimism and the predictive risk and establish their uniform consistency under mild conditions our results hold for models of moderately growing size and allow the quantile function to be incorrectly specified empirical evidence from our estimates is encouraging as it compares favorably with cross validation

simultaneous estimation of location parameters under quadratic loss simultaneous estimation of p p 3 location parameters are considered under quadratic loss explicit estimators which dominate the best invariant one are given mainly when coordinates of the best invariant one are independently identically and symmetrically distributed effectiveness of integration by parts in evaluating the risk function of the dominating estimator is shown for three typical continuous distributions uniform double exponential and t further explicit dominating estimators are given in terms of second and fourth moments of the best invariant one