the cramer rao bound for estimating a sparse parameter vector the goal of this contribution is to characterize the best achievable mean squared error mse in estimating a sparse deterministic parameter from measurements corrupted by gaussian noise to this end an appropriate definition of bias in the sparse setting is developed and the constrained cramer rao bound crb is obtained this bound is shown to equal the crb of an estimator with knowledge of the support set for almost all feasible parameter values consequently in the unbiased case our bound is identical to the mse of the oracle estimator combined with the fact that the crb is achieved at high signal to noise ratios signal to noise ratio snrs by the maximum likelihood technique our result provides a new interpretation for the common practice of using the oracle estimator as a gold standard against which practical approaches are compared

compressed sensing with basis mismatch performance bounds and sparse based estimator compressed sensing cs is a promising emerging domain that outperforms the classical limit of the shannon sampling theory if the measurement vector can be approximated as the linear combination of few basis vectors extracted from a redundant dictionary matrix unfortunately in realistic scenario the knowledge of this basis or equivalently of the entire dictionary is often uncertain i e corrupted by a basis mismatch bm error the consequence of the bm problem is that the estimation accuracy in terms of bayesian mean square error bmse of popular sparse based estimators collapses even if the support is perfectly estimated and in the high signal to noise ratio snr regime this saturation effect considerably limits the effective viability of these estimation schemes in the first part of this work the bayesian cramer rao bound bcrb is derived for cs model with unstructured bm we show that the bcrb foresees the saturation effect of the estimation accuracy of standard sparse based estimators as for instance the omp cosamp or the bp in addition we provide an approximation of this bmse threshold in the second part and in the context of the structured bm model a new estimation scheme called bias correction estimator bice is proposed and its statistical properties are studied the bice acts as a post processing estimation layer for any sparse based estimator and mitigates considerably the bm degradation finally the bice i is a blind algorithm i e is unaware of the uncorrupted dictionary matrix ii is generic since it can be associated to any sparse based estimator iii is fast i e the additional computational cost remains low and iv has good statistical properties to illustrate our results and propositions the bice is applied in the challenging context of the compressive sampling of non bandlimited impulsive signals

time delay estimation from low rate samples a union of subspaces approach time delay estimation arises in many applications in which a multipath medium has to be identified from pulses transmitted through the channel previous methods for time delay recovery either operate on the analog received signal or require sampling at the nyquist rate of the transmitted pulse in this paper we develop a unified approach to time delay estimation from low rate samples this problem can be formulated in the broader context of sampling over an infinite union of subspaces although sampling over unions of subspaces has been receiving growing interest previous results either focus on unions of finite dimensional subspaces or finite unions the framework we develop here leads to perfect recovery of the multipath delays from samples of the channel output at the lowest possible rate even in the presence of overlapping transmitted pulses and allows for a variety of different sampling methods the sampling rate depends only on the number of multipath components and the transmission rate but not on the bandwidth of the probing signal this result can be viewed as a sampling theorem over an infinite union of infinite dimensional subspaces by properly manipulating the low rate samples we show that the time delays can be recovered using the well known esprit algorithm combining results from sampling theory with those obtained in the context of direction of arrival estimation we develop sufficient conditions on the transmitted pulse and the sampling functions in order to ensure perfect recovery of the channel parameters at the minimal possible rate

finite element solution of the helmholtz equation with high wave number part i the h version of the fem abstract the paper addresses the properties of finite element solutions for the helmholtz equation the h version of the finite element method with piecewise linear approximation is applied to a one dimensional model problem new results are shown on stability and error estimation of the discrete model in all propositions assumptions are made on the magnitude of hk only where k is the wavelength and h is the stepwidth of the fe mesh previous analytical results had been shown with the assumption that k 2 h is small for medium and high wavenumber these results do not cover the meshsizes that are applied in practical applications the main estimate reveals that the error in h 1 norm of discrete solutions for the helmholtz equation is polluted when k 2 h is not small the error is then not quasioptimal i e the relation of the fe error to the error of best approximation generally depends on the wavenumber k it is noted that the pollution term in the relative error is of the same order as the phase lead of the numerical solution in the result of this analysis thorough and rigorous understanding of error behavior throughout the range of convergence is gained numerical results are presented that show sharpness of the error estimates and highlight some phenomena of the discrete solution behavior the h p version of the fem is studied in part ii

cram acute text e r rao bound for constrained parameter estimation using lehmann unbiasedness the constrained cram acute text e r rao bound ccrb is a lower bound on the mean squared error mse of estimators that satisfy some unbiasedness conditions although the ccrb unbiasedness conditions are satisfied asymptotically by the constrained maximum likelihood cml estimator in the non asymptotic region these conditions are usually too strict and the commonly used estimators such as the cml estimator do not satisfy them therefore the ccrb may not be a lower bound on the mse matrix of such estimators in this paper we propose a new definition for unbiasedness under constraints denoted by c unbiasedness which is based on using lehmann unbiasedness with a weighted mse wmse risk and taking into account the parametric constraints in addition to c unbiasedness a cram acute text e r rao type bound on the wmse of c unbiased estimators denoted as lehmann unbiased ccrb lu ccrb is derived this bound is a scalar bound that depends on the chosen weighted combination of estimation errors it is shown that c unbiasedness is less restrictive than the ccrb unbiasedness conditions thus the set of estimators that satisfy the ccrb unbiasedness conditions is a subset of the set of c unbiased estimators and the proposed lu ccrb may be an informative lower bound in cases where the corresponding ccrb is not in the simulations we examine linear and nonlinear estimation problems under nonlinear constraints in which the cml estimator is shown to be c unbiased and the lu ccrb is an informative lower bound on the wmse while the corresponding ccrb on the wmse is not a lower bound and is not informative in the non asymptotic region

legendre spectral galerkin method for electromagnetic scattering from large cavities the paper is concerned with the electromagnetic scattering from a large cavity embedded in an infinite ground plane which is governed by a helmholtz type equation with nonlocal hypersingular transparent boundary condition on the aperture we first present some stability estimates with the explicit dependency of wavenumber for the helmholtz type cavity problem then a legendre spectral galerkin method is proposed in which the legendre gauss interpolatory approximation is applicable to the hypersingular integral and a legendre galerkin scheme is used for the approximation to the helmholtz equation the existence and the uniqueness of the approximation solution are established for large wavenumbers the stability and the spectral convergence of the numerical method are then proved illustrative numerical results presented confirm our theoretical estimates and show that the proposed spectral method compared with low order finite difference methods is especially effective for problems with large wavenumbers

the rkhs approach to minimum variance estimation revisited variance bounds sufficient statistics and exponential families the mathematical theory of reproducing kernel hilbert spaces rkhss provides powerful tools for minimum variance estimation mve problems here we extend the classical rkhs based analysis of mve in several directions we develop a geometric formulation of five known lower bounds on the estimator variance barankin bound cramer rao bound constrained cramer rao bound bhattacharyya bound and hammersley chapman robbins bound in terms of orthogonal projections onto a subspace of the rkhs associated with a given mve problem we show that under mild conditions the barankin bound the tightest possible lower bound on the estimator variance is a lower semicontinuous function of the parameter vector we also show that the rkhs associated with an mve problem remains unchanged if the observation is replaced by a sufficient statistic finally for mve problems conforming to an exponential family of distributions we derive novel closed form lower bounds on the estimator variance and show that a reduction of the parameter set leaves the minimum achievable variance unchanged

image denoising via sparse and redundant representations over learned dictionaries we address the image denoising problem where zero mean white and homogeneous gaussian additive noise is to be removed from a given image the approach taken is based on sparse and redundant representations over trained dictionaries using the k svd algorithm we obtain a dictionary that describes the image content effectively two training options are considered using the corrupted image itself or training on a corpus of high quality image database since the k svd is limited in handling small image patches we extend its deployment to arbitrary image sizes by defining a global image prior that forces sparsity over patches in every location in the image we show how such bayesian treatment leads to a simple and effective denoising algorithm this leads to a state of the art denoising performance equivalent and sometimes surpassing recently published leading alternative denoising methods

lower bounds on the mean squared error of low rank matrix reconstruction we investigate the behavior of the mean square error mse of low rank matrix reconstruction and its special case matrix completion we first derive the constrained cramer rao bound crb on the mse matrix of any locally unbiased estimator and then analyze the behavior of the constrained crb when a subset of entries of the underlying matrix is randomly observed we design an alternating minimization procedure to compute the maximum likelihood estimator mle for the low rank matrix and demonstrate through numerical simulations that the performance of the mle approaches the constrained crb when the signal to noise ratio is high applying a chapman robbins type barankin bound allows us to derive lower bounds on the worst case scalar mse we demonstrate that the worst case scalar mse is infinite even if the model is identifiable however the infinite scalar mse is achieved only on a set of low rank matrices with measure zero we discuss the implications of these bounds and compare them with the empirical performance of the matrix lasso estimator and the existing bounds in the literature

performance bounds and design criteria for estimating finite rate of innovation signals in this paper we consider the problem of estimating finite rate of innovation fri signals from noisy measurements and specifically analyze the interaction between fri techniques and the underlying sampling methods we first obtain a fundamental limit on the estimation accuracy attainable regardless of the sampling method next we provide a bound on the performance achievable using any specific sampling approach essential differences between the noisy and noise free cases arise from this analysis in particular we identify settings in which noise free recovery techniques deteriorate substantially under slight noise levels thus quantifying the numerical instability inherent in such methods this instability which is only present in some families of fri signals is shown to be related to a specific type of structure which can be characterized by viewing the signal model as a union of subspaces finally we develop a methodology for choosing the optimal sampling kernels for linear reconstruction based on a generalization of the karhunen loeve transform the results are illustrated for several types of time delay estimation problems

asymptotic achievability of the cramer rao bound for noisy compressive sampling we consider a model of the form y ax n where x isin cm is sparse with at most l nonzero coefficients in unknown locations y isin cn is the observation vector a isin cn times m is the measurement matrix and n isin cn is the gaussian noise we develop a cramer rao bound on the mean squared estimation error of the nonzero elements of x corresponding to the genie aided estimator gae which is provided with the locations of the nonzero elements of x intuitively the mean squared estimation error of any estimator without the knowledge of the locations of the nonzero elements of x is no less than that of the gae assuming that l n is fixed we establish the existence of an estimator that asymptotically achieves the cramer rao bound without any knowledge of the locations of the nonzero elements of x as n rarr infin for a a random gaussian matrix whose elements are drawn i i d according to n 0 1

a new derivation of constrained cramer rao bound via norm minimization the constrained cramer rao bound ccrb is widely used to evaluate the estimation performance for deterministic parameters with parametric constraints an alternative derivation of the ccrb is presented from the perspective of solving a norm minimization problem under a set of linear constraints the parametric constraints are embedded in the linear constraints by constructing a proper basis to represent the estimation error this derivation avoids sophisticated matrix manipulations and has a clear physical meaning that the parametric constraints cut down the bases used to represent the estimation error and hence reduce the minimum norm of the error and the bound can be achieved if and only if the error is in the subspace spanned by the bases the result is applicable to biased estimators and singular fisher information matrices

a generalized finite element method for solving the helmholtz equation in two dimensions with minimal pollution when using the galerkin fem for solving the helmholtz equation in two dimensions the error of the corresponding solution differs substantially from the error of the best approximation and this effect increases with higher wave number k in this paper we will design a generalized finite element method gfem for the helmholtz equation such that the pollution effect is minimal

projection design for statistical compressive sensing a tight frame based approach in this paper we develop a framework to design sensing matrices for compressive sensing applications that lead to good mean squared error mse performance subject to sensing cost constraints by capitalizing on the mse of the oracle estimator whose performance has been shown to act as a benchmark to the performance of standard sparse recovery algorithms we use the fact that a parseval tight frame is the closest design in the frobenius norm sense to the solution of a convex relaxation of the optimization problem that relates to the minimization of the mse of the oracleestimator with respect to the equivalent sensing matrix subject to sensing energy constraints based on this result we then propose two sensing matrix designs that exhibit two key properties the designs are closed form rather than iterative the designs exhibit superior performance in relation to other designs in the literature which is revealed by our numerical investigation in various scenarios with different sparse recovery algorithms including basis pursuit de noise bpdn the dantzig selector and orthogonal matching pursuit omp

lower bounds for parametric estimation with constraints a chapman robbins form of the barankin bound is used to derive a multiparameter cramer rao cr type lower bound on estimator error covariance when the parameter theta in r sup n is constrained to lie in a subset of the parameter space a simple form for the constrained cr bound is obtained when the constraint set theta sub c can be expressed as a smooth functional inequality constraint it is shown that the constrained cr bound is identical to the unconstrained cr bound at the regular points of theta sub c i e where no equality constraints are active on the other hand at those points theta in theta sub c where pure equality constraints are active the full rank fisher information matrix in the unconstrained cr bound must be replaced by a rank reduced fisher information matrix obtained as a projection of the full rank fisher matrix onto the tangent hyperplane of the full rank fisher matrix onto the tangent hyperplane of the constraint set at theta a necessary and sufficient condition involving the forms of the constraint and the likelihood function is given for the bound to be achievable and examples for which the bound is achieved are presented in addition to providing a useful generalization of the cr bound the results permit analysis of the gain in achievable mse performance due to the imposition of particular constraints on the parameter space without the need for a global reparameterization

greene s residue criterion for the breakup of invariant tori of volume preserving maps abstract invariant tori play a fundamental role in the dynamics of symplectic and volume preserving maps codimension one tori are particularly important as they form barriers to transport such tori foliate the phase space of integrable volume preserving maps with one action and d angles for the area preserving case greene s residue criterion is often used to predict the destruction of tori from the properties of nearby periodic orbits even though kam theory applies to the three dimensional case the robustness of tori in such systems is still poorly understood we study a three dimensional reversible volume preserving analogue of chirikov s standard map with one action and two angles we investigate the preservation and destruction of tori under perturbation by computing the residue of nearby periodic orbits we find tori with diophantine rotation vectors in the spiral mean cubic algebraic field the residue is used to generate the critical function of the map and find a candidate for the most robust torus

unbiased estimation of a sparse vector in white gaussian noise the problem studied in this paper is unbiased estimation of a sparse nonrandom vector corrupted by additive white gaussian noise it is shown that while there are infinitely many unbiased estimators for this problem none of them has uniformly minimum variance therefore the focus is placed on locally minimum variance unbiased lmvu estimators simple closed form lower and upper bounds on the variance of lmvu estimators or equivalently on the barankin bound bb are derived these bounds allow an estimation of the threshold region separating the low signal to noise ratio snr and high snr regimes and they indicate the asymptotic behavior of the bb at high snr in addition numerical lower and upper bounds are derived these are tighter than the closed form bounds and thus characterize the bb more accurately numerical studies compare the proposed characterizations of the bb with established biased estimation schemes and demonstrate that while unbiased estimators perform poorly at low snr they may perform better than biased estimators at high snr an interesting conclusion of this analysis is that the high snr behavior of the bb depends solely on the value of the smallest nonzero entry of the sparse vector and that this type of dependence is also exhibited by the performance of certain practical estimators

cramer rao type bounds for sparse bayesian learning in this paper we derive hybrid bayesian and marginalized cramer rao lower bounds hcrb bcrb and mcrb for the single and multiple measurement vector sparse bayesian learning sbl problem of estimating compressible vectors and their prior distribution parameters we assume the unknown vector to be drawn from a compressible student t prior distribution we derive crbs that encompass the deterministic or random nature of the unknown parameters of the prior distribution and the regression noise variance we extend the mcrb to the case where the compressible vector is distributed according to a general compressible prior distribution of which the generalized pareto distribution is a special case we use the derived bounds to uncover the relationship between the compressibility and mean square error mse in the estimates further we illustrate the tightness and utility of the bounds through simulations by comparing them with the mse performance of two popular sbl based estimators we find that the mcrb is generally the tightest among the bounds derived and that the mse performance of the expectation maximization em algorithm coincides with the mcrb for the compressible vector we also illustrate the dependence of the mse performance of sbl based estimators on the compressibility of the vector for several values of the number of observations and at different signal powers

fundamentals of statistical signal processing estimation theory minimum variance unbiased estimation cramer rao lower bound linear models general minimum variance unbiased estimation best linear unbiased estimators maximum likelihood estimation least squares method of moments the bayesian philosophy general bayesian estimators linear bayesian estimators kalman filters summary of estimators extension for complex data and parameters

spectral approximation of the helmholtz equation with high wave numbers a complete error analysis is performed for the spectral galerkin approximation of a model helmholtz equation with high wave numbers the analysis presented in this paper does not rely on the explicit knowledge of continuous discrete green s functions and does not require any mesh condition to be satisfied furthermore new error estimates are also established for multi dimensional radial and spherical symmetric domains illustrative numerical results in agreement with the theoretical analysis are presented

the superconvergence of the composite trapezoidal rule for hadamard finite part integrals the composite trapezoidal rule has been well studied and widely applied for numerical integrations and numerical solution of integral equations with smooth or weakly singular kernels however this quadrature rule has been less employed for hadamard finite part integrals due to the fact that its global convergence rate for hadamard finite part integrals with p 1 order singularity is p order lower than that for the riemann integrals in general in this paper we study the superconvergence of the composite trapezoidal rule for hadamard finite part integrals with the second order and the third order singularity respectively we obtain superconvergence estimates at some special points and prove the uniqueness of the superconvergence points numerical experiments confirm our theoretical analysis and show that the composite trapezoidal rule is efficient for hadamard finite part integrals by noting the superconvergence phenomenon

blind minimax estimation we consider the linear regression problem of estimating an unknown deterministic parameter vector based on measurements corrupted by colored gaussian noise we present and analyze blind minimax estimators bmes which consist of a bounded parameter set minimax estimator whose parameter set is itself estimated from measurements thus our approach does not require any prior assumption or knowledge and the proposed estimator can be applied to any linear regression problem we demonstrate analytically that the bmes strictly dominate the least squares ls estimator i e they achieve lower mean squared error mse for any value of the parameter vector both stein s estimator and its positive part correction can be derived within the blind minimax framework furthermore our approach can be readily extended to a wider class of estimation problems than stein s estimator which is defined only for white noise and nontransformed measurements we show through simulations that the bmes generally outperform previous extensions of stein s technique

on the non existence of unbiased estimators in constrained estimation problems we address the problem of existence of unbiased constrained parameter estimators we show that if the constrained set of parameters is compact and the hypothesized distributions are absolutely continuous with respect to one another then there exists no unbiased estimator weaker conditions for the absence of unbiased constrained estimators are also specified we provide several examples which demonstrate the utility of these conditions

sparse conjoint analysis through maximum likelihood estimation conjoint analysis ca is a classical tool used in preference assessment where the objective is to estimate the utility function of an individual or a group of individuals based on expressed preference data an example is choice based ca for consumer profiling i e unveiling consumer utility functions based solely on choices between products a statistical model for choice based ca is investigated in this paper unlike recent classification based approaches a sparsity aware gaussian maximum likelihood ml formulation is proposed to estimate the model parameters drawing from related robust parsimonious modeling approaches the model uses sparsity constraints to account for outliers and to detect the salient features that influence decisions contributions include conditions for statistical identifiability derivation of the pertinent cramer rao lower bound crlb and ml consistency conditions for the proposed sparse nonlinear model the proposed ml approach lends itself naturally to l1 type convex relaxations which are well suited for distributed implementation based on the alternating direction method of multipliers admm a particular decomposition is advocated which bypasses the apparent need for outlier communication thus maintaining scalability the performance of the proposed ml approach is demonstrated by comparing against the associated crlb and prior state of the art using both synthetic and real data sets