the cramer rao bound for estimating a sparse parameter vector the goal of this contribution is to characterize the best achievable mean squared error mse in estimating a sparse deterministic parameter from measurements corrupted by gaussian noise to this end an appropriate definition of bias in the sparse setting is developed and the constrained cramer rao bound crb is obtained this bound is shown to equal the crb of an estimator with knowledge of the support set for almost all feasible parameter values consequently in the unbiased case our bound is identical to the mse of the oracle estimator combined with the fact that the crb is achieved at high signal to noise ratios signal to noise ratio snrs by the maximum likelihood technique our result provides a new interpretation for the common practice of using the oracle estimator as a gold standard against which practical approaches are compared

lapped transforms for efficient transform subband coding two lapped transforms for subband transform coding of signals are introduced a version of the lapped orthogonal transform lot which can be efficiently computed for any transform length and the modulated lapped transform mlt which is based on a modulated quadrature mirror qmf bank the mlt can also be efficiently computed by means of a type iv discrete sine transform dst iv the lot and the mlt are both asymptotically optimal lapped transforms for coding an ar 1 signal with a high intersample correlation coefficient the coding gains of the lot and mlt of length m are higher than that of the discrete cosine transform dct of the same length they are actually close to the coding gains obtained with a dct of length 2m an mlt based adaptive transform coder act for speech signals is simulated the code is essentially free from frame rate noise and has a better spectral resolution that dct based atc systems

enumeration of downsampling lattices in two dimensional multirate systems lattices are essential for resampling in two dimensional multirate systems however the resampling matrix for a given lattice is not unique for a given resampling ratio how many different lattices exist and how to parameterize them have not been addressed before in this correspondence such enumeration and parametrization are presented for the cases where the resampling ratio is prime or a composite number the set of nonseparable resamplers is also studied examples and an application of the proposed enumeration and parametrization are included for demonstration

coprimality of certain families of integer matrices commuting coprime pairs of integer matrices have been of interest in multidimensional multirate systems and more recently in array processing in multirate systems they arise for example in the design of interchangeable cascades of decimator and expander matrices in array processing they arise in the construction of dense coarrays from sparse sensors located on a pair of lattices for the important case of two dimensional signals these matrices have size 2 2 in this paper the condition for coprimality is derived for several classes of 2 2 integer matrices namely circulant skew circulant and triangular families the first two are also commuting families for each class the special case of adjugate pairs which automatically commute is also elaborated it is also shown that the problem of testing coprimality of two 2 2 matrices is equvialent to testing coprimality of a pair of triangular matrices which can be done almost by inspection also considered is the case of 3 3 triangular matrices and their adjugates which have potential applications in three dimensional signal processing

structures for anticausal inverses and application in multirate filter banks anticausal or time reversed inversion of digital filters has gained importance in the implementation of digital filter banks anticausal inversion has in the past been shown to be possible by using block processing with appropriate state initialization with a b c d denoting the state space description of a structure implementing a filter g z the anticausal inverse can be essentially regarded as a filter structure having an inverted state space description which we denote as a spl circ b spl circ c spl circ d spl circ it is usually not efficient to implement the state space equations given by a spl circ b spl circ c spl circ d spl circ directly because of excessive multiplier count rather one seeks to find an efficient structure having the inverse description a spl circ b spl circ c spl circ d spl circ while this can be done by inspection in simple cases such as the direct form structure systematic procedures for other important structures have yet to be developed we derive anticausal inverse structures corresponding to several standard iir filter structures such as the direct form cascade form coupled form and the entire family of iir lattice structures including the tapped cascaded lattice we introduce the notion of a causal dual which we find convenient in the derivations we show that the limit cycle free property of the original structure is inherited by the causal dual in some but not all cases

compressed sensing with basis mismatch performance bounds and sparse based estimator compressed sensing cs is a promising emerging domain that outperforms the classical limit of the shannon sampling theory if the measurement vector can be approximated as the linear combination of few basis vectors extracted from a redundant dictionary matrix unfortunately in realistic scenario the knowledge of this basis or equivalently of the entire dictionary is often uncertain i e corrupted by a basis mismatch bm error the consequence of the bm problem is that the estimation accuracy in terms of bayesian mean square error bmse of popular sparse based estimators collapses even if the support is perfectly estimated and in the high signal to noise ratio snr regime this saturation effect considerably limits the effective viability of these estimation schemes in the first part of this work the bayesian cramer rao bound bcrb is derived for cs model with unstructured bm we show that the bcrb foresees the saturation effect of the estimation accuracy of standard sparse based estimators as for instance the omp cosamp or the bp in addition we provide an approximation of this bmse threshold in the second part and in the context of the structured bm model a new estimation scheme called bias correction estimator bice is proposed and its statistical properties are studied the bice acts as a post processing estimation layer for any sparse based estimator and mitigates considerably the bm degradation finally the bice i is a blind algorithm i e is unaware of the uncorrupted dictionary matrix ii is generic since it can be associated to any sparse based estimator iii is fast i e the additional computational cost remains low and iv has good statistical properties to illustrate our results and propositions the bice is applied in the challenging context of the compressive sampling of non bandlimited impulsive signals

time delay estimation from low rate samples a union of subspaces approach time delay estimation arises in many applications in which a multipath medium has to be identified from pulses transmitted through the channel previous methods for time delay recovery either operate on the analog received signal or require sampling at the nyquist rate of the transmitted pulse in this paper we develop a unified approach to time delay estimation from low rate samples this problem can be formulated in the broader context of sampling over an infinite union of subspaces although sampling over unions of subspaces has been receiving growing interest previous results either focus on unions of finite dimensional subspaces or finite unions the framework we develop here leads to perfect recovery of the multipath delays from samples of the channel output at the lowest possible rate even in the presence of overlapping transmitted pulses and allows for a variety of different sampling methods the sampling rate depends only on the number of multipath components and the transmission rate but not on the bandwidth of the probing signal this result can be viewed as a sampling theorem over an infinite union of infinite dimensional subspaces by properly manipulating the low rate samples we show that the time delays can be recovered using the well known esprit algorithm combining results from sampling theory with those obtained in the context of direction of arrival estimation we develop sufficient conditions on the transmitted pulse and the sampling functions in order to ensure perfect recovery of the channel parameters at the minimal possible rate

frame theory based analysis and design of oversampled filter banks direct computational method this paper studies the frames corresponding to oversampled filter banks fbs for this class of fb frames we present a state space parameterization of perfect reconstruction fb frames and explicit and numerically efficient formulas to compute the tightest frame bounds to obtain the dual fb frame and to construct a tight paraunitary fb frame from a given untight nonparaunitary fb frame the derivation uses well developed techniques from modern control theory which results in the unified formulas for generic infinite impulse response iir and finite impulse response fir fbs these formulas involve only algebraic manipulations of real matrices and can be computed efficiently reliably and exactly without the approximation required in the existing methods for generic fbs the results provide a unified framework for frame theory based analysis and systematic design of oversampled filter banks

perfect reconstruction fir filter banks some properties and factorizations perfect reconstruction finite impulse response fir filter banks are analyzed both in the z transform and time domains showing the alternatives between designs in the two domains various classes of perfect reconstruction schemes are indicated and relations between previously known systems are given windowed modulated filter banks with low computational complexity and perfect reconstruction are shown new factorizations of polyphase filter matrices leading in particular to linear phase filters are given the computational complexity and the architecture of the new structures are indicated

the generalized lapped pseudo biorthogonal transform oversampled linear phase perfect reconstruction filterbanks with lattice structures we investigate a lattice structure for a special class of n channel oversampled linear phase perfect reconstruction filterbanks with a decimation factor m smaller than n we deal with systems in which all analysis and synthesis filters have the same finite impulse response fir length and share the same center of symmetry we provide the minimal lattice factorization of a polyphase matrix of a particular class of these oversampled filterbanks fbs all filter coefficients are parameterized by rotation angles and positive values the resulting lattice structure is able to provide fast implementation and allows us to determine the filter coefficients by solving an unconstrained optimization problem we consider next the case where we are given the generalized lapped pseudo biorthogonal transform glpbt lattice structure with specific parameters and we a priori know the correlation matrix of noise that is added in the transform domain in this case we provide an alternative lattice structure that suppress the noise we show that the proposed systems with the lattice structure cover a wide range of linear phase perfect reconstruction fbs we also introduce a new cost function for oversampled fb design that can be obtained by generalizing the conventional coding gain finally we exhibit several design examples and their properties

orthonormal and biorthonormal filter banks as convolvers and convolutional coding gain convolution theorems for filter bank transformers are introduced both uniform and nonuniform decimation ratios are considered and orthonormal as well as biorthonormal cases are addressed all the theorems are such that the original convolution reduces to a sum of shorter decoupled convolutions in the subbands that is there is no need to have cross convolution between subbands for the orthonormal case expressions for optimal bit allocation and the optimized coding gain are derived the contribution to coding gain comes partly from the nonuniformity of the signal spectrum and partly from nonuniformity of the filter spectrum with one of the convolved sequences taken to be the unit pulse function e coding gain expressions reduce to those for traditional subband and transform coding the filter bank convolver has about the same computational complexity as a traditional convolver if the analysis bank has small complexity compared to the convolution itself

cram acute text e r rao bound for constrained parameter estimation using lehmann unbiasedness the constrained cram acute text e r rao bound ccrb is a lower bound on the mean squared error mse of estimators that satisfy some unbiasedness conditions although the ccrb unbiasedness conditions are satisfied asymptotically by the constrained maximum likelihood cml estimator in the non asymptotic region these conditions are usually too strict and the commonly used estimators such as the cml estimator do not satisfy them therefore the ccrb may not be a lower bound on the mse matrix of such estimators in this paper we propose a new definition for unbiasedness under constraints denoted by c unbiasedness which is based on using lehmann unbiasedness with a weighted mse wmse risk and taking into account the parametric constraints in addition to c unbiasedness a cram acute text e r rao type bound on the wmse of c unbiased estimators denoted as lehmann unbiased ccrb lu ccrb is derived this bound is a scalar bound that depends on the chosen weighted combination of estimation errors it is shown that c unbiasedness is less restrictive than the ccrb unbiasedness conditions thus the set of estimators that satisfy the ccrb unbiasedness conditions is a subset of the set of c unbiased estimators and the proposed lu ccrb may be an informative lower bound in cases where the corresponding ccrb is not in the simulations we examine linear and nonlinear estimation problems under nonlinear constraints in which the cml estimator is shown to be c unbiased and the lu ccrb is an informative lower bound on the wmse while the corresponding ccrb on the wmse is not a lower bound and is not informative in the non asymptotic region

rational sampling filter banks based on iir filters the problem of splitting the spectrum of a digital signal by using nonuniform infinite impulse response iir filter banks is addressed near perfect reconstruction npr is considered the method uses the modulation of different iir prototypes the cancellation of the main aliasing components constrains the prototypes to be dependent on each other by using this approach linear phase prototypes are needed and noncausal filtering is required numerical examples of filter bank design are given and the computational complexity is compared with the finite impulse response fir case

paraunitary filter banks over finite fields in real and complex fields unitary and paraunitary pu matrices have found many applications in signal processing there has been interest in extending these ideas to the case of finite fields we study the theory of pu filter banks fbs in gf q with q prime various properties of unitary and pu matrices in finite fields are studied in particular a number of factorization theorems are given we show that i all unitary matrices in gf q are factorizable in terms of householder like matrices and permutation matrices and ii the class of first order pu matrices the lapped orthogonal transform in finite fields can always be expressed as a product of degree one or degree two building blocks if q 2 we do not need degree two building blocks while many properties of pu matrices in finite fields are similar to those of pu matrices in complex field there are a number of differences for example unlike the conventional pu systems in finite fields there are pu systems that are unfactorizable in terms of smaller building blocks in fact in the special case of 2 spl times 2 systems all pu matrices that are factorizable in terms of degree one building blocks are diagonal matrices we derive results for both the cases of gf 2 and gf q with q 2 even though they share some similarities there are many differences between these two cases

the rkhs approach to minimum variance estimation revisited variance bounds sufficient statistics and exponential families the mathematical theory of reproducing kernel hilbert spaces rkhss provides powerful tools for minimum variance estimation mve problems here we extend the classical rkhs based analysis of mve in several directions we develop a geometric formulation of five known lower bounds on the estimator variance barankin bound cramer rao bound constrained cramer rao bound bhattacharyya bound and hammersley chapman robbins bound in terms of orthogonal projections onto a subspace of the rkhs associated with a given mve problem we show that under mild conditions the barankin bound the tightest possible lower bound on the estimator variance is a lower semicontinuous function of the parameter vector we also show that the rkhs associated with an mve problem remains unchanged if the observation is replaced by a sufficient statistic finally for mve problems conforming to an exponential family of distributions we derive novel closed form lower bounds on the estimator variance and show that a reduction of the parameter set leaves the minimum achievable variance unchanged

lapped unimodular transform and its factorization two types of lapped transforms have been studied in detail in the literature namely the lapped orthogonal transform lot and its extension the biorthogonal lapped transform bolt in this paper we will study the lapped unimodular transform lut all three transforms are first order matrices with finite impulse response fir inverses we will show that like lot and bolt all luts can be factorized into degree one unimodular matrices the factorization is both minimal and complete we will also show that all first order systems with fir inverses can be minimally factorized as a cascade of degree one lot bolt and lut building blocks two examples will be given to demonstrate that despite having a very small system delay the luts have a satisfactory performance in comparison with lot and bolt

multivariate mimo fir inverses the problem of computing exact finite impulse response fir inverses for multivariate multiple input multiple output mimo fir systems is considered necessary and sufficient conditions for invertibility are given along with computation techniques random systems and structured systems are defined necessary and sufficient conditions for the almost sure invertibility of structured random systems are derived bounds on the orders of the inverse filters are computed

image denoising via sparse and redundant representations over learned dictionaries we address the image denoising problem where zero mean white and homogeneous gaussian additive noise is to be removed from a given image the approach taken is based on sparse and redundant representations over trained dictionaries using the k svd algorithm we obtain a dictionary that describes the image content effectively two training options are considered using the corrupted image itself or training on a corpus of high quality image database since the k svd is limited in handling small image patches we extend its deployment to arbitrary image sizes by defining a global image prior that forces sparsity over patches in every location in the image we show how such bayesian treatment leads to a simple and effective denoising algorithm this leads to a state of the art denoising performance equivalent and sometimes surpassing recently published leading alternative denoising methods

lower bounds on the mean squared error of low rank matrix reconstruction we investigate the behavior of the mean square error mse of low rank matrix reconstruction and its special case matrix completion we first derive the constrained cramer rao bound crb on the mse matrix of any locally unbiased estimator and then analyze the behavior of the constrained crb when a subset of entries of the underlying matrix is randomly observed we design an alternating minimization procedure to compute the maximum likelihood estimator mle for the low rank matrix and demonstrate through numerical simulations that the performance of the mle approaches the constrained crb when the signal to noise ratio is high applying a chapman robbins type barankin bound allows us to derive lower bounds on the worst case scalar mse we demonstrate that the worst case scalar mse is infinite even if the model is identifiable however the infinite scalar mse is achieved only on a set of low rank matrices with measure zero we discuss the implications of these bounds and compare them with the empirical performance of the matrix lasso estimator and the existing bounds in the literature

performance bounds and design criteria for estimating finite rate of innovation signals in this paper we consider the problem of estimating finite rate of innovation fri signals from noisy measurements and specifically analyze the interaction between fri techniques and the underlying sampling methods we first obtain a fundamental limit on the estimation accuracy attainable regardless of the sampling method next we provide a bound on the performance achievable using any specific sampling approach essential differences between the noisy and noise free cases arise from this analysis in particular we identify settings in which noise free recovery techniques deteriorate substantially under slight noise levels thus quantifying the numerical instability inherent in such methods this instability which is only present in some families of fri signals is shown to be related to a specific type of structure which can be characterized by viewing the signal model as a union of subspaces finally we develop a methodology for choosing the optimal sampling kernels for linear reconstruction based on a generalization of the karhunen loeve transform the results are illustrated for several types of time delay estimation problems

asymptotic achievability of the cramer rao bound for noisy compressive sampling we consider a model of the form y ax n where x isin cm is sparse with at most l nonzero coefficients in unknown locations y isin cn is the observation vector a isin cn times m is the measurement matrix and n isin cn is the gaussian noise we develop a cramer rao bound on the mean squared estimation error of the nonzero elements of x corresponding to the genie aided estimator gae which is provided with the locations of the nonzero elements of x intuitively the mean squared estimation error of any estimator without the knowledge of the locations of the nonzero elements of x is no less than that of the gae assuming that l n is fixed we establish the existence of an estimator that asymptotically achieves the cramer rao bound without any knowledge of the locations of the nonzero elements of x as n rarr infin for a a random gaussian matrix whose elements are drawn i i d according to n 0 1

linear phase perfect reconstruction filter bank lattice structure design and application in image coding a lattice structure for an m channel linear phase perfect reconstruction filter bank lpprfb based on the singular value decomposition svd is introduced the lattice can be proven to use a minimal number of delay elements and to completely span a large class of lpprfbs all analysis and synthesis filters have the same fir length sharing the same center of symmetry the lattice also structurally enforces both linear phase and perfect reconstruction properties is capable of providing fast and efficient implementation and avoids the costly matrix inversion problem in the optimization process from a block transform perspective the new lattice can be viewed as representing a family of generalized lapped biorthogonal transform glbt with an arbitrary number of channels m and arbitrarily large overlap the relaxation of the orthogonal constraint allows the glbt to have significantly different analysis and synthesis basis functions which can then be tailored appropriately to fit a particular application several design examples are presented along with a high performance glbt based progressive image coder to demonstrate the potential of the new transforms

a new derivation of constrained cramer rao bound via norm minimization the constrained cramer rao bound ccrb is widely used to evaluate the estimation performance for deterministic parameters with parametric constraints an alternative derivation of the ccrb is presented from the perspective of solving a norm minimization problem under a set of linear constraints the parametric constraints are embedded in the linear constraints by constructing a proper basis to represent the estimation error this derivation avoids sophisticated matrix manipulations and has a clear physical meaning that the parametric constraints cut down the bases used to represent the estimation error and hence reduce the minimum norm of the error and the bound can be achieved if and only if the error is in the subspace spanned by the bases the result is applicable to biased estimators and singular fisher information matrices

recursive time varying filter banks for subband image coding filter banks subband wavelets and multiresolution decompositions that employ recursive filters have been considered previously and are recognized for their efficiency in partitioning the frequency spectrum this paper presents an analysis of a new infinite impulse response iir filter bank in which these computationally efficient filters may be changed adaptively in response to the input the new filter bank framework is presented and discussed in the context of subband image coding in the absence of quantization errors exact reconstruction can be achieved by the proper choice of an adaptation scheme it is shown that recursive linear time varying ltv filter banks can yield improvement over conventional ones

coprime sensing via chinese remaindering over quadratic fields part i array designs a coprime antenna array consists of two or more sparse subarrays featuring enhanced degrees of freedom dof and reduced mutual coupling this paper introduces a new class of planar coprime arrays based on the theory of ideal lattices in quadratic number fields a splitting prime p can be decomposed into the product of two distinct prime ideals which give rise to the two sparse subarrays their virtual difference coarray enjoys a quadratic gain in dof thanks to the generalized chinese remainder theorem crt to enlarge the contiguous aperture of the coarray we present hole free symmetric crt arrays with simple closed form expressions the ring of gaussian integers and the ring of eisenstein integers are considered as examples to demonstrate the procedure of designing coprime arrays with eisenstein integers our design yields a difference coarray that is a subset of the hexagonal lattice offering a significant gain in dof over the rectangular lattice given the same physical areas maximization of crt arrays in the aspect of essentialness and the superior performance in the context of angle estimation will be presented in the companion paper part ii

a new approach for estimation of statistically matched wavelet this paper presents a new approach for the estimation of wavelets that is matched to a given signal in the statistical sense based on this approach a number of new methods to estimate statistically matched wavelets are proposed the paper first proposes a new method for the estimation of statistically matched two band compactly supported biorthogonal wavelet system second a new method is proposed to estimate statistically matched semi orthogonal two band wavelet system that results in compactly supported or infinitely supported wavelet next the proposed method of estimating two band wavelet system is generalized to m band wavelet system here the key idea lies in the estimation of analysis wavelet filters from a given signal this is similar to a sharpening filter used in image enhancement the output of analysis highpass filter branch is viewed to be equivalent to an error in estimating the middle sample from the neighborhood to minimize this error a minimum mean square error mmse criterion is employed since wavelet expansion acts like karhunen loe spl grave ve type expansion for generalized 1 f sup spl beta processes it is assumed that the given signal is a sample function of an mth order fractional brownian motion therefore the autocorrelation structure of a generalized 1 f sup spl beta process is used in the estimation of analysis filters using the mmse criterion we then present methods to design a finite impulse response infinite impulse response fir iir biorthogonal perfect reconstruction filterbank leading to the estimation of a compactly supported infinitely supported statistically matched wavelet the proposed methods are very simple simulation results to validate the proposed theory are presented for different synthetic self similar signals as well as music and speech clips estimated wavelets for different signals are compared with standard biorthogonal 9 7 and 5 3 wavelets for the application of compression and are shown to have better results

sparse conjoint analysis through maximum likelihood estimation conjoint analysis ca is a classical tool used in preference assessment where the objective is to estimate the utility function of an individual or a group of individuals based on expressed preference data an example is choice based ca for consumer profiling i e unveiling consumer utility functions based solely on choices between products a statistical model for choice based ca is investigated in this paper unlike recent classification based approaches a sparsity aware gaussian maximum likelihood ml formulation is proposed to estimate the model parameters drawing from related robust parsimonious modeling approaches the model uses sparsity constraints to account for outliers and to detect the salient features that influence decisions contributions include conditions for statistical identifiability derivation of the pertinent cramer rao lower bound crlb and ml consistency conditions for the proposed sparse nonlinear model the proposed ml approach lends itself naturally to l1 type convex relaxations which are well suited for distributed implementation based on the alternating direction method of multipliers admm a particular decomposition is advocated which bypasses the apparent need for outlier communication thus maintaining scalability the performance of the proposed ml approach is demonstrated by comparing against the associated crlb and prior state of the art using both synthetic and real data sets

exact and robust reconstructions of integer vectors based on multidimensional chinese remainder theorem md crt the robust chinese remainder theorem crt has been recently proposed for robustly reconstructing a large nonnegative integer from erroneous remainders it has found many applications in signal processing including phase unwrapping and frequency estimation under sub nyquist sampling motivated by the applications in multidimensional md signal processing in this article we propose the md crt and robust md crt for integer vectors specifically by rephrasing the abstract crt for rings in number theoretic terms we first derive the md crt for integer vectors with respect to a general set of integer matrix moduli which provides an algorithm to uniquely reconstruct an integer vector from its remainders if it is in the fundamental parallelepiped of the lattice generated by a least common right multiple of all the moduli for some special forms of moduli we present explicit reconstruction formulae moreover we derive the robust md crt for integer vectors when the remaining integer matrices of all the moduli left divided by their greatest common left divisor gcld are pairwise commutative and coprime two different reconstruction algorithms are proposed and accordingly two different conditions on the remainder error bound for the reconstruction robustness are obtained which are related to a quarter of the minimum distance of the lattice generated by the gcld of all the moduli or the smith normal form of the gcld

role of anticausal inverses in multirate filter banks i system theoretic fundamentals in a maximally decimated filter bank with identical decimation ratios for all channels the perfect reconstructibility property and the nature of reconstruction filters causality stability fir property and so on depend on the properties of the polyphase matrix various properties and capabilities of the filter bank depend on the properties of the polyphase matrix as well as the nature of its inverse in this paper we undertake a study of the types of inverses and characterize them according to their system theoretic properties i e properties of state space descriptions mcmillan degree degree of determinant and so forth we find in particular that causal polyphase matrices with anticausal inverses have an important role in filter bank theory we study their properties both for the fir and iir cases techniques for implementing anticausal iir inverses based on state space descriptions are outlined it is found that causal fir matrices with anticausal fir inverses cafacafi have a key role in the characterization of fir filter banks

projection design for statistical compressive sensing a tight frame based approach in this paper we develop a framework to design sensing matrices for compressive sensing applications that lead to good mean squared error mse performance subject to sensing cost constraints by capitalizing on the mse of the oracle estimator whose performance has been shown to act as a benchmark to the performance of standard sparse recovery algorithms we use the fact that a parseval tight frame is the closest design in the frobenius norm sense to the solution of a convex relaxation of the optimization problem that relates to the minimization of the mse of the oracleestimator with respect to the equivalent sensing matrix subject to sensing energy constraints based on this result we then propose two sensing matrix designs that exhibit two key properties the designs are closed form rather than iterative the designs exhibit superior performance in relation to other designs in the literature which is revealed by our numerical investigation in various scenarios with different sparse recovery algorithms including basis pursuit de noise bpdn the dantzig selector and orthogonal matching pursuit omp

lower bounds for parametric estimation with constraints a chapman robbins form of the barankin bound is used to derive a multiparameter cramer rao cr type lower bound on estimator error covariance when the parameter theta in r sup n is constrained to lie in a subset of the parameter space a simple form for the constrained cr bound is obtained when the constraint set theta sub c can be expressed as a smooth functional inequality constraint it is shown that the constrained cr bound is identical to the unconstrained cr bound at the regular points of theta sub c i e where no equality constraints are active on the other hand at those points theta in theta sub c where pure equality constraints are active the full rank fisher information matrix in the unconstrained cr bound must be replaced by a rank reduced fisher information matrix obtained as a projection of the full rank fisher matrix onto the tangent hyperplane of the full rank fisher matrix onto the tangent hyperplane of the constraint set at theta a necessary and sufficient condition involving the forms of the constraint and the likelihood function is given for the bound to be achievable and examples for which the bound is achieved are presented in addition to providing a useful generalization of the cr bound the results permit analysis of the gain in achievable mse performance due to the imposition of particular constraints on the parameter space without the need for a global reparameterization

role of anticausal inverses in multirate filter banks ii the fir case factorizations and biorthogonal lapped transforms for pt i see ibid vol 43 no 5 p 1090 1990 in part i we studied the system theoretic properties of discrete time transfer matrices in the context of inversion and classified them according to the types of inverses they had in particular we outlined the role of causal fir matrices with anticausal fir inverses abbreviated cafacafi in the characterization of fir perfect reconstruction pr filter banks essentially all fir pr filter banks can be characterized by causal fir polyphase matrices having anticausal fir inverses in this paper we introduce the most general degree one cafacafi building block and consider the problem of factorizing cafacafi systems into these building blocks factorizability conditions are developed a special class of cafacafi systems called the biorthogonal lapped transform bolt is developed and shown to be factorizable this is a generalization of the well known lapped orthogonal transform lot examples of unfactorizable cafacafi systems are also demonstrated finally it is shown that any causal fir matrix with fir inverse can be written as a product of a factorizable cafacafi system and a unimodular matrix

unbiased estimation of a sparse vector in white gaussian noise the problem studied in this paper is unbiased estimation of a sparse nonrandom vector corrupted by additive white gaussian noise it is shown that while there are infinitely many unbiased estimators for this problem none of them has uniformly minimum variance therefore the focus is placed on locally minimum variance unbiased lmvu estimators simple closed form lower and upper bounds on the variance of lmvu estimators or equivalently on the barankin bound bb are derived these bounds allow an estimation of the threshold region separating the low signal to noise ratio snr and high snr regimes and they indicate the asymptotic behavior of the bb at high snr in addition numerical lower and upper bounds are derived these are tighter than the closed form bounds and thus characterize the bb more accurately numerical studies compare the proposed characterizations of the bb with established biased estimation schemes and demonstrate that while unbiased estimators perform poorly at low snr they may perform better than biased estimators at high snr an interesting conclusion of this analysis is that the high snr behavior of the bb depends solely on the value of the smallest nonzero entry of the sparse vector and that this type of dependence is also exhibited by the performance of certain practical estimators

time domain oversampled lapped transforms theory structure and application in image coding this paper generalizes time domain lapped transforms tdlts proposed by tran et al to oversampled systems thus leading to time domain oversampled lapped transforms tdolts these new transforms correspond to a subclass of oversampled linear phase perfect reconstruction filterbanks olpprfbs which can be implemented by adding a prefilter before the discrete cosine transform dct and a post filter after the inverse discrete cosine transform idct structures of the pre and post filters are developed and the frame theoretic properties of tdolts are analyzed a new parameterization of lattice matrices through the givens qr factorization is proposed for unconstrained optimization comparisons with other parameterization methods are also included several design examples along with some image coding results are presented to demonstrate the validity of the theory and the potential of tdolts in image coding especially in error resilient coding

sparse sensing with co prime samplers and arrays this paper considers the sampling of temporal or spatial wide sense stationary wss signals using a co prime pair of sparse samplers several properties and applications of co prime samplers are developed first for uniform spatial sampling with m and n sensors where m and n are co prime with appropriate interelement spacings the difference co array has o mn freedoms which can be exploited in beamforming and in direction of arrival estimation an m point dft filter bank and an n point dft filter bank can be used at the outputs of the two sensor arrays and their outputs combined in such a way that there are effectively mn bands i e mn narrow beams with beamwidths proportional to 1 mn a result following from co primality the ideas are applicable to both active and passive sensing though the details and tradeoffs are different time domain sparse co prime samplers also generate a time domain co array with o mn freedoms which can be used to estimate the autocorrelation at much finer lags than the sample spacings this allows estimation of power spectrum of an arbitrary signal with a frequency resolution proportional to 2 mnt even though the pairs of sampled sequences xc ntn and xc mtn in the time domain can be arbitrarily sparse in fact from the sparse set of samples xc ntn and xc mtn one can estimate o mn frequencies in the range t it will be shown that the co array based method for estimating sinusoids in noise offers many advantages over methods based on the use of chinese remainder theorem and its extensions examples are presented throughout to illustrate the various concepts

instantaneous erasures in oversampled filter banks conditions for output perfect reconstruction in this paper we aim at finding the conditions that an oversampled filter bank ofb should satisfy in order to maintain its perfect reconstruction property when erasures happen in the subband domain this problem has been addressed before mainly from frame theoretic point of view and only for the case of what we call in this paper classic erasure in the frame theoretic approach the stable filter banks are associated with frames in l2 bbz and a subband erasure is defined as the deletion of the frame expansion coefficients corresponding to the frame vectors resulting from each filter and all its translated versions this is equivalent to assuming that all the samples of a subband have been completely lost classic erasure and this is why in this approach it is always assumed that each channel is either working perfectly or not at all in this paper we extend this notion of erasure to a situation where subband channels are allowed to be on or off arbitrarily in each time instance and we define a new type of erasure called instantaneous erasure using an approach based on the time domain analysis of perfect reconstruction property we introduce general conditions for perfect reconstruction of the output and also the sufficient conditions for two classes of filter banks causal ofbs with causal inverse and ofbs with maximum robustness against classic erasure

perfect reconstruction iir digital filter banks supporting nonexpansive linear signal extensions in this paper perfect reconstruction polyphase infinite impulse response iir filter banks involving causal and anticausal inverses are examined for finite length signals first a novel and efficient nonexpansive perfect reconstruction algorithm based on the state space implementation is presented then the proposed method is extended to support linear signal extensions at the boundaries in a nonexpansive manner the powerfulness of the proposed algorithm is demonstrated with the image compression results

unidirectional covering codes a code c spl sube z sup n sub 2 where z sub 2 0 1 has unidirectional covering radius r if r is the smallest integer so that any word in z sup n sub 2 can be obtained from at least one codeword c spl isin c by replacing either 1s by 0s in at most r coordinates or 0s by 1s in at most r coordinates the minimum cardinality of such a code is denoted by e n r upper bounds on this function are here obtained by constructing codes using tabu search lower bounds on the other hand are mainly obtained by integer programming and exhaustive search best known bounds on e n r for n spl les 13 and r spl les 6 are tabulated

lagrange wavelets for signal processing this paper deals with the design of interpolating wavelets based on a variety of lagrange functions combined with novel signal processing techniques for digital imaging halfband lagrange wavelets b spline lagrange wavelets and gaussian lagrange lagrange distributed approximating functional daf wavelets are presented as specific examples of the generalized lagrange wavelets our approach combines the perceptually dependent visual group normalization vgn technique and a softer logic masking slm method these are utilized to rescale the wavelet coefficients remove perceptual redundancy and obtain good visual performance for digital image processing

cramer rao type bounds for sparse bayesian learning in this paper we derive hybrid bayesian and marginalized cramer rao lower bounds hcrb bcrb and mcrb for the single and multiple measurement vector sparse bayesian learning sbl problem of estimating compressible vectors and their prior distribution parameters we assume the unknown vector to be drawn from a compressible student t prior distribution we derive crbs that encompass the deterministic or random nature of the unknown parameters of the prior distribution and the regression noise variance we extend the mcrb to the case where the compressible vector is distributed according to a general compressible prior distribution of which the generalized pareto distribution is a special case we use the derived bounds to uncover the relationship between the compressibility and mean square error mse in the estimates further we illustrate the tightness and utility of the bounds through simulations by comparing them with the mse performance of two popular sbl based estimators we find that the mcrb is generally the tightest among the bounds derived and that the mse performance of the expectation maximization em algorithm coincides with the mcrb for the compressible vector we also illustrate the dependence of the mse performance of sbl based estimators on the compressibility of the vector for several values of the number of observations and at different signal powers

eigenstructure approach for characterization of fir pr filterbanks with order one polyphase a new approach for the characterization of m channel finite impulse response fir perfect reconstruction pr filterbanks is proposed by appropriately restricting the eigenstructure of the polyphase matrix of the bank a complete characterization of order one polyphase matrices is obtained in which the polynomial part is in a block diagonal form nilpotent matrices play a crucial role in the structure this structure allows imposing restrictions on the order of the inverse of the polyphase matrix and or analysis synthesis delay reconstruction delay next we derive an alternate complete characterization in terms of the degree of the determinant and the mcmillan degree of order one polyphase matrix which we call the dyadic based characterization the characterization of vaidyanathan and chen 1995 for matrices with anticausal inverse turns out to be a special case of the proposed characterization the dyadic based characterization is more suitable for design without any above mentioned restriction since it allows better initialization we finally present design examples with different cost functions

the commutativity of up downsampling in two dimensions the authors state and prove a theorem solving the problem of commutativity in two dimensions it is shown under which conditions upsampling and downsampling can be interchanged in two dimensions this is the generalization to arbitrary two dimensional lattices of the result that one dimensional upsampling and downsampling commute if and only if their sampling rates are coprime some illustrative examples are given the result holds for arbitrary sampling lattices

fundamentals of statistical signal processing estimation theory minimum variance unbiased estimation cramer rao lower bound linear models general minimum variance unbiased estimation best linear unbiased estimators maximum likelihood estimation least squares method of moments the bayesian philosophy general bayesian estimators linear bayesian estimators kalman filters summary of estimators extension for complex data and parameters

fractional biorthogonal partners in channel equalization and signal interpolation the concept of biorthogonal partners has been introduced recently by the authors the work presented here is an extension of some of these results to the case where the upsampling and downsampling ratios are not integers but rational numbers hence the name fractional biorthogonal partners the conditions for the existence of stable and of finite impulse response fir fractional biorthogonal partners are derived it is also shown that the fir solutions when they exist are not unique this property is further explored in one of the applications of fractional biorthogonal partners namely the fractionally spaced equalization in digital communications the goal is to construct zero forcing equalizers zfes that also combat the channel noise the performance of these equalizers is assessed through computer simulations another application considered is the all fir interpolation technique with the minimum amount of oversampling required in the input signal we also consider the extension of the least squares approximation problem to the setting of fractional biorthogonal partners

novel system inversion algorithm with application to oversampled perfect reconstruction filter banks we derive a novel algorithm for linear discrete time system inversion with decision delay and frequency weighted norm criterion complexity of the algorithm grows only linearly with the decision delay the algorithm also applies to certain singular cases where optimal inverses may be nonunique in that case the set of optimal inverses is parametrized a scilab implementation of the algorithm is provided applications in oversampled perfect reconstruction filter banks are given

theory of sparse coprime sensing in multiple dimensions coprime sampling and coprime sensor arrays have been introduced recently for the one dimensional 1 d case and applications in beamforming and direction finding discussed a pair of coprime arrays can be used to sample a wide sense stationary signal sparsely and then reconstruct the autocorrelation at a significantly denser set of points all applications based on autocorrelation e g spectrum and doa estimation benefit from this property it was also shown in the past that coprimality can be exploited in the frequency domain by using a pair of coprime dft filter banks to produce the effect of a much denser tiling in the frequency domain compared to what the two filter banks can individually achieve this paper extends these ideas to multiple dimensions in the 1 d case the samples or sensors lie on a pair of uniform grids whereas in the multidimensional case they lie on a pair of multidimensional lattices not necessarily rectangular this makes the developments mathematically more intricate first several properties of coarrays of lattices are derived it is shown how one can get dense coarrays from sparse arrays on non rectangular lattices this requires that the lattice generating matrices m and n be commuting and coprime to be defined multidimensional dft filter banks for applications such as beamforming with commuting coprime lattice arrays are then described and it is shown that a very dense tiling of the frequency plane can be obtained from the two sparse lattice arrays a particular family of commuting coprime matrices called adjugate pairs are considered in some detail and shown to have attractive properties a brief review of the 1 d case is included at the beginning for convenience

a general approach to coprime pairs of matrices based on minors in signal processing coprime pairs of integer matrices have an important role in the study of multidimensional multirate systems and in multidimensional array processing in this paper a general theorem for coprimality of pairs of matrices is first presented based on the minors of an associated composite matrix first a necessary and sufficient set of conditions for coprimality is presented based on minors this result applies to matrices of any size several useful consequences of the result are discussed and it is first applied to the 2 2 case including special cases such as toeplitz matrices commuting toeplitz matrices circulants and skew circulants the condition under which a 2 2 integer matrix and its ajdugate are coprime is also derived the minor based result is then applied extensively for the 3 3 case to derive new coprimality conditions for several matrix pairs in particular adjugates of matrices are considered in detail because adjugates always commute with the parent matrices the so called generalized 3 3 circulant matrix and its adjugate are then analyzed in detail generalized circulants also form a commuting family special cases such as 3 3 circulant pairs and skew circulant pairs are also elaborated

theory of paraunitary filter banks over fields of characteristic two motivated by our wavelet framework for error control coding we proceed to develop an important family of wavelet transforms over finite fields paraunitary pu filter banks that are realizations of orthogonal wavelets by multirate filters are an important subclass of perfect reconstruction pr filter banks a parameterization of pu filter banks that covers all possible pu systems is very desirable in error control coding because it provides a framework for optimizing the free parameters to maximize coding performance this paper undertakes the problem of classifying all pu matrices with entries from a polynomial ring where the coefficients of the polynomials are taken from finite fields it constructs householder transformations that are used as elementary operations for the realization of all unitary matrices then it introduces elementary pu building blocks and a factorization technique that is specialized to obtain a complete realization for all pu filter banks over fields of characteristic two this is proved for the 2 spl times 2 case and conjectured for the m spl times m case where m spl ges 3 using these elementary building blocks we can construct all pu filter banks over fields of characteristic two these filter banks can be used to implement transforms which in turn provide a powerful new perspective on the problems of constructing and decoding arbitrary rate error correcting codes

blind minimax estimation we consider the linear regression problem of estimating an unknown deterministic parameter vector based on measurements corrupted by colored gaussian noise we present and analyze blind minimax estimators bmes which consist of a bounded parameter set minimax estimator whose parameter set is itself estimated from measurements thus our approach does not require any prior assumption or knowledge and the proposed estimator can be applied to any linear regression problem we demonstrate analytically that the bmes strictly dominate the least squares ls estimator i e they achieve lower mean squared error mse for any value of the parameter vector both stein s estimator and its positive part correction can be derived within the blind minimax framework furthermore our approach can be readily extended to a wider class of estimation problems than stein s estimator which is defined only for white noise and nontransformed measurements we show through simulations that the bmes generally outperform previous extensions of stein s technique

on the non existence of unbiased estimators in constrained estimation problems we address the problem of existence of unbiased constrained parameter estimators we show that if the constrained set of parameters is compact and the hypothesized distributions are absolutely continuous with respect to one another then there exists no unbiased estimator weaker conditions for the absence of unbiased constrained estimators are also specified we provide several examples which demonstrate the utility of these conditions

discrete wavelet transform based on cyclic convolutions a filterbank in which cyclic convolutions are used in place of linear convolutions will be referred to as a cyclic convolution filterbank ccfb a dyadic tree structured ccfb can be used to perform a discrete wavelet transform suitable for coding based on symmetric extension methods this paper derives two types of efficient implementation techniques for the tree structured ccfb one using complex arithmetic and one using only real arithmetic in addition the present paper analyzes in detail the perfect reconstruction pr condition for the two channel ccfb and shows that this condition is much less restrictive than that of usual two channel filterbanks when each of the two channel ccfbs constituting a tree structured ccfb is designed to be pr the whole system is pr a quadrature mirror filter qmf ccfb having the pr property is demonstrated to be easily designed using a standard filter design subroutine in contrast designing a linear phase pr fir qmf bank that has good frequency response is not possible when filters are realized by linear convolutions

covering codes with improved density we prove a general recursive inequality concerning spl mu sup r the asymptotic least density of the best binary covering codes of radius r in particular this inequality implies that spl mu sup r spl les e spl middot rlogr logr loglogr 2 which significantly improves the best known density 2 sup r r sup r r 1 r our inequality also holds for covering codes over arbitrary alphabets

joint channel and carrier offset estimation in cdma communications this paper addresses the problem of cdma multiuser detection in the presence of unknown multipath channels and residual carriers an analytic algorithm that provides closed form channel and carrier offset estimates is proposed the algorithm first converts the multiuser estimation problem into single user problems and then analytically solves the resulting nonlinear multivariate optimization problems using a polynomial matrix projection property the performance of the proposed algorithm is investigated through first order perturbation analysis we also calculate the cramer rao bound crb to illustrate the efficiency of the new algorithm the analyzes are supported by computer simulations

factorability of lossless time varying filters and filter banks we study the factorability of linear time varying ltv lossless filters and filter banks we give a complete characterization of all degree one lossless ltv systems and show that all degree one lossless systems can be decomposed into a time dependent unitary matrix followed by a lossless dyadic based ltv system the lossless dyadic based system has several properties that make it useful in the factorization of lossless ltv systems the traditional lapped orthogonal transform lot is also generalized to the ltv case we identify two classes of tvlots namely the invertible inverse lossless iil and noninvertible inverse lossless nil tvlots the minimum number of delays required to implement a tvlot is shown to be a nondecreasing function of time and it is a constant if and only if the tvlot is iil we also show that all iil tvlots can be factorized uniquely into the proposed degree one lossless building block the factorization is minimal in terms of the delay elements for nil tvlots there are factorable and unfactorable examples both necessary and sufficient conditions for the factorability of lossless ltv systems are given we also introduce the concept of strong eternal reachability ser and strong eternal observability seo of ltv systems the ser and seo of an implementation of ltv systems imply the minimality of the structure using these concepts we are able to show that the cascade structure for a factorable iil ltv system is minimal that implies that if a iil ltv system is factorable in terms of the lossless dyadic based building blocks the factorization is minimal in terms of delays as well as the number of building blocks we also prove the bibo stability of the ltv normalized iir lattice

the role of integer matrices in multidimensional multirate systems various theoretical issues in multidimensional m d multirate signal processing are formulated and solved in the problems considered the decimation matrix and the expansion matrix are nondiagonal so that extensions of 1 d results are nontrivial the m d polyphase implementation technique for rational sampling rate alterations the perfect reconstruction properties for the m d delay chain systems and the periodicity matrices of decimated m d signals both deterministic and statistical are treated the discussions are based on several key properties of integer matrices including greatest common divisors and least common multiples these properties are reviewed