the cramer rao bound for estimating a sparse parameter vector the goal of this contribution is to characterize the best achievable mean squared error mse in estimating a sparse deterministic parameter from measurements corrupted by gaussian noise to this end an appropriate definition of bias in the sparse setting is developed and the constrained cramer rao bound crb is obtained this bound is shown to equal the crb of an estimator with knowledge of the support set for almost all feasible parameter values consequently in the unbiased case our bound is identical to the mse of the oracle estimator combined with the fact that the crb is achieved at high signal to noise ratios signal to noise ratio snrs by the maximum likelihood technique our result provides a new interpretation for the common practice of using the oracle estimator as a gold standard against which practical approaches are compared

compressed sensing with basis mismatch performance bounds and sparse based estimator compressed sensing cs is a promising emerging domain that outperforms the classical limit of the shannon sampling theory if the measurement vector can be approximated as the linear combination of few basis vectors extracted from a redundant dictionary matrix unfortunately in realistic scenario the knowledge of this basis or equivalently of the entire dictionary is often uncertain i e corrupted by a basis mismatch bm error the consequence of the bm problem is that the estimation accuracy in terms of bayesian mean square error bmse of popular sparse based estimators collapses even if the support is perfectly estimated and in the high signal to noise ratio snr regime this saturation effect considerably limits the effective viability of these estimation schemes in the first part of this work the bayesian cramer rao bound bcrb is derived for cs model with unstructured bm we show that the bcrb foresees the saturation effect of the estimation accuracy of standard sparse based estimators as for instance the omp cosamp or the bp in addition we provide an approximation of this bmse threshold in the second part and in the context of the structured bm model a new estimation scheme called bias correction estimator bice is proposed and its statistical properties are studied the bice acts as a post processing estimation layer for any sparse based estimator and mitigates considerably the bm degradation finally the bice i is a blind algorithm i e is unaware of the uncorrupted dictionary matrix ii is generic since it can be associated to any sparse based estimator iii is fast i e the additional computational cost remains low and iv has good statistical properties to illustrate our results and propositions the bice is applied in the challenging context of the compressive sampling of non bandlimited impulsive signals

time delay estimation from low rate samples a union of subspaces approach time delay estimation arises in many applications in which a multipath medium has to be identified from pulses transmitted through the channel previous methods for time delay recovery either operate on the analog received signal or require sampling at the nyquist rate of the transmitted pulse in this paper we develop a unified approach to time delay estimation from low rate samples this problem can be formulated in the broader context of sampling over an infinite union of subspaces although sampling over unions of subspaces has been receiving growing interest previous results either focus on unions of finite dimensional subspaces or finite unions the framework we develop here leads to perfect recovery of the multipath delays from samples of the channel output at the lowest possible rate even in the presence of overlapping transmitted pulses and allows for a variety of different sampling methods the sampling rate depends only on the number of multipath components and the transmission rate but not on the bandwidth of the probing signal this result can be viewed as a sampling theorem over an infinite union of infinite dimensional subspaces by properly manipulating the low rate samples we show that the time delays can be recovered using the well known esprit algorithm combining results from sampling theory with those obtained in the context of direction of arrival estimation we develop sufficient conditions on the transmitted pulse and the sampling functions in order to ensure perfect recovery of the channel parameters at the minimal possible rate

maximum correntropy criterion for robust face recognition in this paper we present a sparse correntropy framework for computing robust sparse representations of face images for recognition compared with the state of the art l1norm based sparse representation classifier src which assumes that noise also has a sparse representation our sparse algorithm is developed based on the maximum correntropy criterion which is much more insensitive to outliers in order to develop a more tractable and practical approach we in particular impose nonnegativity constraint on the variables in the maximum correntropy criterion and develop a half quadratic optimization technique to approximately maximize the objective function in an alternating way so that the complex optimization problem is reduced to learning a sparse representation through a weighted linear least squares problem with nonnegativity constraint at each iteration our extensive experiments demonstrate that the proposed method is more robust and efficient in dealing with the occlusion and corruption problems in face recognition as compared to the related state of the art methods in particular it shows that the proposed method can improve both recognition accuracy and receiver operator characteristic roc curves while the computational cost is much lower than the src algorithms

collaborative multi sensor classification via sparsity based representation in this paper we propose a general collaborative sparse representation framework for multi sensor classification which takes into account the correlations as well as complementary information between heterogeneous sensors simultaneously while considering joint sparsity within each sensor s observations we also robustify our models to deal with the presence of sparse noise and low rank interference signals specifically we demonstrate that incorporating the noise or interference signal as a low rank component in our models is essential in a multi sensor classification problem when multiple co located sources sensors simultaneously record the same physical event we further extend our frameworks to kernelized models which rely on sparsely representing a test sample in terms of all the training samples in a feature space induced by a kernel function a fast and efficient algorithm based on alternative direction method is proposed where its convergence to an optimal solution is guaranteed extensive experiments are conducted on several real multi sensor data sets and results are compared with the conventional classifiers to verify the effectiveness of the proposed methods

cram acute text e r rao bound for constrained parameter estimation using lehmann unbiasedness the constrained cram acute text e r rao bound ccrb is a lower bound on the mean squared error mse of estimators that satisfy some unbiasedness conditions although the ccrb unbiasedness conditions are satisfied asymptotically by the constrained maximum likelihood cml estimator in the non asymptotic region these conditions are usually too strict and the commonly used estimators such as the cml estimator do not satisfy them therefore the ccrb may not be a lower bound on the mse matrix of such estimators in this paper we propose a new definition for unbiasedness under constraints denoted by c unbiasedness which is based on using lehmann unbiasedness with a weighted mse wmse risk and taking into account the parametric constraints in addition to c unbiasedness a cram acute text e r rao type bound on the wmse of c unbiased estimators denoted as lehmann unbiased ccrb lu ccrb is derived this bound is a scalar bound that depends on the chosen weighted combination of estimation errors it is shown that c unbiasedness is less restrictive than the ccrb unbiasedness conditions thus the set of estimators that satisfy the ccrb unbiasedness conditions is a subset of the set of c unbiased estimators and the proposed lu ccrb may be an informative lower bound in cases where the corresponding ccrb is not in the simulations we examine linear and nonlinear estimation problems under nonlinear constraints in which the cml estimator is shown to be c unbiased and the lu ccrb is an informative lower bound on the wmse while the corresponding ccrb on the wmse is not a lower bound and is not informative in the non asymptotic region

models of the water retention curve for soils with a fractal pore size distribution the relationship between water content and water potential for a soil is termed its water retention curve this basic hydraulic property is closely related to the soil pore size distribution for which it serves as a conventional method of measurement in this paper a general model of the water retention curve is derived for soils whose pore size distribution is fractal in the sense of the mandelbrot number size distribution this model which contains two adjustable parameters the fractal dimension and the upper limiting value of the fractal porosity is shown to include other fractal approaches to the water retention curve as special cases application of the general model to a number of published data sets covering a broad range of soil texture indicated that unique independent values of the two adjustable parameters may be difficult to obtain by statistical analysis of water retention data for a given soil discrimination among different fractal approaches thus will require water retention data of high density and precision

on the existence and the applications of modified equations for stochastic differential equations in this paper we describe a general framework for deriving modified equations for stochastic differential equations sdes with respect to weak convergence modified equations are derived for a variety of numerical methods such as the euler or the milstein method existence of higher order modified equations is also discussed in the case of linear sdes using the gaussianity of the underlying solutions we derive an sde which the numerical method solves exactly in the weak sense applications of modified equations in the numerical study of langevin equations is also discussed

correcting for unknown errors in sparse high dimensional function approximation we consider sparsity based techniques for the approximation of high dimensional functions from random pointwise evaluations to date almost all the works published in this field contain some a priori assumptions about the error corrupting the samples that are hard to verify in practice in this paper we instead focus on the scenario where the error is unknown we study the performance of four sparsity promoting optimization problems weighted quadratically constrained basis pursuit weighted lasso weighted square root lasso and weighted lad lasso from the theoretical perspective we prove uniform recovery guarantees for these decoders deriving recipes for the optimal choice of the respective tuning parameters on the numerical side we compare them in the pure function approximation case and in applications to uncertainty quantification of odes and pdes with random inputs our main conclusion is that the lesser known square root lasso is better suited for high dimensional approximation than the other procedures in the case of bounded noise since it avoids both theoretically and numerically the need for parameter tuning

the rkhs approach to minimum variance estimation revisited variance bounds sufficient statistics and exponential families the mathematical theory of reproducing kernel hilbert spaces rkhss provides powerful tools for minimum variance estimation mve problems here we extend the classical rkhs based analysis of mve in several directions we develop a geometric formulation of five known lower bounds on the estimator variance barankin bound cramer rao bound constrained cramer rao bound bhattacharyya bound and hammersley chapman robbins bound in terms of orthogonal projections onto a subspace of the rkhs associated with a given mve problem we show that under mild conditions the barankin bound the tightest possible lower bound on the estimator variance is a lower semicontinuous function of the parameter vector we also show that the rkhs associated with an mve problem remains unchanged if the observation is replaced by a sufficient statistic finally for mve problems conforming to an exponential family of distributions we derive novel closed form lower bounds on the estimator variance and show that a reduction of the parameter set leaves the minimum achievable variance unchanged

an algorithmic introduction to numerical simulation of stochastic differential equations a practical and accessible introduction to numerical methods for stochastic differential equations is given the reader is assumed to be familiar with euler s method for deterministic differential equations and to have at least an intuitive feel for the concept of a random variable however no knowledge of advanced probability theory or stochastic processes is assumed the article is built around 10 matlab programs and the topics covered include stochastic integration the euler maruyama method milstein s method strong and weak convergence linear stability and the stochastic chain rule

exponential bounds implying construction of compressed sensing matrices error correcting codes and neighborly polytopes by random sampling in counting faces of randomly projected polytopes when the projection radically lowers dimension the authors proved an asymptotic sampling theorem for sparse signals showing that n random measurements permit to reconstruct an n vector having k nonzeros provided n 2 k log n n 1 o 1 reconstruction uses 1 minimization they also proved an asymptotic rate theorem showing existence of real error correcting codes for messages of length n which can correct all possible k element error patterns using just n generalized checksum bits where n 2e k log n n 1 o 1 decoding uses 1 minimization both results require an asymptotic framework with n growing large for applications on the other hand we are concerned with specific triples k n n we exhibit triples k n n for which compressed sensing matrices and real error correcting codes surely exist and can be obtained with high probability by random sampling these derive from exponential bounds on the probability of drawing bad matrices the bounds give conditions effective at finite n and converging to the known sharp asymptotic conditions for large n compared to other finite n bounds known to us they are much stronger and much more explicit our bounds derive from asymptotics in counting faces of randomly projected polytopes when the projection radically lowers dimension counting the expected number of k dimensional faces of the randomly projected simplex tn 1 and cross polytope cn we develop here finite n bounds on the expected discrepancy between the number of k faces of the projected polytope aq and its generator q for q tn 1 and cn our bounds also imply existence of interesting geometric objects thus we exhibit triples k n n for which polytopes with 2n vertices can be centrally k neighborly

lower bounds on the mean squared error of low rank matrix reconstruction we investigate the behavior of the mean square error mse of low rank matrix reconstruction and its special case matrix completion we first derive the constrained cramer rao bound crb on the mse matrix of any locally unbiased estimator and then analyze the behavior of the constrained crb when a subset of entries of the underlying matrix is randomly observed we design an alternating minimization procedure to compute the maximum likelihood estimator mle for the low rank matrix and demonstrate through numerical simulations that the performance of the mle approaches the constrained crb when the signal to noise ratio is high applying a chapman robbins type barankin bound allows us to derive lower bounds on the worst case scalar mse we demonstrate that the worst case scalar mse is infinite even if the model is identifiable however the infinite scalar mse is achieved only on a set of low rank matrices with measure zero we discuss the implications of these bounds and compare them with the empirical performance of the matrix lasso estimator and the existing bounds in the literature

image denoising via sparse and redundant representations over learned dictionaries we address the image denoising problem where zero mean white and homogeneous gaussian additive noise is to be removed from a given image the approach taken is based on sparse and redundant representations over trained dictionaries using the k svd algorithm we obtain a dictionary that describes the image content effectively two training options are considered using the corrupted image itself or training on a corpus of high quality image database since the k svd is limited in handling small image patches we extend its deployment to arbitrary image sizes by defining a global image prior that forces sparsity over patches in every location in the image we show how such bayesian treatment leads to a simple and effective denoising algorithm this leads to a state of the art denoising performance equivalent and sometimes surpassing recently published leading alternative denoising methods

performance bounds and design criteria for estimating finite rate of innovation signals in this paper we consider the problem of estimating finite rate of innovation fri signals from noisy measurements and specifically analyze the interaction between fri techniques and the underlying sampling methods we first obtain a fundamental limit on the estimation accuracy attainable regardless of the sampling method next we provide a bound on the performance achievable using any specific sampling approach essential differences between the noisy and noise free cases arise from this analysis in particular we identify settings in which noise free recovery techniques deteriorate substantially under slight noise levels thus quantifying the numerical instability inherent in such methods this instability which is only present in some families of fri signals is shown to be related to a specific type of structure which can be characterized by viewing the signal model as a union of subspaces finally we develop a methodology for choosing the optimal sampling kernels for linear reconstruction based on a generalization of the karhunen loeve transform the results are illustrated for several types of time delay estimation problems

barankin type lower bound on multiple change point estimation we compute lower bounds on the mean square error of multiple change point estimation in this context the parameters are discrete and the cramer rao bound is not applicable consequently we focus on computing the barankin bound bb the greatest lower bound on the covariance of any unbiased estimator which is still valid for discrete parameters in particular we compute the multi parameter version of the hammersley chapman robbins which is a barankin type lower bound we first give the structure of the so called barankin information matrix bim and derive a simplified form of the bb we show that the particular case of two change points is fundamental to finding the inverse of this matrix several closed form expressions of the elements of bim are given for changes in the parameters of gaussian and poisson distributions the computation of the bb requires finding the supremum of a finite set of positive definite matrices with respect to the loewner partial ordering although each matrix in this set of candidates is a lower bound on the covariance matrix of the estimator the existence of a unique supremum w r t to this set i e the tightest bound might not be guaranteed to overcome this problem we compute a suitable minimal upper bound to this set given by the matrix associated with the loewner john ellipsoid of the set of hyper ellipsoids associated to the set of candidate lower bound matrices finally we present some numerical examples to compare the proposed approximated bb with the performance achieved by the maximum likelihood estimator

information and asymptotic efficiency in parametric nonparametric models asymptotic lower bounds for estimation of the parameters of models with both parametric and nonparametric components are given in the form of representation theorems for regular estimates and asymptotic minimax bounds the methods used involve i the notion of a hellinger differentiable root density where part of the differentiation is with respect to the nonparametric part of the model to obtain appropriate scores and ii calculation of the effective score for the real or vector finite dimensional parameter of interest as that component of the score function orthogonal to all nuisance parameter scores perhaps infinite dimensional the resulting asymptotic information for estimation of the parametric component of the model is just 4 times the squared l 2 norm of the effective score a corollary of these results is a simple necessary condition for adaptive estimation adaptation is possible only if the scores for the parameter of interest are orthogonal to the scores for the nuisance function or nonparametric part of the model examples considered include the one sample location model with and without symmetry mixture models the two sample shift model and cox s proportional hazards model

asymptotic achievability of the cramer rao bound for noisy compressive sampling we consider a model of the form y ax n where x isin cm is sparse with at most l nonzero coefficients in unknown locations y isin cn is the observation vector a isin cn times m is the measurement matrix and n isin cn is the gaussian noise we develop a cramer rao bound on the mean squared estimation error of the nonzero elements of x corresponding to the genie aided estimator gae which is provided with the locations of the nonzero elements of x intuitively the mean squared estimation error of any estimator without the knowledge of the locations of the nonzero elements of x is no less than that of the gae assuming that l n is fixed we establish the existence of an estimator that asymptotically achieves the cramer rao bound without any knowledge of the locations of the nonzero elements of x as n rarr infin for a a random gaussian matrix whose elements are drawn i i d according to n 0 1

a new derivation of constrained cramer rao bound via norm minimization the constrained cramer rao bound ccrb is widely used to evaluate the estimation performance for deterministic parameters with parametric constraints an alternative derivation of the ccrb is presented from the perspective of solving a norm minimization problem under a set of linear constraints the parametric constraints are embedded in the linear constraints by constructing a proper basis to represent the estimation error this derivation avoids sophisticated matrix manipulations and has a clear physical meaning that the parametric constraints cut down the bases used to represent the estimation error and hence reduce the minimum norm of the error and the bound can be achieved if and only if the error is in the subspace spanned by the bases the result is applicable to biased estimators and singular fisher information matrices

a decomposition theory for binary linear codes the decomposition theory of matroids initiated by paul seymour in the 1980s has had an enormous impact on research in matroid theory this theory when applied to matrices over the binary field yields a powerful decomposition theory for binary linear codes in this paper we give an overview of this code decomposition theory and discuss some of its implications in the context of the recently discovered formulation of maximum likelihood ml decoding of a binary linear code over a binary input discrete memoryless channel as a linear programming problem we translate matroid theoretic results of grotschel and truemper from the combinatorial optimization literature to give examples of nontrivial families of codes for which the ml decoding problem can be solved in time polynomial in the length of the code one such family is that consisting of codes for which the codeword polytope is identical to the koetter vontobel fundamental polytope derived from the entire dual code cperp however we also show that such families of codes are not good in a coding theoretic sense either their dimension or their minimum distance must grow sublinearly with code length as a consequence we have that decoding by linear programming when applied to good codes cannot avoid failing occasionally due to the presence of pseudocode words

expander codes using expander graphs we construct a new family of asymptotically good linear error correcting codes these codes have linear time sequential decoding algorithms and logarithmic time parallel decoding algorithms that use a linear number of processors we present both randomized and explicit constructions of these codes experimental results demonstrate the good performance of the randomly chosen codes

maximum likelihood estimation from sign measurements with sensing matrix perturbation the problem of estimating an unknown deterministic parameter vector from sign measurements with a perturbed sensing matrix is studied in this paper we analyze the best achievable mean square error mse performance by exploring the corresponding cramer rao lower bound crlb to estimate the parameter the maximum likelihood ml estimator is utilized and its consistency is proved we show that compared with the perturbed free setting the perturbation on the sensing matrix exacerbates the performance of the ml estimator in most cases however suitable perturbation may improve the performance in some special cases then we reformulate the original ml estimation problem as a convex optimization problem which can be solved efficiently furthermore theoretical analysis implies that the perturbation ignored estimation is a scaled version with the same direction of the ml estimation finally numerical simulations are performed to validate our theoretical analysis

numerical methods for second order stochastic differential equations we seek numerical methods for second order stochastic differential equations that reproduce the stationary density accurately for all values of damping a complete analysis is possible for scalar linear second order equations damped harmonic oscillators with additive noise where the statistics are gaussian and can be calculated exactly in the continuous time and discrete time cases a matrix equation is given for the stationary variances and correlation for methods using one gaussian random variable per timestep the only runge kutta method with a nonsingular tableau matrix that gives the exact steady state density for all values of damping is the implicit midpoint rule numerical experiments comparing the implicit midpoint rule with heun and leapfrog methods on nonlinear equations with additive or multiplicative noise produce behavior similar to the linear case

motion segmentation in the presence of outlying incomplete or corrupted trajectories in this paper we study the problem of segmenting tracked feature point trajectories of multiple moving objects in an image sequence using the affine camera model this problem can be cast as the problem of segmenting samples drawn from multiple linear subspaces in practice due to limitations of the tracker occlusions and the presence of nonrigid objects in the scene the obtained motion trajectories may contain grossly mistracked features missing entries or corrupted entries in this paper we develop a robust subspace separation scheme that deals with these practical issues in a unified mathematical framework our methods draw strong connections between lossy compression rank minimization and sparse representation we test our methods extensively on the hopkins155 motion segmentation database and other motion sequences with outliers and missing data we compare the performance of our methods to state of the art motion segmentation methods based on expectation maximization and spectral clustering for data without outliers or missing information the results of our methods are on par with the state of the art results and in many cases exceed them in addition our methods give surprisingly good performance in the presence of the three types of pathological trajectories mentioned above all code and results are publicly available at http perception csl uiuc edu coding motion

an interpretation of the moore penrose generalized inverse of a singular fisher information matrix it is proved that in a non bayesian parametric estimation problem if the fisher information matrix fim is singular unbiased estimators for the unknown parameter will not exist cramer rao bound crb a popular tool to lower bound the variances of unbiased estimators seems inapplicable in such situations in this correspondence we show that the moore penrose generalized inverse of a singular fim can be interpreted as the crb corresponding to the minimum variance among all choices of minimum constraint functions this result ensures the logical validity of applying the moore penrose generalized inverse of an fim as the covariance lower bound when the fim is singular furthermore the result can be applied as a performance bound on the joint design of constraint functions and unbiased estimators

modified energy for split step methods applied to the linear schrodinger equation we consider the linear schrodinger equation and its discretization by split step methods where the part corresponding to the laplace operator is approximated by the midpoint rule we show that the numerical solution coincides with the exact solution of a modified partial differential equation at each time step this shows the existence of a modified energy preserved by the numerical scheme this energy is close to the exact energy if the numerical solution is smooth as a consequence we give uniform regularity estimates for the numerical solution over arbitrarily long time

robust subspace discovery via relaxed rank minimization this letter examines the problem of robust subspace discovery from input data samples instances in the presence of overwhelming outliers and corruptions a typical example is the case where we are given a set of images each image contains for example a face at an unknown location of an unknown size our goal is to identify or detect the face in the image and simultaneously learn its model we employ a simple generative subspace model and propose a new formulation to simultaneously infer the label information and learn the model using low rank optimization solving this problem enables us to simultaneously identify the ownership of instances to the subspace and learn the corresponding subspace model we give an efficient and effective algorithm based on the alternating direction method of multipliers and provide extensive simulations and experiments to verify the effectiveness of our method the proposed scheme can also be used to tackle many related high dimensional combinatorial selection problems

robust wiener kolmogorov theory a minimax formulation is considered for the problem of designing robust linear causal estimators of linear functions of discrete time wide sense stationary signals when knowledge of the signal and or noise spectra is inexact the solution is given under mild regularity conditions in terms of a least favorable pair of spectra thus reducing the minimax problem to a direct maximization problem which in many cases can be solved easily it is noted that this design method leads in particular to robust n step predictors robust causal filters and robust n lag smoothers the method of design is illustrated by a thorough development of the special case of one step noiseless prediction further solutions are given explicitly for the problem of robust causal filtering of an uncertain signal in white noise and numerical examples are given for this case which illustrate the effectiveness of this design

postprocessed integrators for the high order integration of ergodic sdes the concept of effective order is a popular methodology in the deterministic literature for the construction of efficient and accurate integrators for differential equations over long times the idea is to enhance the accuracy of a numerical method by using an appropriate change of variables called the processor we show that this technique can be extended to the stochastic context for the construction of new high order integrators for the sampling of the invariant measure of ergodic systems the approach is illustrated with modifications of the stochastic theta method applied to brownian dynamics where postprocessors achieving order two are introduced numerical experiments including stiff ergodic systems illustrate the efficiency and versatility of the approach

the risk unbiased cramer rao bound for non bayesian multivariate parameter estimation how accurately can one estimate a deterministic parameter subject to other unknown deterministic model nuisance parameters the most popular answer to this question is given by the cramer rao bound crb the main assumption behind the derivation of the crb is local unbiased estimation of all model parameters the foundations of this paper rely on doubting this assumption generally in multivariate parameter estimation each parameter in its turn can be treated as a single parameter of interest whereas the other model parameters are treated as nuisance as their misknowledge interferes with the estimation of the parameter of interest this approach is utilized in this paper to provide a fresh look at deterministic parameter estimation a new cramer rao cr type bound is derived without assuming unbiased estimation of the nuisance parameters rather than that we apply lehmann s concept of unbiasedness for a risk that measures the distance between the estimator and the locally best unbiased estimator which assumes perfect knowledge of the nuisance parameters the proposed risk unbiased crb rucrb is proven to be asymptotically attainable by the maximum likelihood estimator while being tighter than the conventional crb furthermore simulations verify the asymptotic achievability of the rucrb by the maximum likelihood estimator for an array processing problem

improving multilevel monte carlo for stochastic differential equations with application to the langevin equation this paper applies several well known tricks from the numerical treatment of deterministic differential equations to improve the efficiency of the multilevel monte carlo mlmc method for stochastic differential equations sdes and especially the langevin equation we use modified equations analysis as an alternative to strong approximation theory for the integrator and we apply this to introduce mlmc for langevin type equations with integrators based on operator splitting we combine this with extrapolation and investigate the use of discrete random variables in place of the gaussian increments which is a well known technique for the weak approximation of sdes we show that for small noise problems discrete random variables can lead to an increase in efficiency of almost two orders of magnitude for practical levels of accuracy

numerical point of view on calculus for functions assuming finite infinite and infinitesimal values over finite infinite and infinitesimal domains abstract the goal of this paper consists of developing a new more physical and numerical in comparison with standard and non standard analysis approaches point of view on calculus with functions assuming infinite and infinitesimal values it uses recently introduced infinite and infinitesimal numbers being in accordance with the principle the part is less than the whole observed in the physical world around us these numbers have a strong practical advantage with respect to traditional approaches they are representable at a new kind of a computer the infinity computer able to work numerically with all of them an introduction to the theory of physical and mathematical continuity and differentiation including subdifferentials for functions assuming finite infinite and infinitesimal values over finite infinite and infinitesimal domains is developed in the paper this theory allows one to work with derivatives that can assume not only finite but infinite and infinitesimal values as well it is emphasized that the newly introduced notion of the physical continuity allows one to see the same mathematical object as a continuous or a discrete one in dependence on the wish of the researcher i e as it happens in the physical world where the same object can be viewed as a continuous or a discrete in dependence on the instrument of the observation used by the researcher connections between pure mathematical concepts and their computational realizations are continuously emphasized through the text numerous examples are given

long time accuracy of lie trotter splitting methods for langevin dynamics a new characterization of sufficient conditions for the lie trotter splitting to capture the numerical invariant measure of nonlinear ergodic langevin dynamics up to an arbitrary order is discussed our characterization relies on backward error analysis and needs weaker assumptions than assumed so far in the literature in particular neither high weak order of the splitting scheme nor symplecticity are necessary to achieve high order approximation of the invariant measure of the langevin dynamics numerical experiments confirm our theoretical findings

efficient algorithm for solving semi infinite programming problems and their applications to nonuniform filter bank designs an efficient algorithm for solving semi infinite programming problems is proposed in this paper the index set is constructed by adding only one of the most violated points in a refined set of grid points by applying this algorithm for solving the optimum nonuniform symmetric antisymmetric linear phase finite impulse response fir filter bank design problems the time required to obtain a globally optimal solution is much reduced compared with that of the previous proposed algorithm

long run accuracy of variational integrators in the stochastic context this paper presents a lie trotter splitting for inertial langevin equations geometric langevin algorithm and analyzes its long time statistical properties the splitting is defined as a composition of a variational integrator with an ornstein uhlenbeck flow assuming that the exact solution and the splitting are geometrically ergodic the paper proves the discrete invariant measure of the splitting approximates the invariant measure of inertial langevin equations to within the accuracy of the variational integrator in representing the hamiltonian in particular if the variational integrator admits no energy error then the method samples the invariant measure of inertial langevin equations without error numerical validation is provided using explicit variational integrators with first second and fourth order accuracy

projection design for statistical compressive sensing a tight frame based approach in this paper we develop a framework to design sensing matrices for compressive sensing applications that lead to good mean squared error mse performance subject to sensing cost constraints by capitalizing on the mse of the oracle estimator whose performance has been shown to act as a benchmark to the performance of standard sparse recovery algorithms we use the fact that a parseval tight frame is the closest design in the frobenius norm sense to the solution of a convex relaxation of the optimization problem that relates to the minimization of the mse of the oracleestimator with respect to the equivalent sensing matrix subject to sensing energy constraints based on this result we then propose two sensing matrix designs that exhibit two key properties the designs are closed form rather than iterative the designs exhibit superior performance in relation to other designs in the literature which is revealed by our numerical investigation in various scenarios with different sparse recovery algorithms including basis pursuit de noise bpdn the dantzig selector and orthogonal matching pursuit omp

stochastic homogenization for an energy conserving multi scale toy model of the atmosphere we study a hamiltonian toy model for a lagrangian fluid parcel in the semi geostrophic limit which exhibits slow and fast dynamics we first reinject unresolved fast dynamics into the deterministic equation through a stochastic parametrization that respects the conservation of the energy of the deterministic system in a second step we use stochastic singular perturbation theory to derive an effective reduced stochastic differential equation for the slow dynamics we verify the results in numerical simulations

lower bounds for parametric estimation with constraints a chapman robbins form of the barankin bound is used to derive a multiparameter cramer rao cr type lower bound on estimator error covariance when the parameter theta in r sup n is constrained to lie in a subset of the parameter space a simple form for the constrained cr bound is obtained when the constraint set theta sub c can be expressed as a smooth functional inequality constraint it is shown that the constrained cr bound is identical to the unconstrained cr bound at the regular points of theta sub c i e where no equality constraints are active on the other hand at those points theta in theta sub c where pure equality constraints are active the full rank fisher information matrix in the unconstrained cr bound must be replaced by a rank reduced fisher information matrix obtained as a projection of the full rank fisher matrix onto the tangent hyperplane of the full rank fisher matrix onto the tangent hyperplane of the constraint set at theta a necessary and sufficient condition involving the forms of the constraint and the likelihood function is given for the bound to be achievable and examples for which the bound is achieved are presented in addition to providing a useful generalization of the cr bound the results permit analysis of the gain in achievable mse performance due to the imposition of particular constraints on the parameter space without the need for a global reparameterization

parameter estimation from quantized observations in multiplicative noise environments the problem of distributed parameter estimation from binary quantized observations is studied when the unquantized observations are corrupted by combined multiplicative and additive gaussian noise these results are applicable to sensor networks where the sensors observe a parameter in combined additive and nonadditive noise and to a case where dispersed receivers are employed with analog communication over fading channels where the receivers employ binary quantization before noise free digital communications to a fusion center we first discuss the case in which all the quantizers use an identical threshold the parameter identifiability condition is given and surprisingly it is shown that unless the common threshold is chosen properly and the parameter lies in an open interval the parameter will not generally be identifiable in contrast to the additive noise case the best achievable mean square error mse performance is characterized by deriving the corresponding cramer rao lower bound crlb a closed form expression describing the corresponding maximum likelihood ml estimator is presented the stability of the performance of the ml estimators is improved when a nonidentical threshold strategy is utilized to estimate the unknown parameter the thresholds are designed by maximizing the minimum asymptotic relative efficiency are between quantized and unquantized ml estimators although the ml estimation problem is nonconvex it is shown that one can relax the optimization to make it convex the solution to the relaxed problem is used as an initial solution in a gradient algorithm to solve the original problem next the case where both the variances of the additive noise and multiplicative noise are unknown is studied the corresponding crlb is obtained and the ml estimation problem is transformed to a convex optimization which can be solved efficiently finally numerical simulations are performed to substantiate the theoretical analysis

lexicographic multi objective linear programming using grossone methodology theory and algorithm abstract numerous problems arising in engineering applications can have several objectives to be satisfied an important class of problems of this kind is lexicographic multi objective problems where the first objective is incomparably more important than the second one which in its turn is incomparably more important than the third one etc in this paper lexicographic multi objective linear programming lmolp problems are considered to tackle them traditional approaches either require solution of a series of linear programming problems or apply a scalarization of weighted multiple objectives into a single objective function the latter approach requires finding a set of weights that guarantees the equivalence of the original problem and the single objective one and the search of correct weights can be very time consuming in this work a new approach for solving lmolp problems using a recently introduced computational methodology allowing one to work numerically with infinities and infinitesimals is proposed it is shown that a smart application of infinitesimal weights allows one to construct a single objective problem avoiding the necessity to determine finite weights the equivalence between the original multi objective problem and the new single objective one is proved a simplex based algorithm working with finite and infinitesimal numbers is proposed implemented and discussed results of some numerical experiments are provided

block sparse recovery via convex optimization given a dictionary that consists of multiple blocks and a signal that lives in the range space of only a few blocks we study the problem of finding a block sparse representation of the signal i e a representation that uses the minimum number of blocks motivated by signal image processing and computer vision applications such as face recognition we consider the block sparse recovery problem in the case where the number of atoms in each block is arbitrary possibly much larger than the dimension of the underlying subspace to find a block sparse representation of a signal we propose two classes of nonconvex optimization programs which aim to minimize the number of nonzero coefficient blocks and the number of nonzero reconstructed vectors from the blocks respectively since both classes of problems are np hard we propose convex relaxations and derive conditions under which each class of the convex programs is equivalent to the original nonconvex formulation our conditions depend on the notions of mutual and cumulative subspace coherence of a dictionary which are natural generalizations of existing notions of mutual and cumulative coherence we evaluate the performance of the proposed convex programs through simulations as well as real experiments on face recognition we show that treating the face recognition problem as a block sparse recovery problem improves the state of the art results by 10 with only 25 of the training data

unbiased estimation of a sparse vector in white gaussian noise the problem studied in this paper is unbiased estimation of a sparse nonrandom vector corrupted by additive white gaussian noise it is shown that while there are infinitely many unbiased estimators for this problem none of them has uniformly minimum variance therefore the focus is placed on locally minimum variance unbiased lmvu estimators simple closed form lower and upper bounds on the variance of lmvu estimators or equivalently on the barankin bound bb are derived these bounds allow an estimation of the threshold region separating the low signal to noise ratio snr and high snr regimes and they indicate the asymptotic behavior of the bb at high snr in addition numerical lower and upper bounds are derived these are tighter than the closed form bounds and thus characterize the bb more accurately numerical studies compare the proposed characterizations of the bb with established biased estimation schemes and demonstrate that while unbiased estimators perform poorly at low snr they may perform better than biased estimators at high snr an interesting conclusion of this analysis is that the high snr behavior of the bb depends solely on the value of the smallest nonzero entry of the sparse vector and that this type of dependence is also exhibited by the performance of certain practical estimators

robust recovery of subspace structures by low rank representation in this paper we address the subspace clustering problem given a set of data samples vectors approximately drawn from a union of multiple subspaces our goal is to cluster the samples into their respective subspaces and remove possible outliers as well to this end we propose a novel objective function named low rank representation lrr which seeks the lowest rank representation among all the candidates that can represent the data samples as linear combinations of the bases in a given dictionary it is shown that the convex program associated with lrr solves the subspace clustering problem in the following sense when the data is clean we prove that lrr exactly recovers the true subspace structures when the data are contaminated by outliers we prove that under certain conditions lrr can exactly recover the row space of the original data and detect the outlier as well for data corrupted by arbitrary sparse errors lrr can also approximately recover the row space with theoretical guarantees since the subspace membership is provably determined by the row space these further imply that lrr can perform robust subspace clustering and error correction in an efficient and effective way

information theoretic image formation the emergent role of information theory in image formation is surveyed unlike the subject of information theoretic communication theory information theoretic imaging is far from a mature subject the possible role of information theory in problems of image formation is to provide a rigorous framework for defining the imaging problem for defining measures of optimality used to form estimates of images for addressing issues associated with the development of algorithms based on these optimality criteria and for quantifying the quality of the approximations the definition of the imaging problem consists of an appropriate model for the data and an appropriate model for the reproduction space which is the space within which image estimates take values each problem statement has an associated optimality criterion that measures the overall quality of an estimate the optimality criteria include maximizing the likelihood function and minimizing mean squared error for stochastic problems and minimizing squared error and discrimination for deterministic problems the development of algorithms is closely tied to the definition of the imaging problem and the associated optimality criterion algorithms with a strong information theoretic motivation are obtained by the method of expectation maximization related alternating minimization algorithms are discussed in quantifying the quality of approximations global and local measures are discussed global measures include the mean squared error and discrimination between an estimate and the truth and probability of error for recognition or hypothesis testing problems local measures include fisher information

nonlinear programming and grossone quadratic programing and the role of constraint qualifications abstract a novel and interesting approach to infinite and infinitesimal numbers was recently proposed in a series of papers and a book by sergeyev this novel numeral system is based on the use of a new infinite unit of measure the number grossone indicated by the numeral the number of elements of the set in of natural numbers based on the use of de cosmis and de leone 2012 have then proposed a new exact differentiable penalty function for constrained optimization problems in this paper these results are specialized to the important case of quadratic problems with linear constraints moreover the crucial role of constraint qualification conditions well know in constraint minimization literature is also discussed with reference to the new proposed penalty function

on intrinsic cramer rao bounds for riemannian submanifolds and quotient manifolds we study cramer rao bounds crb s for estimation problems on riemannian manifolds in s t smith covariance subspace and intrinsic cramer rao bounds ieee trans signal process vol 53 no 5 1610 1630 2005 the author gives intrinsic crb s in the form of matrix inequalities relating the covariance of estimators and the fisher information of estimation problems we focus on estimation problems whose parameter space bar cal p is a riemannian submanifold or a riemannian quotient manifold of a parent space cal p that is estimation problems on manifolds with either deterministic constraints or ambiguities the crb s in the aforementioned reference would be expressed w r t bases of the tangent spaces to bar cal p in some cases though it is more convenient to express covariance and fisher information w r t bases of the tangent spaces to cal p we give crb s w r t such bases expressed in terms of the geodesic distances on the parameter space the bounds are valid even for singular fisher information matrices in two examples we show how the crb s for synchronization problems including a type of sensor network localization problem differ in the presence or absence of anchors leading to bounds for estimation on either submanifolds or quotient manifolds with very different interpretations

high order numerical approximation of the invariant measure of ergodic sdes we introduce new sufficient conditions for a numerical method to approximate with high order of accuracy the invariant measure of an ergodic system of stochastic differential equations independently of the weak order of accuracy of the method we then present a systematic procedure based on the framework of modified differential equations for the construction of stochastic integrators that capture the invariant measure of a wide class of ergodic sdes brownian and langevin dynamics with an accuracy independent of the weak order of the underlying method numerical experiments confirm our theoretical findings

on the end performance metric estimator selection it is well known that appropriately biasing an estimator can potentially lead to a lower mean square error mse than the achievable mse within the class of unbiased estimators nevertheless the choice of an appropriate bias is generally unclear and only recently there have been attempts to systematize such a selection these systematic approaches aim at introducing mse bounds that are lower than the unbiased cramer rao bound crb for all values of the unknown parameters and at choosing biased estimators that beat the standard maximum likelihood ml and or least squares ls estimators in the finite sample case in this paper we take these approaches one step further and investigate the same problem from the aspect of an end performance metric different than the classical mse this study is motivated by recent advances in the area of system identification indicating that the optimal experiment design should be done by taking into account the end performance metric of interest and not by quantifying a quadratic distance of the unknown model from the true one

robust group lasso model and recoverability abstract this paper considers the problem of recovering a group sparse signal matrix y y 1 y l from sparsely corrupted measurements m a 1 y 1 a l y l s where a i s are known sensing matrices and s is an unknown sparse error matrix a robust group lasso rgl model is proposed to recover y and s through simultaneously minimizing the l 2 1 norm of y and the l 1 norm of s under the measurement constraints we prove that y and s can be exactly recovered from the rgl model with high probability for a very general class of a i s

usage of infinitesimals in the menger s sponge model of porosity abstract the present work concerns the calculation of the infinitesimal porosity by using the menger s sponge model this computation is based on the grossone theory considering the pore volume estimation for the menger s sponge and afterwards the classical definition of the porosity given by the ratio between the volume of voids and the total volume voids plus solid phase the aim is to investigate the different solutions given by the standard characterization of the porosity and the grossone theory without the direct estimation of the fractal dimension once the utility of this procedure had been clarified the focus moves to possible practical applications in which infinitesimal parts can play a fundamental role the discussion on this matter still remains open

cramer rao type bounds for sparse bayesian learning in this paper we derive hybrid bayesian and marginalized cramer rao lower bounds hcrb bcrb and mcrb for the single and multiple measurement vector sparse bayesian learning sbl problem of estimating compressible vectors and their prior distribution parameters we assume the unknown vector to be drawn from a compressible student t prior distribution we derive crbs that encompass the deterministic or random nature of the unknown parameters of the prior distribution and the regression noise variance we extend the mcrb to the case where the compressible vector is distributed according to a general compressible prior distribution of which the generalized pareto distribution is a special case we use the derived bounds to uncover the relationship between the compressibility and mean square error mse in the estimates further we illustrate the tightness and utility of the bounds through simulations by comparing them with the mse performance of two popular sbl based estimators we find that the mcrb is generally the tightest among the bounds derived and that the mse performance of the expectation maximization em algorithm coincides with the mcrb for the compressible vector we also illustrate the dependence of the mse performance of sbl based estimators on the compressibility of the vector for several values of the number of observations and at different signal powers

solving ordinary differential equations on the infinity computer by working with infinitesimals numerically there exists a huge number of numerical methods that iteratively construct approximations to the solution y x of an ordinary differential equation ode y x f x y starting from an initial value y 0 y x 0 and using a finite approximation step h that influences the accuracy of the obtained approximation in this paper a new framework for solving odes is presented for a new kind of a computer the infinity computer it has been patented and its working prototype exists the new computer is able to work numerically with finite infinite and infinitesimal numbers giving so the possibility to use different infinitesimals numerically and in particular to take advantage of infinitesimal values of h to show the potential of the new framework a number of results is established it is proved that the infinity computer is able to calculate derivatives of the solution y x and to reconstruct its taylor expansion of a desired order numerically without finding the respective derivatives analytically or symbolically by the successive derivation of the ode as it is usually done when the taylor method is applied methods using approximations of derivatives obtained thanks to infinitesimals are discussed and a technique for an automatic control of rounding errors is introduced numerical examples are given

high weak order methods for stochastic differential equations based on modified equations inspired by recent advances in the theory of modified differential equations we propose a new methodology for constructing numerical integrators with high weak order for the time integration of stochastic differential equations this approach is illustrated with the constructions of new methods of weak order two in particular semi implicit integrators well suited for stiff mean square stable stochastic problems and implicit integrators that exactly conserve all quadratic first integrals of a stochastic dynamical system numerical examples confirm the theoretical results and show the versatility of our methodology

calculating effective diffusivities in the limit of vanishing molecular diffusion in this paper we study the problem of the numerical calculation by monte carlo methods of the effective diffusivity for a particle moving in a periodic divergent free velocity field in the limit of vanishing molecular diffusion in this limit traditional numerical methods typically fail since they do not represent accurately the geometry of the underlying deterministic dynamics we propose a stochastic splitting method that takes into account the volume preserving property of the equations of motion in the absence of noise and when inertial effects can be neglected an extension of the method is then proposed for the cases where the noise has a non trivial time correlation structure and when inertial effects cannot be neglected the method of modified equations is used to explain failings of euler based methods the new stochastic geometric integrators are shown to outperform standard euler based integrators various asymptotic limits of physical interest are investigated by means of numerical experiments using the new integrators

the power of convex relaxation near optimal matrix completion this paper is concerned with the problem of recovering an unknown matrix from a small fraction of its entries this is known as the matrix completion problem and comes up in a great number of applications including the famous netflix prize and other similar questions in collaborative filtering in general accurate recovery of a matrix from a small number of entries is impossible but the knowledge that the unknown matrix has low rank radically changes this premise making the search for solutions meaningful this paper presents optimality results quantifying the minimum number of entries needed to recover a matrix of rank r exactly by any method whatsoever the information theoretic limit more importantly the paper shows that under certain incoherence assumptions on the singular vectors of the matrix recovery is possible by solving a convenient convex program as soon as the number of entries is on the order of the information theoretic limit up to logarithmic factors this convex program simply finds among all matrices consistent with the observed entries that with minimum nuclear norm as an example we show that on the order of nr log n samples are needed to recover a random n x n matrix of rank r by any method and to be sure nuclear norm minimization succeeds as soon as the number of entries is of the form nr polylog n

maximum likelihood estimation the cramer rao bound and the method of scoring with parameter constraints maximum likelihood ml estimation is a popular approach to solving many signal processing problems many of these problems cannot be solved analytically and so numerical techniques such as the method of scoring are applied however in many scenarios it is desirable to modify the ml problem with the inclusion of additional side information often this side information is in the form of parametric constraints which the ml estimate mle must now satisfy we unify the asymptotic constrained ml cml theory with the constrained cramer rao bound ccrb theory by showing the cml estimate cmle is asymptotically efficient with respect to the ccrb we also generalize the classical method of scoring using the ccrb to include the constraints satisfying the constraints after each iterate convergence properties and examples verify the usefulness of the constrained scoring approach as a particular example an alternative and more general cmle is developed for the complex parameter linear model with linear constraints a novel proof of the efficiency of this estimator is provided using the ccrb

fundamentals of statistical signal processing estimation theory minimum variance unbiased estimation cramer rao lower bound linear models general minimum variance unbiased estimation best linear unbiased estimators maximum likelihood estimation least squares method of moments the bayesian philosophy general bayesian estimators linear bayesian estimators kalman filters summary of estimators extension for complex data and parameters

choosing between open and closed loop experiments in linear system identification this correspondence shows that open loop experiments are optimal for a broad class of systems when the system input is constrained in addition we show that for a general class of systems when the output power is constrained closed loop experiments are optimal both results use a strong notion of optimality and use expressions for estimation accuracy which are nonasymptotic in model order but asymptotic in data length

multivariate regression with gross errors on manifold valued data we consider the topic of multivariate regression on manifold valued output that is for a multivariate observation its output response lies on a manifold moreover we propose a new regression model to deal with the presence of grossly corrupted manifold valued responses a bottleneck issue commonly encountered in practical scenarios our model first takes a correction step on the grossly corrupted responses via geodesic curves on the manifold then performs multivariate linear regression on the corrected data this results in a nonconvex and nonsmooth optimization problem on riemannian manifolds to this end we propose a dedicated approach named palmr by utilizing and extending the proximal alternating linearized minimization techniques for optimization problems on euclidean spaces theoretically we investigate its convergence property where it is shown to converge to a critical point under mild conditions empirically we test our model on both synthetic and real diffusion tensor imaging data and show that our model outperforms other multivariate regression models when manifold valued responses contain gross errors and is effective in identifying gross errors

exploration of the non asymptotic bias and variance of stochastic gradient langevin dynamics applying standard markov chain monte carlo mcmc algorithms to large data sets is computationally infeasible the recently proposed stochastic gradient langevin dynamics sgld method circumvents this problem in three ways it generates proposed moves using only a subset of the data it skips the metropolis hastings accept reject step and it uses sequences of decreasing step sizes in teh et al 2014 we provided the mathematical foundations for the decreasing step size sgld including consistency and a central limit theorem however in practice the sgld is run for a relatively small number of iterations and its step size is not decreased to zero the present article investigates the behaviour of the sgld with fixed step size in particular we characterise the asymptotic bias explicitly along with its dependence on the step size and the variance of the stochastic gradient on that basis a modified sgld which removes the asymptotic bias due to the variance of the stochastic gradients up to first order in the step size is derived moreover we are able to obtain bounds on the finite time bias variance and mean squared error mse the theory is illustrated with a gaussian toy model for which the bias and the mse for the estimation of moments can be obtained explicitly for this toy model we study the gain of the sgld over the standard euler method in the limit of large data sets

robust pca with partial subspace knowledge in recent work robust principal components analysis pca has been posed as a problem of recovering a low rank matrix bf l and a sparse matrix bf s from their sum bf m bf l bf s and a provably exact convex optimization solution called pcp has been proposed this work studies the following problem suppose that we have partial knowledge about the column space of the low rank matrix bf l can we use this information to improve the pcp solution i e allow recovery under weaker assumptions we propose here a simple but useful modification of the pcp idea called modified pcp that allows us to use this knowledge we derive its correctness result which shows that when the available subspace knowledge is accurate modified pcp indeed requires significantly weaker incoherence assumptions than pcp extensive simulations are also used to illustrate this comparisons with pcp and other existing work are shown for a stylized real application as well finally we explain how this problem naturally occurs in many applications involving time series data i e in what is called the online or recursive robust pca problem a corollary for this case is also given

acquiring linear subspaces for face recognition under variable lighting previous work has demonstrated that the image variation of many objects human faces in particular under variable lighting can be effectively modeled by low dimensional linear spaces even when there are multiple light sources and shadowing basis images spanning this space are usually obtained in one of three ways a large set of images of the object under different lighting conditions is acquired and principal component analysis pca is used to estimate a subspace alternatively synthetic images are rendered from a 3d model perhaps reconstructed from images under point sources and again pca is used to estimate a subspace finally images rendered from a 3d model under diffuse lighting based on spherical harmonics are directly used as basis images in this paper we show how to arrange physical lighting so that the acquired images of each object can be directly used as the basis vectors of a low dimensional linear space and that this subspace is close to those acquired by the other methods more specifically there exist configurations of k point light source directions with k typically ranging from 5 to 9 such that by taking k images of an object under these single sources the resulting subspace is an effective representation for recognition under a wide range of lighting conditions since the subspace is generated directly from real images potentially complex and or brittle intermediate steps such as 3d reconstruction can be completely avoided nor is it necessary to acquire large numbers of training images or to physically construct complex diffuse harmonic light fields we validate the use of subspaces constructed in this fashion within the context of face recognition

lp decoding corrects a constant fraction of errors we show that for low density parity check ldpc codes whose tanner graphs have sufficient expansion the linear programming lp decoder of feldman karger and wainwright can correct a constant fraction of errors a random graph will have sufficient expansion with high probability and recent work shows that such graphs can be constructed efficiently a key element of our method is the use of a dual witness a zero valued dual solution to the decoding linear program whose existence proves decoding success we show that as long as no more than a certain constant fraction of the bits are flipped by the channel we can find a dual witness this new method can be used for proving bounds on the performance of any lp decoder even in a probabilistic setting our result implies that the word error rate of the lp decoder decreases exponentially in the code length under the binary symmetric channel bsc this is the first such error bound for ldpc codes using an analysis based on pseudocodewords recent work by koetter and vontobel shows that lp decoding and min sum decoding of ldpc codes are closely related by the graph cover structure of their pseudocodewords in their terminology our result implies that that there exist families of ldpc codes where the minimum bsc pseudoweight grows linearly in the block length

recursive robust pca or recursive sparse recovery in large but structured noise this paper studies the recursive robust principal components analysis problem if the outlier is the signal of interest this problem can be interpreted as one of recursively recovering a time sequence of sparse vectors st in the presence of large but structured noise lt the structure that we assume on lt is that lt is dense and lies in a low dimensional subspace that is either fixed or changes slowly enough a key application where this problem occurs is in video surveillance where the goal is to separate a slowly changing background lt from moving foreground objects st on the fly to solve the above problem in recent work we introduced a novel solution called recursive projected cs reprocs in this paper we develop a simple modification of the original reprocs idea and analyze it this modification assumes knowledge of a subspace change model on the lt s under mild assumptions and a denseness assumption on the unestimated part of the subspace of lt at various times we show that with high probability the proposed approach can exactly recover the support set of st at all times and the reconstruction errors of both st and lt are upper bounded by a time invariant and small value in simulation experiments we observe that the last assumption holds as long as there is some support change of st every few frames

dense error correction via ell 1 minimization this paper studies the problem of recovering a sparse signal x n from highly corrupted linear measurements y ax e m where e is an unknown error vector whose nonzero entries may be unbounded motivated by an observation from face recognition in computer vision this paper proves that for highly correlated and possibly overcomplete dictionaries a any sufficiently sparse signal x can be recovered by solving an l1 minimization problem min x 1 e 1 subject to y ax e more precisely if the fraction of the support of the error e is bounded away from one and the support of a is a very small fraction of the dimension m then as m becomes large the above l1 minimization succeeds for all signals x and almost all sign and support patterns of e this result suggests that accurate recovery of sparse signals is possible and computationally feasible even with nearly 100 of the observations corrupted the proof relies on a careful characterization of the faces of a convex polytope spanned together by the standard crosspolytope and a set of independent identically distributed i i d gaussian vectors with nonzero mean and small variance dubbed the cross and bouquet cab model simulations and experiments corroborate the findings and suggest extensions to the result

blind minimax estimation we consider the linear regression problem of estimating an unknown deterministic parameter vector based on measurements corrupted by colored gaussian noise we present and analyze blind minimax estimators bmes which consist of a bounded parameter set minimax estimator whose parameter set is itself estimated from measurements thus our approach does not require any prior assumption or knowledge and the proposed estimator can be applied to any linear regression problem we demonstrate analytically that the bmes strictly dominate the least squares ls estimator i e they achieve lower mean squared error mse for any value of the parameter vector both stein s estimator and its positive part correction can be derived within the blind minimax framework furthermore our approach can be readily extended to a wider class of estimation problems than stein s estimator which is defined only for white noise and nontransformed measurements we show through simulations that the bmes generally outperform previous extensions of stein s technique

wideband spectrum sensing from compressed measurements using spectral prior information wideband spectrum sensing wss encompasses a collection of techniques intended to estimate or to decide over the occupancy parameters of a wide frequency band however broad bands require expensive acquisition systems thus motivating the use of compressive schemes in this context previous works in compressive wss have already realized that great compression rates can be achieved if only second order statistics are of interest in spectrum sensing in this paper we go a step further by exploiting spectral prior information that is typically available in practice in order to reduce the sampling rate even more the signal model assumes that the acquisition is done by means of an analog to information converter a2i the input signal is the linear combination of a number of signals whose second order statistics are known and the goal is to estimate decide over the coefficients of this combination the problem is thus a particular instance of the well known structured covariance estimation problem unfortunately the algorithms used in this area are extremely complex for inexpensive spectrum sensors so that alternative techniques need to be devised exploiting the fact that the basis matrices are toeplitz we use the asymptotic theory of circulant matrices to propose a dimensionality reduction technique that simplifies existing structured covariance estimation algorithms achieving a similar performance at a much lower computational cost

on the non existence of unbiased estimators in constrained estimation problems we address the problem of existence of unbiased constrained parameter estimators we show that if the constrained set of parameters is compact and the hypothesized distributions are absolutely continuous with respect to one another then there exists no unbiased estimator weaker conditions for the absence of unbiased constrained estimators are also specified we provide several examples which demonstrate the utility of these conditions

the sierpinski curve viewed by numerical computations with infinities and infinitesimals abstract the sierpinski curve is one of the most known space filling curves and one with the highest number of applications we present a recently proposed computational methodology based on the infinite quantity called grossone to investigate the behavior of two different constructions of the sierpinski curve we emphasize that adopting this point of view we have infinitely many sierpinski curves depending contrarily to traditional analysis on each specific starting configuration of particular interest are some power series expansions in the new infinitesimal quantities emerging from the study of the considered curves

error estimates on ergodic properties of discretized feynman kac semigroups we consider the numerical analysis of the time discretization of feynman kac semigroups associated with diffusion processes these semigroups naturally appear in several fields such as large deviation theory diffusion monte carlo or non linear filtering we present errors estimates a la talay tubaro on their invariant measures when the underlying continuous stochastic differential equation is discretized as well as on the leading eigenvalue of the generator of the dynamics which corresponds to the rate of creation of probability this provides criteria to construct efficient integration schemes of feynman kac dynamics as well as a mathematical justification of numerical results already observed in the diffusion monte carlo community our analysis is illustrated by numerical simulations

sparse conjoint analysis through maximum likelihood estimation conjoint analysis ca is a classical tool used in preference assessment where the objective is to estimate the utility function of an individual or a group of individuals based on expressed preference data an example is choice based ca for consumer profiling i e unveiling consumer utility functions based solely on choices between products a statistical model for choice based ca is investigated in this paper unlike recent classification based approaches a sparsity aware gaussian maximum likelihood ml formulation is proposed to estimate the model parameters drawing from related robust parsimonious modeling approaches the model uses sparsity constraints to account for outliers and to detect the salient features that influence decisions contributions include conditions for statistical identifiability derivation of the pertinent cramer rao lower bound crlb and ml consistency conditions for the proposed sparse nonlinear model the proposed ml approach lends itself naturally to l1 type convex relaxations which are well suited for distributed implementation based on the alternating direction method of multipliers admm a particular decomposition is advocated which bypasses the apparent need for outlier communication thus maintaining scalability the performance of the proposed ml approach is demonstrated by comparing against the associated crlb and prior state of the art using both synthetic and real data sets

new results on deterministic cramer rao bounds for real and complex parameters the cramer rao bounds crb is a lower bound of great interest for system analysis and design in the asymptotic region high signal to noise ratio snr and or large number of snapshots as it is simple to calculate and it is usually possible to obtain closed form expressions the first part of the paper is a generalization to complex parameters of the barankin rationale for deriving mse lower bounds that is the minimization of a norm under a set of linear constraints with the norm minimization approach the study of fisher information matrix fim singularity constrained crb and regularity conditions become straightforward corollaries of the derivation the second part provides new results useful for system analysis and design a general reparameterization inequality the equivalence between reparameterization and equality constraints and an explicit relationship between parameters unidentifiability and fim singularity

statistical am fm models extended kalman filter demodulation cramer rao bounds and speech analysis a stochastic dynamical system model for describing time signals that are jointly amplitude am and frequency fm modulated is presented the signal is assumed to be bandpass perhaps originating from a filter bank applied to a broadband signal and includes the constraint that the magnitude of the complex baseband signal is positive motivated by speech processing and the desire for narrowband modulating signals time is divided into frames and the modulating signals are smoothly interpolated across each frame the model allows a detailed characterization of the bandwidth of the modulating signals and the statistical character of the measurement noise an adaptive estimation algorithm based on extended kalman filtering ideas for extracting the modulating signals from the measured signal is described and demonstrated on both voiced and unvoiced speech signals the cramer rao bound on the performance of any estimator is computed