information theoretic determination of minimax rates of convergence we present some general results determining minimax bounds on statistical risk for density estimation based on certain information theoretic considerations these bounds depend only on metric entropy conditions and are used to identify the minimax rates of convergence

adaptive estimation over anisotropic functional classes via oracle approach we address the problem of adaptive minimax estimation in white gaussian noise model under lp loss 1 p 1 on the anisotropic nikolskii classes we present the es timation procedure based on a new data driven selection scheme from the family of kernel estimators with varying bandwidths for proposed estimator we establish so called lp norm oracle inequality and use it for deriving minimax adaptive results we prove the existence of rate adaptive estimators and fully characterize behavior of the minimax risk for different relationships between regularity parameters and norm indexes in definitions of the functional class and of the risk in particular some new asymptotics of the minimax risk are discovered including necessary and sufficient conditions for existencea uniformly consistent estimator we provide also with detailed overview of existing methods and results and formulate open problems in adaptive minimax estimation

optimal model selection for stationary data under various mixing conditions we build penalized least squares estimators of the marginal density of a stationary process using the slope algorithm and resampling penalties when the data are beta or tau mixing these estimators satisfy oracle inequalities with leading constant asymptotically equal to 1

maximum likelihood wavelet density estimation with applications to image and shape matching density estimation for observational data plays an integral role in a broad spectrum of applications e g statistical data analysis and information theoretic image registration of late wavelet based density estimators have gained in popularity due to their ability to approximate a large class of functions adapting well to difficult situations such as when densities exhibit abrupt changes the decision to work with wavelet density estimators brings along with it theoretical considerations e g non negativity integrability and empirical issues e g computation of basis coefficients that must be addressed in order to obtain a bona fide density in this paper we present a new method to accurately estimate a non negative density which directly addresses many of the problems in practical wavelet density estimation we cast the estimation procedure in a maximum likelihood framework which estimates the square root of the density allowing us to obtain the natural non negative density representation analysis of this method will bring to light a remarkable theoretical connection with the fisher information of the density and consequently lead to an efficient constrained optimization procedure to estimate the wavelet coefficients we illustrate the effectiveness of the algorithm by evaluating its performance on mutual information based image registration shape point set alignment and empirical comparisons to known densities the present method is also compared to fixed and variable bandwidth kernel density estimators

maximum likelihood estimation of a multi dimensional log concave density fn is shown to have smaller mean integrated squared error compared with kernel based methods even when we allow the use of a theoret ical optimal fixed bandwidth for the kernel estimator that would not be available in practice we also present a real data clustering example which shows that our methodology can be used in conjunction with the expectation maximization algorithm to fit finite mixtures of log concave densities

wavelet variance based estimation for composite stochastic processes this article presents a new estimation method for the parameters of a time series model we consider here composite gaussian processes that are the sum of independent gaussian processes which in turn explain an important aspect of the time series as is the case in engineering and natural sciences the proposed estimation method offers an alternative to classical estimation based on the likelihood that is straightforward to implement and often the only feasible estimation method with complex models the estimator furnishes results as the optimization of a criterion based on a standardized distance between the sample wavelet variances wv estimates and the model based wv indeed the wv provides a decomposition of the variance process through different scales so that they contain the information about different features of the stochastic model we derive the asymptotic properties of the proposed estimator for inference and perform a simulation study to compare our estimator to the mle and the lse with

frequentist optimality of bayesian wavelet shrinkage rules for gaussian and non gaussian noise the present paper investigates theoretical performance of various bayesian wavelet shrinkage rules in a nonparametric regression model with i i d errors which are not necessarily normally distributed the main purpose is comparison of various bayesian models in terms of their frequentist asymptotic optimality in sobolev and besov spaces we establish a relationship between hyperparameters verify that the majority of bayesian models studied so far achieve theoretical optimality state which bayesian models cannot achieve optimal convergence rate and explain why it happens

wavelet thresholding for non necessarily gaussian noise idealism for various types of noise exponential normal mixture compactly supported wavelet thresholding methods are studied problems linked to the existence of optimal thresholds are tackled and minimaxity properties of the methods also analyzed a coefficient dependent method for choosing thresholds is also briefly presented

density estimation and random variate generation using multilayer networks in this paper we consider two important topics density estimation and random variate generation we present a framework that is easily implemented using the familiar multilayer neural network first we develop two new methods for density estimation a stochastic method and a related deterministic method both methods are based on approximating the distribution function the density being obtained by differentiation in the second part of the paper we develop new random number generation methods our methods do not suffer from some of the restrictions of existing methods in that they can be used to generate numbers from any density provided that certain smoothness conditions are satisfied one of our methods is based on an observed inverse relationship between the density estimation process and random number generation we present two variants of this method a stochastic and a deterministic version we propose a second method that is based on a novel control formulation of the problem where a controller network is trained to shape a given density into the desired density we justify the use of all the methods that we propose by providing theoretical convergence results in particular we prove that the l sub spl infin convergence to the true density for both the density estimation and random variate generation techniques occurs at a rate o log log n n sup 1 spl epsi 2 where n is the number of data points and spl epsi can be made arbitrarily small for sufficiently smooth target densities this bound is very close to the optimally achievable convergence rate under similar smoothness conditions also for comparison the l sub 2 root mean square rms convergence rate of a positive kernel density estimator is o n sup 2 5 when the optimal kernel width is used we present numerical simulations to illustrate the performance of the proposed density estimation and random variate generation methods in addition we present an extended introduction and bibliography that serves as an overview and reference for the practitioner

on adaptive minimax density estimation on r d we address the problem of adaptive minimax density estimation on mathbb r d with mathbb l p loss on the anisotropic nikol skii classes we fully characterize behavior of the minimax risk for different relationships between regularity parameters and norm indexes in definitions of the functional class and of the risk in particular we show that there are four different regimes with respect to the behavior of the minimax risk we develop a single estimator which is nearly optimal in order over the complete scale of the anisotropic nikol skii classes our estimation procedure is based on a data driven selection of an estimator from a fixed family of kernel estimators

density estimation by wavelet thresholding density estimation is a commonly used test case for nonparametric estimation methods we explore the asymptotic properties of estimators based on thresholding of empirical wavelet coefficients minimax rates of convergence are studied over a large range of besov function classes b pq and for a range of global l p error measures 1 p p some form of nonlinearity is essential since the minimax linear estimators are suboptimal by polynomial powers of n a second approach using an approximation of a gaussian white noise model in a mallows metric is used to attain exactly optimal rates of convergence for quadratic error p 2

minimax rate adaptive estimation over continuous hyper parameters we study minimax rate adaptive estimation for density classes indexed by continuous hyper parameters the classes are assumed to be partially ordered in terms of inclusion relationship under a mild condition on the minimax risks we show that a minimax rate adaptive estimator can be constructed for the classes

interpolation methods for nonlinear wavelet regression with irregularly spaced design we introduce interpolation methods that enable nonlinear wavelet estimators to be employed with stochastic design or nondyadic regular design in problems of nonparametric regression this approach allows relatively rapid computation involving dyadic approximations to wavelet after interpolation techniques new types of interpolation are described enabling first order variance reduction at the expense of second order increases in bias the effect of interpolation on threshold choice is addressed and appropriate thresholds are suggested for error distributions with as few as four finite moments

new denoising scheme for magnetic resonance spectroscopy signals a new scheme for denoising magnetic resonance spectroscopy mrs signals is presented this scheme is based on projecting noisy mrs signals in different domains consecutively and performing noise filtering operations in these domains the domains are chosen such that the noise portion which is inseparable from the desired signal in one domain is separable in the other a set of stable linear time frequency sltf transforms with different resolutions was selected for these projections as an example scheme evaluation was performed using extensive mrs signals with various noise levels compared with one domain denoising it was observed that the proposed scheme gives superior results that compensate for the excess computational requirements the proposed scheme supersedes also the wavelet packet denoising schemes

adaptive bayesian estimation using a gaussian random field with inverse gamma bandwidth we consider nonparametric bayesian estimation inference using a rescaled smooth gaussian field as a prior for a multidimensional function the rescaling is achieved using a gamma variable and the procedure can be viewed as choosing an inverse gamma bandwidth the procedure is studied from a frequentist perspective in three statistical settings involving replicated observations density estimation regression and classification we prove that the resulting posterior distribution shrinks to the distribution that generates the data at a speed which is minimax optimal up to a logarithmic factor whatever the regularity level of the data generating distribution thus the hierachical bayesian procedure with a fixed prior is shown to be fully adaptive

general empirical bayes wavelet methods and exactly adaptive minimax estimation in many statistical problems stochastic signals can be represented as a sequence of noisy wavelet coefficients in this paper we develop general empirical bayes methods for the estimation of true signal our estimators approximate certain oracle separable rules and achieve adaptation to ideal risks and exact minimax risks in broad collections of classes of signals in particular our estimators are uniformly adaptive to the minimum risk of separable estimators and the exact minimax risks simultaneously in besov balls of all smoothness and shape indices and they are uniformly superefficient in convergence rates in all compact sets in besov spaces with a finite secondary shape parameter furthermore in classes nested between besov balls of the same smoothness index our estimators dominate threshold and james stein estimators within an infinitesimal fraction of the minimax risks more general block empirical bayes estimators are developed both white noise with drift and nonparametric regression are considered

the mean consistency of wavelet estimators for convolutions of the density functions abstract in practical applications people sometimes do not know whether the estimated function is smooth and it is reasonable to consider the consistency of an estimator furthermore the acquired data are usually contaminated by various random noises in this paper we develop the wavelet estimators for m fold convolutions of the unknown density functions and consider their l p 1 p consistency under noiseless and additive noise situations respectively finally simulation studies illustrate the good performances of our nonparametric wavelet estimators

wedgelets nearly minimax estimation of edges we study a simple horizon model for the problem of recovering an image from noisy data in this model the image has an edge with fi holder regularity adopting the viewpoint of computational harmonic analysis we develop an overcomplete collection of atoms called wedgelets dyadically organized indicator functions with a variety of locations scales and orientations the wedgelet representation provides nearly optimal representations of objects in the horizon model as measured by minimax description length we show how to rapidly compute a wedgelet approximation to noisy data by flnding a special edgelet decorated recursive partition which minimizes a complexity penalized sum of squares this estimate using su cient sub pixel resolution achieves nearly the minimax mean squared error in the horizon model in fact the method is adaptive in the sense that it achieves nearly the minimax risk for any value of the unknown degree of regularity of the horizon 1 fi 2 wedgelet analysis and de noising may be used successfully outside the horizon model we study images modelled as indicators of star shaped sets with smooth bound aries and show that complexity penalized wedgelet partitioning achieves nearly the minimax risk in that setting also

nonlinear black box models in system identification mathematical foundations we discuss several aspects of the mathematical foundations of the nonlinear black box identification problem we shall see that the quality of the identification procedure is always a result of a certain trade off between the expressive power of the model we try to identify the larger the number of parameters used to describe the model the more flexible is the approximation and the stochastic error which is proportional to the number of parameters a consequence of this trade off is the simple fact that a good approximation technique can be the basis of a good identification algorithm from this point of view we consider different approximation methods and pay special attention to spatially adaptive approximants we introduce wavelet and neuron approximations and show that they are spatially adaptive then we apply the acquired approximation experience to estimation problems finally we consider some implications of these theoretical developments for the practically implemented versions of the spatially adaptive algorithms

adaptive inference for the mean of a gaussian process in functional data summary we propose and analyse fully data driven methods for inference about the mean function of a gaussian process from a sample of independent trajectories of the process observed at random time points and corrupted by additive random error our methods are based on thresholded least squares estimators relative to an approximating function basis the variable threshold levels are determined from the data and the resulting estimates adapt to the unknown sparsity of the mean function relative to the approximating basis these results are obtained via novel oracle inequalities which are further used to derive the rates of convergence of our mean estimates in addition we construct confidence balls that adapt to the unknown regularity of the mean and covariance function of the stochastic process they are easy to compute since they do not require explicit estimation of the covariance operator of the process a simulation study shows that the new method performs very well in practice and is robust against large variations that may be introduced by the random error terms

poisson inverse problems in this paper we focus on nonparametric estimators in inverse problems for poisson processes involving the use of wavelet decompositions adopting an adaptive wavelet galerkin discretization we find that our method combines the well known theoretical advantages of wavelet vaguelette decompositions for inverse problems in terms of optimally adapting to the unknown smoothness of the solution together with the remarkably simple closed form expressions of galerkin inversion methods adapting the results of barron and sheu ann statist 19 1991 1347 1369 to the context of log intensity functions approximated by wavelet series with the use of the kullback leibler distance between two point processes we also present an asymptotic analysis of convergence rates that justifies our approach in order to shed some light on the theoretical results obtained and to examine the accuracy of our estimates in finite samples we illustrate our method by the analysis of some simulated examples

a multirate dsp model for estimation of discrete probability density functions the problem of estimating a probability density function pdf from measurements has been widely studied by many researchers even though much work has been done in the area of pdf estimation most of it was focused on the continuous case we propose a new model based approach for modeling and estimating discrete probability density functions or probability mass functions this approach is based on multirate signal processing theory and it has several advantages over the conventional histogram method we illustrate the pdf estimation procedure and analyze the statistical properties of the pdf estimates based on this model a novel scheme is introduced that can be used for estimating the pdf in the presence of noise furthermore the proposed ideas are extended to the more general case of estimating multivariate pdfs finally we also consider practical issues such as optimizing the coefficients of a digital filter which is an integral part of the model this allows us to apply the proposed model to solve real world problems simulation results are given where appropriate in order to demonstrate the ideas

universal algorithms for learning theory part i piecewise constant functions this paper is concerned with the construction and analysis of a universal estimator for the regression problem in supervised learning universal means that the estimator does not depend on any a priori assumptions about the regression function to be estimated the universal estimator studied in this paper consists of a least square fitting procedure using piecewise constant functions on a partition which depends adaptively on the data the partition is generated by a splitting procedure which differs from those used in cart algorithms it is proven that this estimator performs at the optimal convergence rate for a wide class of priors on the regression function namely as will be made precise in the text if the regression function is in any one of a certain class of approximation spaces or smoothness spaces of order not exceeding one a limitation resulting because the estimator uses piecewise constants measured relative to the marginal measure then the estimator converges to the regression function in the least squares sense with an optimal rate of convergence in terms of the number of samples the estimator is also numerically feasible and can be implemented on line

asymptotic equivalence theory for nonparametric regression with random design this paper establishes the global asymptotic equivalence between the nonparametric regression with random design and the white noise under sharp smoothness conditions on an unknown regression or drift function the asymptotic equivalence is established by constructing explicit equivalence mappings between the nonparametric regression and the whitenoise experiments which provide synthetic observations and synthetic asymptotic solutions from any one of the two experiments with asymptotic properties identical to the true observations and given asymptotic solutions from the other the impact of such asymptotic equivalence results is that an investigation in one nonparametric problem automatically yields asymptotically analogous results in all other asymptotically equivalent nonparametric problems 1 introduction the purpose of this paper is to establish the global asymptotic equivalence between the nonparametric regression with random design and the white noise under sharp smoothness conditions on an unknown regression or drift function we establish this asymptotic equivalence by constructing explicit equivalence mappings between the nonparametric regression and the white noise problems as in brown and low 1996 for their asymptotic equivalence results the equivalence mapping from the nonparametric regression to the white noise provides synthetic observations of the white noise from the nonparametric regression such that the distributions of the synthetic observations are asymptotically equivalent to those of the true observations of the white noise for any asymptotic solution to a white noise problem the application of the solution to the synthetic observations provides an asymptotic solution to the corresponding nonparametric regression problem with identical asymptotic properties likewise the equivalence mapping from the white noise produces synthetic observations of the nonparametric regression problem and synthetic asymptotic solutions to white noise problems based on those of the corresponding nonparametric regression problems the impact of such asymptotic equivalence results is that an investigation in one nonparametric problem automatically yields asymptotically analogous results in

multiwavelet density estimation accurate density estimation methodologies play an integral role in a variety of scientific disciplines with applications including simulation models decision support tools and exploratory data analysis in the past histograms and kernel density estimators have been the predominant tools of choice primarily due to their ease of use and mathematical simplicity more recently the use of wavelets for density estimation has gained in popularity due to their ability to approximate a large class of functions including those with localized abrupt variations however a well known attribute of wavelet bases is that they cannot be simultaneously symmetric orthogonal and compactly supported multiwavelets more general vector valued constructions of wavelets overcome this disadvantage making them natural choices for estimating density functions many of which exhibit local symmetries around features such as a mode we extend the methodology of wavelet density estimation to use multiwavelet bases and illustrate several empirical examples of multiwavelet density estimation

quasi linear wavelet estimation abstract the main paradigm of the modern wavelet theory of spatial adaptation formulated by donoho and johnstone is that there is a divergence between the linear minimax adaptation theory and the heuristic guiding algorithm development that leads to the necessity of using strongly nonlinear adaptive thresholded methods on the other hand it is well known that linear adaptive estimates are the best whenever an estimated function is smooth is it possible to suggest a quasi linear wavelet estimate by adding to a linear adaptive estimate a minimal number of nonlinear terms on finest scales that offers advantages of linear adaptive estimates and at the same time matches asymptotic properties of strongly nonlinear procedures like the benchmark sureshrink the answer is yes and we discuss quasi linear estimation both theoretically and via a monte carlo study in particular i show that asymptotically a quasi linear procedure not only matches properties of sureshrink over the besov scale but also allows

multiscale poisson intensity and density estimation the nonparametric poisson intensity and density estimation methods studied in this paper offer near minimax convergence rates for broad classes of densities and intensities with arbitrary levels of smoothness the methods and theory presented here share many of the desirable features associated with wavelet based estimators computational speed spatial adaptivity and the capability of detecting discontinuities and singularities with high resolution unlike traditional wavelet based approaches which impose an upper bound on the degree of smoothness to which they can adapt the estimators studied here guarantee nonnegativity and do not require any a priori knowledge of the underlying signal s smoothness to guarantee near optimal performance at the heart of these methods lie multiscale decompositions based on free knot free degree piecewise polynomial functions and penalized likelihood estimation the degrees as well as the locations of the polynomial pieces can be adapted to the observed data resulting in near minimax optimal convergence rates for piecewise analytic signals in particular the error of this estimator converges at nearly the parametric rate these methods can be further refined in two dimensions and it is demonstrated that platelet based estimators in two dimensions exhibit similar near optimal error convergence rates for images consisting of smooth surfaces separated by smooth boundaries

nonquadratic estimators of a quadratic functional estimation of a quadratic functional over parameter spaces that are not quadratically convex is considered it is shown in contrast to the theory for quadratically convex parameter spaces that optimal quadratic rules are often rate suboptimal in such cases minimax rate optimal procedures are constructed based on local thresholding these nonquadratic procedures are sometimes fully efficient even when optimal quadratic rules have slow rates of convergence moreover it is shown that when estimating a quadratic functional nonquadratic procedures may exhibit different elbow phenomena than quadratic procedures

deterministic learning for maximum likelihood estimation through neural networks in this paper a general method for the numerical solution of maximum likelihood estimation mle problems is presented it adopts the deterministic learning dl approach to find close approximations to ml estimator functions for the unknown parameters of any given density the method relies on the choice of a proper neural network and on the deterministic generation of samples of observations of the likelihood function thus avoiding the problem of generating samples with the unknown density under mild assumptions consistency and convergence with favorable rates to the true ml estimator function can be proved simulation results are provided to show the good behavior of the algorithm compared to the corresponding exact solutions

the discriminant elastic graph matching algorithm applied to frontal face verification in this paper a generalized framework for face verification is proposed employing discriminant techniques in all phases of elastic graph matching the proposed algorithm is called discriminant elastic graph matching degm algorithm in the first step of the proposed method degm discriminant techniques at the node feature vectors are used for feature selection in the sequel the two local similarity values i e the similarity measure for the projected node feature vector and the node deformation are combined in a discriminant manner in order to form the new local similarity measure moreover the new local similarity values at the nodes of the elastic graph are weighted by coefficients that are derived as well from discriminant analysis in order to form a total similarity measure between faces the proposed method exploits the individuality of the human face and the discriminant information of elastic graph matching in order to improve the verification performance of elastic graph matching we have applied the proposed scheme to a modified morphological elastic graph matching algorithm all experiments have been conducted in the xm2vts database resulting in very low error rates for the test sets

risk bounds for model selection via penalization performance bounds for criteria for model selection are devel oped using recent theory for sieves the model selection criteria are based on an empirical loss or contrast function with an added penalty term moti vated by empirical process theory and roughly proportional to the number of parameters needed to describe the model divided by the number of ob servations most of our examples involve density or regression estimation settings and we focus on the problem of estimating the unknown density or regression function we show that the quadratic risk of the minimum penal ized empirical contrast estimatoris bounded by an index of the accuracy of the sieve this accuracy index quanties the trade off among the candidate models between the approximation error and parameter dimension relative to sample size if we choose a list of models which exhibit good approximation prop erties with respect to different classes of smoothness the estimator can be simultaneously minimax rate optimal in each of those classes this is what is usually called adaptation the type of classes of smoothness in which one gets adaptation depends heavily on the list of models if too many models are involved in order to get accurate approximation of many wide classes of functions simultaneously it may happen that the estimator is only approx

minimax risk over hyperrectangles and implications consider estimating the mean of a standard gaussian shift when that mean is known to lie in an orthosymmetric quadratically convex set in l 2 such sets include ellipsoids hyperrectangles and l p bodies with p 2 the minimax risk among linear estimates is within 25 of the minimax risk among all estimates the minimax risk among truncated series estimates is within a factor 4 44 of the minimax risk this implies that the difficulty of estimation a statistical quantity is measured fairly precisely by the n widths a geometric quantity if the set is not quadratically convex as in the case of l p bodies with p 2 things change appreciably minimax linear estimators may be out performed arbitrarily by nonlinear estimates the ordinary kolmogorov n widths still determine the difficulty of linear estimation but the difficulty of nonlinear estimation is tied to the inner bernstein n widths which can be far smaller essential use is made of a new heuristic that the difficulty of the hardest rectangular subproblem is equal to the difficulty of the full problem

frsde fast reduced set density estimator using minimal enclosing ball approximation reduced set density estimator rsde is an important technique that can be used to replace the classical parzen window estimator pw for saving the computational cost though rsde demonstrates a nicer performance in the density accuracy and the computational time compared with several existing methods it still faces the critical challenge for practical applications because of its high time complexity no less than o n 2 and space complexity o n 2 in training the model weighting coefficients on large data sets in order to overcome this shortcoming a fast reduced set density estimator algorithm frsde is proposed in this study first the relationship between rsde and the minimal enclosing ball problems meb in computational geometry is revealed then the finding that rsde is equivalent to a special meb problem is derived with this finding the fast core set based meb approximation algorithm is introduced to develop the proposed algorithm frsde compared with rsde frsde has the following distinctive advantage it can guarantee that the upper bound of the time complexity is linear with the size n of a large data set and the upper bound of the space complexity is independent of n our experimental results show that the proposed frsde has a competitive performance in the density accuracy and an overwhelming advantage over rsde for large data sets in the data condensation rate and the training time for the weighting coefficients

iterative rate distortion optimization of h 264 with constant bit rate constraint in this paper we apply the primal dual decomposition and subgradient projection methods to solve the rate distortion optimization problem with the constant bit rate constraint the primal decomposition method enables spatial or temporal prediction dependency within a group of picture gop to be processed in the master primal problem as a result we can apply the dual decomposition to minimize independently the lagrangian cost of all the mbs using the reference software model of h 264 furthermore the optimal lagrange multiplier is iteratively derived from the solution of the dual problem as an example we derive the optimal bit allocation condition with the consideration of temporal prediction dependency among the pictures experimental results show that the proposed method achieves better performance than the reference software model of h 264 with rate control

densities spectral densities and modality this paper considers the problem of specifying a simple approximating density function for a given data set x 1 x n simplicity is measured by the number of modes but several different definitions of approximation are introduced the taut string method is used to control the numbers of modes and to produce candidate approximating densities refinements are introduced that improve the local adaptivity of the procedures and the method is extended to spectral densities

adaptive confidence interval for pointwise curve estimation we present a procedure associated with nonlinear wavelet methods that provides adaptive confidence intervals around f x 0 in either a white noise model or a regression setting a suitable modification in the truncation rule for wavelets allows construction of confidence intervals that achieve optimal coverage accuracy up to a logarithmic factor the procedure does not require knowledge of the regularity of the unknown function f it is also efficient for functions with a low degree of regularity

temporally dependent rate distortion optimization for low delay hierarchical video coding low delay hierarchical coding structure ld hcs as one of the most important components in the latest high efficiency video coding hevc standard greatly improves coding performance it groups consecutive p b frames into different layers and encodes them with different quantization parameters qps and reference mechanisms in such a way that temporal dependency among frames can be exploited however due to varying characteristics of video contents temporal dependency among coding units differs significantly from each other in the same or different layers while a fixed ld hcs scheme cannot take full advantage of the dependency leading to a substantial loss in coding performance this paper addresses the temporally dependent rate distortion optimization rdo problem by attempting to exploit varying temporal dependency of different units first the temporal relationship of different frames under the ld hcs is examined and hierarchical temporal propagation chains are constructed to represent the temporal dependency among coding units in different frames then a hierarchical temporally dependent rdo scheme is developed specifically for the ld hcs based on a source distortion propagation model experimental results show that our proposed scheme can achieve 2 5 and 2 3 bd rate gain in average compared with the hevc codec under the same configuration of p and b frames respectively with a negligible increase in encoding time furthermore coupled with qp adaption our proposed method can achieve higher coding gains e g with multi qp optimization about 5 4 and 5 0 bd rate saving in average over the hevc codec under the same setting of p and b frames respectively

nonlinear approximation and the space bv r2 given a function 2 0 q 0 l 2 and a real number t 0 let u f t infg bv g 11 ll vgig where the infimum is taken over all functions g g bv of bounded variation on this and related extremal problems arise in several areas of mathematics such as interpolation of operators and statistical estimation as well as in digital image processing techniques for finding minimizers g for u f t based on variational calculus and nonlinear partial differential equations have been put forward by several authors dms ro ms cl the main disadvantage of these approaches is that they are numerically intensive on the other hand it is well known that more elementary methods based on wavelet shrinkage solve related extremal problems for example the above problem with bv replaced by the besov space b l i see e g cdll however since bv has no simple description in terms of wavelet coefficients it is not clear that minimizers for u f t can be realized in this way we shall show in this paper that simple methods based on haar thresholding provide near minimizers for u f t our analysis of this extremal problem brings forward many interesting relations between haar decompositions and the space bv 1 introduction nonlinear approximation has recently played an impor tant role in several problems of image processing including compression noise removal and feature extraction we have in mind techniques such as wavelet compression djl wavelet shrinkage or thresholding djkp1 wavelet packets cw and greedy algorithms mz dt there has also been an impressive contribution of techniques based on variational calculus and nonlinear partial dif ferential equations see e g dms ro ms cl especially to the problems of noise removal and image segmentation the common point between these two approaches is their ability to adapt to the composite nature of images edge tex tures and smooth regions should be treated adaptively a requirement which is certainly not fulfilled by the classical linear filtering techniques

model selection for density estimation with mathbb l _2 loss we consider here estimation of an unknown probability density s belonging to mathbb l 2 mu where mu is a probability measure we have at hand n i i d observations with density s and use the squared mathbb l 2 norm as our loss function the purpose of this paper is to provide an abstract but completely general method for estimating s by model selection allowing to handle arbitrary families of finite dimensional possibly non linear models and any s in mathbb l 2 mu we shall in particular consider the cases of unbounded densities and bounded densities with unknown mathbb l infty norm and investigate how the mathbb l infty norm of s may influence the risk we shall also provide applications to adaptive estimation and aggregation of preliminary estimators one major technical tool of our approach is a proof of the existence of suitable tests between mathbb l 2 balls with centers belonging to mathbb l infty although of a purely theoretical nature our method leads to results that cannot presently be reached by more concrete ones

bandwidth selection in kernel density estimation oracle inequalities and adaptive minimax optimality we address the problem of density estimation with mathbb l s loss by selection of kernel estimators we develop a selection procedure and derive corresponding mathbb l s risk oracle inequalities it is shown that the proposed selection rule leads to the estimator being minimax adaptive over a scale of the anisotropic nikol skii classes the main technical tools used in our derivations are uniform bounds on the mathbb l s norms of empirical processes developed recently by goldenshluger and lepski ann probab 2011 to appear

orthogonal bases approach for comparing nonnormal continuous distributions summary we present an orthonormal bases approach for detecting general differences among continuous distributions an unknown density function is represented by a finite vector of its estimated fourier coefficients with respect to a suitable orthonormal basis for a wide class of orthonormal bases we establish asymptotic normality of the vector of estimated fourier coefficients and propose an unbiased and consistent estimator of its asymptotic covariance matrix fourier coefficients are modelled as functions of fixed and possibly random effects this approach allows simultaneous detection of distributional differences attributable to various factors in clustered and correlated data with sufficiently large numbers of observations per each cluster with the same fixed and random effects realisations this work was motivated by multi level clustered non gaussian datasets from genetic studies

poisson intensity estimation for tomographic data using a wavelet shrinkage approach we consider a two dimensional 2 d problem of positron emission tomography pet where the random mechanism of the generation of the tomographic data is modeled by poisson processes the goal is to estimate the intensity function which corresponds to emission density using the wavelet vaguelette decomposition wvd we propose an estimator based on thresholding of empirical vaguelette coefficients which attains the minimax rates of convergence on besov function classes furthermore we construct an adaptive estimator which attains the optimal rate of convergence up to a logarithmic term

adaptive confidence sets in l 2 the problem of constructing confidence sets that are adaptive in l 2 loss over a continuous scale of sobolev classes of probability densities is considered adaptation holds where possible with respect to both the radius of the sobolev ball and its smoothness degree and over maximal parameter spaces for which adaptation is possible two key regimes of parameter constellations are identified one where full adaptation is possible and one where adaptation requires critical regions be removed techniques used to derive these results include a general nonparametric minimax test for infinite dimensional null and alternative hypotheses and new lower bounds for l 2 adaptive confidence sets

adaptive nonparametric confidence sets we construct honest confidence regions for a hilbert space valued parameter in various statistical models the confidence sets can be centered at arbitrary adaptive estimators and have diameter which adapts optimally to a given selection of models the latter adaptation is necessarily limited in scope we review the notion of adaptive confidence regions and relate the optimal rates of the diameter of adaptive confidence regions to the minimax rates for testing and estimation applications include the finite normal mean model the white noise model density estimation and regression with random design institute of mathematical statistics 2006

new fast smoothers for multiscale systems we consider the smoothing problem for multiscale stochastic systems based on the wavelet transform these models involve processes indexed by the nodes of a dyadic tree each level of the dyadic tree represents one scale or resolution of the process therefore moving upward on the tree divides the resolution by 2 whereas moving downward multiplies it by 2 the processes are built according to a recursion in scale from coarse to fine to which random details are added to operate the change in scale one must perform an interpolation this is achieved using the qmf pair of operators attached to a wavelet transform these models have proved to be of great value to capture textures or fractal like processes as well as to perform multiresolution sensor fusion an example of which is given here up to now however only subclasses of multiscale systems were amenable to fast algorithms and through different formalisms those relying on haar s wavelet and those involving only one of the two wavelet interpolators we provide here a unifying framework that handles any system based on orthogonal wavelets a smoothing theory is presented to define the field of fast algorithms for markov random fields and give intuition on how to design them this theory reveals the difficulties arising with general multiscale systems we then prove that orthogonality properties of wavelets are the gate to fastness

an exponential inequality for the distribution function of the kernel density estimator with applications to adaptive estimation it is shown that the uniform distance between the distribution function f n k h of the usual kernel density estimator based on an i i d sample from an absolutely continuous law on mathbb r with bandwidth h and the empirical distribution function fn satisfies an exponential inequality this inequality is used to obtain sharp almost sure rates of convergence of f n k h n f n infty under mild conditions on the range of bandwidths hn including the usual mise optimal choices another application is a dvoretzky kiefer wolfowitz type inequality for f n k h f infty where f is the true distribution function the exponential bound is also applied to show that an adaptive estimator can be constructed that efficiently estimates the true distribution function f in sup norm loss and at the same time estimates the density of f if it exists but without assuming it does at the best possible rate of convergence over holder balls again in sup norm loss

minimax estimation via wavelet shrinkage we attempt to recover an unknown function from noisy sampled data using orthonormal bases of compactly supported wavelets we develop a nonlinear method which works in the wavelet domain by simple nonlinear shrinkage of the empirical wavelet coefficients the shrinkage can be tuned to be nearly minimax over any member of a wide range of triebel and besov type smoothness constraints and asymptotically minimax over besov bodies with p q linear estimates cannot achieve even the minimax rates over triebel and besov classes with p 2 so the method can significantly outperform every linear method e g kernel smoothing spline sieve in a minimax sense variants of our method based on simple threshold nonlinear estimators are nearly minimax our method possesses the interpretation of spatial adaptivity it reconstructs using a kernel which may vary in shape and bandwidth from point to point depending on the data least favorable distributions for certain of the triebel and besov scales generate objects with sparse wavelet transforms many real objects have similarly sparse transforms which suggests that these minimax results are relevant for practical problems sequels to this paper which was first drafted in november 1990 discuss practical implementation spatial adaptation properties universal near minimaxity and applications to inverse problems

adapting to unknown smoothness via wavelet shrinkage abstract we attempt to recover a function of unknown smoothness from noisy sampled data we introduce a procedure sureshrink that suppresses noise by thresholding the empirical wavelet coefficients the thresholding is adaptive a threshold level is assigned to each dyadic resolution level by the principle of minimizing the stein unbiased estimate of risk sure for threshold estimates the computational effort of the overall procedure is order n log n as a function of the sample size n sureshrink is smoothness adaptive if the unknown function contains jumps then the reconstruction essentially does also if the unknown function has a smooth piece then the reconstruction is essentially as smooth as the mother wavelet will allow the procedure is in a sense optimally smoothness adaptive it is near minimax simultaneously over a whole interval of the besov scale the size of this interval depends on the choice of mother wavelet we know from a previous paper by the authors that traditional smoot

about adaptive coding on countable alphabets this paper sheds light on adaptive coding with respect to classes of memoryless sources over a countable alphabet defined by an envelope function with finite and non decreasing hazard rate log concave envelope distributions we prove that the auto censuring ac code is adaptive with respect to the collection of such classes the analysis builds on the tight characterization of universal redundancy rate in terms of metric entropy and on a careful analysis of the performance of the ac coding algorithm the latter relies on nonasymptotic bounds for maxima of samples from discrete distributions with finite and nondecreasing hazard rate

wavelet regression in random design with heteroscedastic dependent errors we investigate function estimation in nonparametric regression models with random design and heteroscedastic correlated noise adaptive properties of warped wavelet nonlinear approximations are studied over a wide range of besov scales f b s r and for a variety of l p error measures we consider error distributions with long range dependence parameter 0 2 it is seen that there are three rate phases namely the dense sparse and long range dependence phase depending on the relative values of s p and furthermore we show that long range dependence does not come into play for shape estimation f f the theory is illustrated with some numerical examples

on locally adaptive density estimation abstract theoretical and practical aspects of the sample point adaptive positive kernel density estimator are examined a closed form expression for the mean integrated squared error is obtained through the device of preprocessing the data by binning with this expression the exact behavior of the optimally adaptive smoothing parameter function is studied for the first time the approach differs from most earlier techniques in that bias of the adaptive estimator remains o h 2 and is not improved to the rate o h 4 a practical algorithm is constructed using a modification of least squares cross validation simulated and real examples are presented including comparisons with a fixed bandwidth estimator and a fully automatic version of abramson s adaptive estimator the results are very promising

nonparametric reconstruction of a multifractal function from noisy data we estimate a real valued function f of d variables subject to additive gaussian perturbation at noise level varepsilon 0 under l loss for 1 the main novelty is that f can have an extremely varying local smoothness exhibiting a so called multifractal behaviour the results of jaffard on the frisch parisi conjecture suggest to link the singularity spectrum of f to besov properties of the signal that can be handled by wavelet thresholding for denoising purposes we prove that the optimal minimax rate of estimation of multifractal functions with singularity spectrum d h has explicit representation varepsilon 2v d bullet pi with v d bullet pi min h frac h left d d h right pi 2h d the minimum is taken over a specific domain and the rate is corrected by logarithmic factors in some cases in particular the usual rate varepsilon 2s 2s d is retrieved for monofractal functions with spectrum reduced to a single value s irrespectively of more interestingly the sparse case of estimation over single besov balls has a new interpretation in terms of multifractal analysis

wavelet based testing for serial correlation of unknown form in panel models wavelet analysis is a new mathematical method developed as a unified field of science over the last decade or so as a spatially adaptive analytic tool wavelets are useful for capturing serial correlation where the spectrum has peaks or kinks as can arise from persistent dependence seasonality and other kinds of periodicity this paper proposes a new class of generally applicable wavelet based tests for serial correlation of unknown form in the estimated residuals of a panel regression model where error components can be one way or two way individual and time effects can be fixed or random and regressors may contain lagged dependent variables or deterministic stochastic trending variables our tests are applicable to unbalanced heterogenous panel data they have a convenient null limit n 0 1 distribution no formulation of an alternative model is required and our tests are consistent against serial correlation of unknown form even in the presence of substantial in homogeneity in serial correlation across individuals this is in contrast to existing serial correlation tests for panel models which ignore inhomogeneity in serial correlation across individuals by assuming a common alternative and thus have no power against the alternatives where the average of serial correlations among individuals is close to zero we propose and justify a data driven method to choose the smoothing parameter the finest scale in wavelet spectral estimation making the tests completely operational in practice the data driven finest scale automatically converges to zero under the null hypothesis of no serial correlation and diverges to infinity as the sample size increases under the alternative ensuring the consistency of our tests simulation shows that our tests perform well in small and finite samples relative to some existing tests

normal approximations for wavelet coefficients on spherical poisson fields abstract we compute explicit upper bounds on the distance between the law of a multivariate gaussian distribution and the joint law of wavelet needlet coefficients based on a homogeneous spherical poisson field in particular we develop some results from peccati and zheng 2010 42 based on malliavin calculus and stein s methods to assess the rate of convergence to gaussianity for a triangular array of needlet coefficients with growing dimensions our results are motivated by astrophysical and cosmological applications in particular related to the search for point sources in cosmic rays data

parametric or nonparametric a parametricness index for model selection in model selection literature two classes of criteria perform well asymptotically in different situations bayesian information criterion bic as a representative is consistent in selection when the true model is finite dimensional parametric scenario akaike s information criterion aic performs well in an asymptotic efficiency when the true model is infinite dimensional nonparametric scenario but there is little work that addresses if it is possible and how to detect the situation that a specific model selection problem is in in this work we differentiate the two scenarios theoretically under some conditions we develop a measure parametricness index pi to assess whether a model selected by a potentially consistent procedure can be practically treated as the true model which also hints on aic or bic is better suited for the data for the goal of estimating the regression function a consequence is that by switching between aic and bic based on the pi the resulting regression estimator is simultaneously asymptotically efficient for both parametric and nonparametric scenarios in addition we systematically investigate the behaviors of pi in simulation and real data and show its usefulness

classification based on hybridization of parametric and nonparametric classifiers parametric methods of classification assume specific parametric models for competing population densities e g gaussian population densities can lead to linear and quadratic discriminant analysis and they work well when these model assumptions are valid violation in one or more of these parametric model assumptions often leads to a poor classifier on the other hand nonparametric classifiers e g nearest neighbor and kernel based classifiers are more flexible and free from parametric model assumptions but the statistical instability of these classifiers may lead to poor performance when we have small numbers of training sample observations nonparametric methods however do not use any parametric structure of population densities therefore even when one has some additional information about population densities that important information is not used to modify the nonparametric classification rule this paper makes an attempt to overcome these limitations of parametric and nonparametric approaches and combines their strengths to develop some hybrid classification methods we use some simulated examples and benchmark data sets to examine the performance of these hybrid discriminant analysis tools asymptotic results on their misclassification rates have been derived under appropriate regularity conditions

wavelet based density estimation for noise reduction in plasma simulations using particles for given computational resources the accuracy of plasma simulations using particles is mainly limited by the noise due to limited statistical sampling in the reconstruction of the particle distribution function a method based on wavelet analysis is proposed and tested to reduce this noise the method known as wavelet based density estimation wbde was previously introduced in the statistical literature to estimate probability densities given a finite number of independent measurements its novel application to plasma simulations can be viewed as a natural extension of the finite size particles fsp approach with the advantage of estimating more accurately distribution functions that have localized sharp features the proposed method preserves the moments of the particle distribution function to a good level of accuracy has no constraints on the dimensionality of the system does not require an a priori selection of a global smoothing scale and its able to adapt locally to the smoothness of the density based on the given discrete particle data moreover the computational cost of the denoising stage is of the same order as one time step of a fsp simulation the method is compared with a recently proposed proper orthogonal decomposition based method and it is tested with three particle data sets involving different levels of collisionality and interaction with external and self consistent fields

adaptive density estimation for directional data using needlets this paper is concerned with density estimation of directional data on the sphere we introduce a procedure based on thresholding on a new type of spherical wavelets called needlets we establish a minimax result and prove its optimality we are motivated by astrophysical applications in particular in connection with the analysis of ultra high energy cosmic rays

rate distortion optimization for h 264 interframe coding a general framework and algorithms rate distortion rd optimization for h 264 interframe coding with complete baseline decoding compatibility is investigated on a frame basis using soft decision quantization sdq rather than the standard hard decision quantization we first establish a general framework in which motion estimation quantization and entropy coding in h 264 for the current frame can be jointly designed to minimize a true rd cost given previously coded reference frames we then propose three rd optimization algorithms a graph based algorithm for near optimal sdq in h 264 baseline encoding given motion estimation and quantization step sizes an algorithm for near optimal residual coding in h 264 baseline encoding given motion estimation and an iterative overall algorithm to optimize h 264 baseline encoding for each individual frame given previously coded reference frames with them embedded in the indicated order the graph based algorithm for near optimal sdq is the core given motion estimation and quantization step sizes it is guaranteed to perform optimal sdq if the weak adjacent block dependency utilized in the context adaptive variable length coding of h 264 is ignored for optimization the proposed algorithms have been implemented based on the reference encoder jm82 of h 264 with complete compatibility to the baseline profile experiments show that for a set of typical video testing sequences the graph based algorithm for near optimal sdq the algorithm for near optimal residual coding and the overall algorithm achieve on average 6 8 and 12 respectively rate reduction at the same psnr ranging from 30 to 38 db when compared with the rd optimization method implemented in the h 264 reference software

limits to estimation in stochastic ill conditioned inverse problems using information theoretic methods we develop simple results quantifying a lower bound for minimax estimation a kind of infinite dimensional cramer rao lower bound for signal estimation in possibly nonlinear ill conditioned inverse problems our results reduce calculation to a geometric computation based on a modulus of continuity and make explicit connections with results in the literature on deterministic ill conditioned inverse problems several applications are discussed

formulae for mean integrated squared error of nonlinear wavelet based density estimators we provide an asymptotic formula for the mean integrated squared error mise of nonlinear wavelet based density estimators we show that unlike the analogous situation for kernel density estimators this mise formula is relatively unaffected by assumptions of continuity in particular it is available for densities which are smooth in only a piecewise sense another difference is that in the wavelet case the classical mise formula is valid only for sufficiently small values of the bandwidth for larger bandwidths mise assumes a very different form and hardly varies at all with changing bandwidth this remarkable property guarantees a high level of robustness against oversmoothing not encountered in the context of kernel methods we also use the mise formula to describe an asymptotically optimal empirical bandwidth selection rule

an asymptotic property of model selection criteria probability models are estimated by use of penalized log likelihood criteria related to akaike 1973 information criterion aic and minimum description length mdl the accuracies of the density estimators are shown to be related to the tradeoff between three terms the accuracy of approximation the model dimension and the descriptive complexity of the model classes the asymptotic risk is determined under conditions on the penalty term and is shown to be minimax optimal for some cases as an application we show that the optimal rate of convergence is simultaneously achieved for log densities in sobolev spaces w sub 2 sup s u without knowing the smoothness parameter s and norm parameter u in advance applications to neural network models and sparse density function estimation are also provided

the root unroot algorithm for density estimation as implemented via wavelet block thresholding we propose and implement a density estimation procedure which begins by turning density estimation into a nonparametric regression problem this regression problem is created by binning the original observations into many small size bins and by then applying a suitable form of root transformation to the binned data counts in principle many common nonparametric regression estimators could then be applied to the transformed data we propose use of a wavelet block thresholding estimator in this paper finally the estimated regression function is un rooted by squaring and normalizing the density estimation procedure achieves simultaneously three objectives computational efficiency adaptivity and spatial adaptivity a numerical example and a practical data example are discussed to illustrate and explain the use of this procedure theoretically it is shown that the estimator simultaneously attains the optimal rate of convergence over a wide range of the besov classes the estimator also automatically adapts to the local smoothness of the underlying function and attains the local adaptive minimax rate for estimating functions at a point there are three key steps in the technical argument poissonization quantile coupling and oracle risk bound for block thresholding in the non gaussian setting some of the technical results may be of independent interest

discriminant feature extraction using empirical probability density estimation and a local basis library the authors previously developed the so called local discriminant basis ldb method for signal and image classi3cation problems the original ldb method relies on di4erences in the time frequency energy distribution of each class it selects the subspaces where these energy distributions are well separated by some measure such as the kullback leibler divergence through our experience and experiments on various datasets however we realized that the time frequency energy distribution is not always the best quantity to analyze for classi3cation in this paper we propose to use the discrimination of coordinates based instead on empirical probability densities that is we estimate the probability density of each class in each coordinate in the wavelet packet local trigonometric bases after expanding signals into such bases we then evaluate a power of discrimination of each subspace by selecting the m most discriminant coordinates in terms of the distance among the corresponding densities e g by the kullback leibler divergence among the densities this information is then used for selecting a basis for classi3cation we will demonstrate the capability of this algorithm using both synthetic and real datasets 2002 pattern recognition society published by elsevier science ltd all rights reserved

wavelet deconvolution this paper studies the issue of optimal deconvolution density estimation using wavelets the approach taken here can be considered as orthogonal series estimation in the more general context of the density estimation we explore the asymptotic properties of estimators based on thresholding of estimated wavelet coefficients minimax rates of convergence under the integrated square loss are studied over besov classes b sub spl sigma pq of functions for both ordinary smooth and supersmooth convolution kernels the minimax rates of convergence depend on the smoothness of functions to be deconvolved and the decay rate of the characteristic function of convolution kernels it is shown that no linear deconvolution estimators can achieve the optimal rates of convergence in the besov spaces with p 2 when the convolution kernel is ordinary smooth and super smooth if the convolution kernel is ordinary smooth then linear estimators can be improved by using thresholding wavelet deconvolution estimators which are asymptotically minimax within logarithmic terms adaptive minimax properties of thresholding wavelet deconvolution estimators are also discussed

function estimation via wavelet shrinkage for long memory data in this article we study function estimation via wavelet shrinkage for data with long range dependence we propose a fractional gaussian noise model to approximate nonparametric regression with long range dependence and establish asymptotics for minimax risks because of long range dependence the minimax risk and the minimax linear risk converge to 0 at rates that differ from those for data with independence or short range dependence wavelet estimates with best selection of resolution leveldependent threshold achieve minimax rates over a wide range of spaces cross validation for dependent data is proposed to select the optimal threshold the wavelet estimates significantly outperform linear estimates the key to proving the asymptotic results is a wavelet vaguelette decomposition which decorrelates fractional gaussian noise such wavelet vaguelette decomposition is also very useful in fractal signal processing

pointwise density estimation for biased sample abstract this paper considers pointwise estimation of multivariate density function based on biased sample we provide upper bounds of linear and nonlinear wavelet estimators over l p 1 p risk it should be pointed out that the convergence rate of nonlinear estimator keeps the same as the linear one up to a ln n factor moreover the nonlinear wavelet estimator is adaptive finally simulation studies illustrate the good performances of our nonparametric wavelet estimators

mixing strategies for density estimation general results on adaptive density estimation are obtained with respect to any countable collection of estimation strategies under kullback leibler and squared l 2 losses it is shown that without knowing which strategy works best for the underlying density a single strategy can be constructed by mixing the proposed ones to be adaptive in terms of statistical risks a consequence is that under some mild conditions an asymptotically minimax rate adaptive estimator exists for a given countable collection of density classes that is a single estimator can be constructed to be simultaneously minimax rate optimal for all the function classes being considered a demonstration is given for high dimensional density estimation on 0 1 d where the constructed estimator adapts to smoothness and interaction order over some piecewise besov classes and is consistent for all the densities with finite entropy