ideal spatial adaptation by wavelet shrinkage summary with ideal spatial adaptation an oracle furnishes information about how best to adapt a spatially variable estimator whether piecewise constant piecewise polynomial variable knot spline or variable bandwidth kernel to the unknown function estimation with the aid of an oracle offers dramatic advantages over traditional linear estimation by nonadaptive kernels however it is a priori unclear whether such performance can be obtained by a procedure relying on the data alone we describe a new principle for spatially adaptive estimation selective wavelet reconstruction we show that variable knot spline fits and piecewise polynomial fits when equipped with an oracle to select the knots are not dramatically more powerful than selective wavelet reconstruction with an oracle we develop a practical spatially adaptive method riskshrink which works by shrinkage of empirical wavelet coefficients riskshrink mimics the performance of an oracle for selective wavelet reconstruction as well as it is possible to do so a new inequality in multivariate normal decision theory which we call the oracle inequality shows that attained performance differs from ideal performance by at most a factor of approximately 2 log n where n is the sample size moreover no estimator can give a better guarantee than this within the class of spatially adaptive procedures riskshrink is essentially optimal relying only on the data it comes within a factor log 2 n of the performance of piecewise polynomial and variableknot spline methods equipped with an oracle in contrast it is unknown how or if piecewise polynomial methods could be made to function this well when denied access to an oracle and forced to rely on data alone

a joint inter and intrascale statistical model for bayesian wavelet based image denoising this paper presents a new wavelet based image denoising method which extends a geometrical bayesian framework the new method combines three criteria for distinguishing supposedly useful coefficients from noise coefficient magnitudes their evolution across scales and spatial clustering of large coefficients near image edges these three criteria are combined in a bayesian framework the spatial clustering properties are expressed in a prior model the statistical properties concerning coefficient magnitudes and their evolution across scales are expressed in a joint conditional model the three main novelties with respect to related approaches are 1 the interscale ratios of wavelet coefficients are statistically characterized and different local criteria for distinguishing useful coefficients from noise are evaluated 2 a joint conditional model is introduced and 3 a novel anisotropic markov random field prior model is proposed the results demonstrate an improved denoising performance over related earlier techniques

unbalanced optimal transport dynamic and kantorovich formulations abstract this article presents a new class of distances between arbitrary nonnegative radon measures inspired by optimal transport these distances are defined by two equivalent alternative formulations i a dynamic formulation defining the distance as a geodesic distance over the space of measures ii a static kantorovich formulation where the distance is the minimum of an optimization problem over pairs of couplings describing the transfer transport creation and destruction of mass between two measures both formulations are convex optimization problems and the ability to switch from one to the other depending on the targeted application is a crucial property of our models of particular interest is the wasserstein fisher rao metric recently introduced independently by 7 15 defined initially through a dynamic formulation it belongs to this class of metrics and hence automatically benefits from a static kantorovich formulation

geometric properties of cones with applications on the hellinger kantorovich space and a new distance on the space of probability measures abstract we study general geometric properties of cone spaces and we apply them on the hellinger kantorovich space m x hk we exploit a two parameter scaling property of the hellinger kantorovich metric hk and we prove the existence of a distance shk on the space of probability measures that turns the hellinger kantorovich space m x hk into a cone space over the space of probabilities measures p x shk we provide a two parameter rescaling of geodesics in m x hk and for p x shk we obtain a full characterization of the geodesics we finally prove finer geometric properties including local angle condition and partial k semiconcavity of the squared distances that will be used in a future paper to prove existence of gradient flows on both spaces

wavelet threshold estimators for data with correlated noise wavelet threshold estimators for data with stationary correlated noise are constructed by applying a level dependent soft threshold to the coefficients in the wavelet transform a variety of threshold choices is proposed including one based on an unbiased estimate of mean squared error the practical performance of the method is demonstrated on examples including data from a neurophysiological context the theoretical properties of the estimators are investigated by comparing them with an ideal but unattainable bench mark that can be considered in the wavelet context as the risk obtained by ideal spatial adaptivity and more generally is obtained by the use of an oracle that provides information that is not actually available in the data it is shown that the level dependent threshold estimator performs well relative to the bench mark risk and that its minimax behaviour cannot be improved on in order of magnitude by any other estimator the wavelet domain structure of both short and long range dependent noise is considered and in both cases it is shown that the estimators have near optimal behaviour simultaneously in a wide range of function classes adapting automatically to the regularity properties of the underlying model the proofs of the main results are obtained by considering a more general multivariate normal decision theoretic problem

empirical bayes approach to improve wavelet thresholding for image noise reduction wavelet threshold algorithms replace small magnitude wavelet coefficients with zero and keep or shrink the other coefficients this is basically a local procedure because wavelet coefficients characterize the local regularity of a function although a wavelet transform has decorrelating properties structures in images like edges are never decorrelated completely and these structures appear in the wavelet coefficients a classification based on a local criterion like coefficient magnitude is not the perfect method to distinguish important uncorrupted coefficients from coefficients dominated by noise we therefore introduce a geometrical prior model for configurations of important wavelet coefficients and combine this with local characterization of a classical threshold procedure into a bayesian framework the local characterization is incorporated into the conditional model whereas the prior model describes only configurations not coefficient values more precisely local characterization favors co

plane rotation based evd updating schemes for efficient subspace tracking we present new algorithms based on plane rotations for tracking the eigenvalue decomposition evd of a time varying data covariance matrix these algorithms directly produce eigenvectors in orthonormal form and are well suited for the application of subspace methods to nonstationary data after recasting evd tracking as a simplified rank one evd update problem computationally efficient solutions are obtained in two steps first a new kind of parametric perturbation approach is used to express the eigenvector update as an unimodular orthogonal transform which is represented in exponential matrix form in terms of a reduced set of small unconstrained parameters second two approximate decompositions of this exponential matrix into products of plane or givens rotations are derived one of which being previously unknown these decompositions lead to new plane rotation based evd updating schemes proteus whose main feature is the use of plane rotations for updating the eigenvectors thereby preserving orthonormality finally the proteus schemes are used to derive new evd trackers whose convergence and numerical stability are investigated via simulations one algorithm can track all the signal subspace evd components in only o lm operations where l and m respectively denote the data vector and signal subspace dimensions while achieving a performance comparable to an exact evd approach and maintaining perfect orthonormality of the eigenvectors the new algorithms show no signs of error buildup

gradient flows and evolution variational inequalities in metric spaces i structural properties abstract this is the first of a series of papers devoted to a thorough analysis of the class of gradient flows in a metric space x d that can be characterized by evolution variational inequalities evi we present new results concerning the structural properties of solutions to the evi formulation such as contraction regularity asymptotic expansion precise energy identity stability asymptotic behavior and their link with the geodesic convexity of the driving functional under the crucial assumption of the existence of an evi gradient flow we will also prove two main results the equivalence with the de giorgi variational characterization of curves of maximal slope the convergence of the minimizing movement jko scheme to the evi gradient flow with an explicit and uniform error estimate of order 1 2 with respect to the step size independent of any geometric hypothesis as upper or lower curvature bounds on d in order to avoid any compactness assumption we will also introduce a suitable relaxation of the minimizing movement algorithm obtained by the ekeland variational principle and we will prove its uniform convergence as well

the design and use of steerable filters the authors present an efficient architecture to synthesize filters of arbitrary orientations from linear combinations of basis filters allowing one to adaptively steer a filter to any orientation and to determine analytically the filter output as a function of orientation steerable filters may be designed in quadrature pairs to allow adaptive control over phase as well as orientation the authors show how to design and steer the filters and present examples of their use in the analysis of orientation and phase angularly adaptive filtering edge detection and shape from shading one can also build a self similar steerable pyramid representation the same concepts can be generalized to the design of 3 d steerable filters

convex sobolev inequalities related to unbalanced optimal transport abstract we study the behavior of various lyapunov functionals relative entropies along the solutions of a family of nonlinear drift diffusion reaction equations coming from statistical mechanics and population dynamics these equations can be viewed as gradient flows over the space of radon measures equipped with the hellinger kantorovich distance the driving functionals of the gradient flows are not assumed to be geodesically convex or semi convex we prove new isoperimetric type functional inequalities allowing us to control the relative entropies by their productions which yields the exponential decay of the relative entropies

image denoising using scale mixtures of gaussians in the wavelet domain we describe a method for removing noise from digital images based on a statistical model of the coefficients of an overcomplete multiscale oriented basis neighborhoods of coefficients at adjacent positions and scales are modeled as the product of two independent random variables a gaussian vector and a hidden positive scalar multiplier the latter modulates the local variance of the coefficients in the neighborhood and is thus able to account for the empirically observed correlation between the coefficient amplitudes under this model the bayesian least squares estimate of each coefficient reduces to a weighted average of the local linear estimates over all possible values of the hidden multiplier variable we demonstrate through simulations with images contaminated by additive white gaussian noise that the performance of this method substantially surpasses that of previously published methods both visually and in terms of mean squared error

a new maximum margin algorithm for one class problems and its boosting implementation in this paper each one class problem is regarded as trying to estimate a function that is positive on a desired slab and negative on the complement the main advantage of this viewpoint is that the loss function and the expected risk can be defined to ensure that the slab can contain as many samples as possible inspired by the nature of svms the intuitive margin is also defined as a result a new linear optimization problem to maximize the margin and some theoretically motivated learning algorithms are obtained moreover the proposed algorithms can be implemented by boosting techniques to solve nonlinear one class classifications

gradient flows on wasserstein spaces over compact alexandrov spaces we establish the existence of euclidean tangent cones on wasserstein spaces over compact alexandrov spaces of curvature bounded below by using this riemannian structure we formulate and construct gradient flows of functions on such spaces if the underlying space is a riemannian manifold of nonnegative sectional curvature then our gradient flow of the free energy produces a solution of the linear fokker planck equation

optimal entropy transport problems and a new hellinger kantorovich distance between positive measures we develop a full theory for the new class of optimal entropy transport problems between nonnegative and finite radon measures in general topological spaces these problems arise quite naturally by relaxing the marginal constraints typical of optimal transport problems given a pair of finite measures with possibly different total mass one looks for minimizers of the sum of a linear transport functional and two convex entropy functionals which quantify in some way the deviation of the marginals of the transport plan from the assigned measures as a powerful application of this theory we study the particular case of logarithmic entropy transport problems and introduce the new hellinger kantorovich distance between measures in metric spaces the striking connection between these two seemingly far topics allows for a deep analysis of the geometric properties of the new geodesic distance which lies somehow between the well known hellinger kakutani and kantorovich wasserstein distances

new support vector algorithms we propose a new class of support vector algorithms for regression and classification in these algorithms a parameter lets one effectively control the number of support vectors while this can be useful in its own right the parameterization has the additional benefit of enabling us to eliminate one of the other free parameters of the algorithm the accuracy parameter epsilon in the regression case and the regularization constant c in the classification case we describe the algorithms give some theoretical results concerning the meaning and the choice of and report experimental results

image denoising using total least squares in this paper we present a method for removing noise from digital images corrupted with additive multiplicative and mixed noise an image patch from an ideal image is modeled as a linear combination of image patches from the noisy image we propose to fit this model to the real world image data in the total least square tls sense because the tls formulation allows us to take into account the uncertainties in the measured data we develop a method to reduce the contribution from the irrelevant image patches which will sharpen the edges and reduce edge artifacts at the same time although the proposed algorithm is computationally demanding the image quality of the output image demonstrates the effectiveness of the tls algorithms

learning linear pca with convex semi definite programming the aim of this paper is to learn a linear principal component using the nature of support vector machines svms to this end a complete svm like framework of linear pca svpca for deciding the projection direction is constructed where new expected risk and margin are introduced within this framework a new semi definite programming problem for maximizing the margin is formulated and a new definition of support vectors is established as a weighted case of regular pca our svpca coincides with the regular pca if all the samples play the same part in data compression theoretical explanation indicates that svpca is based on a margin based generalization bound and thus good prediction ability is ensured furthermore the robust form of svpca with a interpretable parameter is achieved using the soft idea in svms the great advantage lies in the fact that svpca is a learning algorithm without local minima because of the convexity of the semi definite optimization problems to validate the performance of svpca several experiments are conducted and numerical results have demonstrated that their generalization ability is better than that of regular pca finally some existing problems are also discussed

estimating the support of a high dimensional distribution suppose you are given some data set drawn from an underlying probability distribution p and you want to estimate a simple subset s of input space such that the probability that a test point drawn from p lies outside of s equals some a priori specified value between 0 and 1 we propose a method to approach this problem by trying to estimate a function f that is positive on s and negative on the complement the functional form of f is given by a kernel expansion in terms of a potentially small subset of the training data it is regularized by controlling the length of the weight vector in an associated feature space the expansion coefficients are found by solving a quadratic programming problem which we do by carrying out sequential optimization over pairs of input patterns we also provide a theoretical analysis of the statistical performance of our algorithm the algorithm is a natural extension of the support vector algorithm to the case of unlabeled data

on the geometry of metric measure spaces ii we introduce a curvature dimension condition cd k n for metric measure spaces it is more restrictive than the curvature bound underline text curv left m text d m right k introduced in sturm k t 2006 on the geometry of metric measure spaces i acta math 196 65 131 which is recovered as the borderline case cd k the additional real parameter n plays the role of a generalized upper bound for the dimension for riemannian manifolds cd k n is equivalent to text ric m left xi xi right k left xi right 2 and dim m n the curvature dimension condition cd k n is stable under convergence for any triple of real numbers k n l the family of normalized metric measure spaces m d m with cd k n and diameter l is compact condition cd k n implies sharp version of the brunn minkowski inequality of the bishop gromov volume comparison theorem and of the bonnet myers theorem moreover it implies the doubling property and local scale invariant poincare inequalities on balls in particular it allows to construct canonical dirichlet forms with gaussian upper and lower bounds for the corresponding heat kernels

approximate polytope ensemble for one class classification in this work a new one class classification ensemble strategy called approximate polytope ensemble is presented the main contribution of the paper is threefold first the geometrical concept of convex hull is used to define the boundary of the target class defining the problem expansions and contractions of this geometrical structure are introduced in order to avoid over fitting second the decision whether a point belongs to the convex hull model in high dimensional spaces is approximated by means of random projections and an ensemble decision process finally a tiling strategy is proposed in order to model non convex structures experimental results show that the proposed strategy is significantly better than state of the art one class classification methods on over 200 datasets highlightsthe methodology uses a convex hull for modeling one class classification problems random projections are used to approximate the convex hull in high dimensional spaces expansions of the approximate hulls are considered to set the optimal operating point exhaustive validation is performed on three different typologies of problems

regularized principal manifolds many settings of unsupervised learning can be viewed as quantization problems the minimization of the expected quantization error subject to some restrictions this allows the use of tools such as regularization from the theory of supervised risk minimization for unsupervised learning this setting turns out to be closely related to principal curves the generative topographic map and robust coding we explore this connection in two ways 1 we propose an algorithm for finding principal manifolds that can be regularized in a variety of ways and 2 we derive uniform convergence bounds and hence bounds on the learning rates of the algorithm in particular we give bounds on the covering numbers which allows us to obtain nearly optimal learning rates for certain types of regularization operators experimental results demonstrate the feasibility of the approach