detecting the presence of mixing with multiscale maximum likelihood a test of homogeneity tries to decide whether observations come from a single distribution or from a mixture of several distributions a powerful theory has been developed for the case where the component distributions are members of an exponential family when no parametric assumptions are appropriate the standard approach is to test for bimodality which is known not to be very sensitive for detecting heterogeneity to develop a more sensitive procedure this article builds on an approach employed in sampling literature and models the component distributions as logarithmically concave densities it is shown how this leads to a special semiparametric model in which the homogeneity problem is equivalent to testing whether a parameter c equals zero versus the alternative that c 0 this setup leads naturally to a novel multiscale maximum likelihood procedure where the multiscale character reflects the desirable property of adaptivity to the unknown value of the parameter c under the alternative to en

some aspects of the spline smoothing approach to non parametric regression curve fitting non parametric regression using cubic splines is an attractive flexible and widely applicable approach to curve estimation although the basic idea was formulated many years ago the method is not as widely known or adopted as perhaps it should be the topics and examples discussed in this paper are intended to promote the understanding and extend the practicability of the spline smoothing methodology particular subjects covered include the basic principles of the method the relation with moving average and other smoothing methods the automatic choice of the amount of smoothing and the use of residuals for diagnostic checking and model adaptation the question of providing inference regions for curves and for relevant properties of curves is approached via a finite dimensional bayesian formulation

consistency and convergence rates of one class svms and related algorithms we determine the asymptotic behaviour of the function computed by support vector machines svm and related algorithms that minimize a regularized empirical convex loss function in the reproducing kernel hilbert space of the gaussian rbf kernel in the situation where the number of examples tends to infinity the bandwidth of the gaussian kernel tends to 0 and the regularization parameter is held fixed non asymptotic convergence bounds to this limit in the l2 sense are provided together with upper bounds on the classification error that is shown to converge to the bayes risk therefore proving the bayes consistency of a variety of methods although the regularization term does not vanish these results are particularly relevant to the one class svm for which the regularization can not vanish by construction and which is shown for the first time to be a consistent density level set estimator

on the estimation of a probability density function by the maximum penalized likelihood method abstract a class of probability density estimates can be obtained by penalizing the likelihood by a functional which depends on the roughness of the logarithm of the density the limiting case of the estimates as the amount of smoothing increasing has a natural form which makes the method attractive for data analysis and which provides a rationale for a particular choice of roughness penalty the estimates are shown to be the solution of an unconstrained convex optimization problem and mild natural conditions are given for them to exist rates of consistency in various norms and conditions for asymptotic normality and approximation by a gaussian process are given thus breaking new ground in the theory of maximum penalized likelihood density estimation author

fusion of hard and soft information in nonparametric density estimation abstract this paper discusses univariate density estimation in situations when the sample hard information is supplemented by soft information about the random phenomenon these situations arise broadly in operations research and management science where practical and computational reasons severely limit the sample size but problem structure and past experiences could be brought in in particular density estimation is needed for generation of input densities to simulation and stochastic optimization models in analysis of simulation output and when instantiating probability models we adopt a constrained maximum likelihood estimator that incorporates any possibly random soft information through an arbitrary collection of constraints we illustrate the breadth of possibilities by discussing soft information about shape support continuity smoothness slope location of modes symmetry density values neighborhood of known density moments and distribution functions the maximization takes place over spaces of extended real valued semicontinuous functions and therefore allows us to consider essentially any conceivable density as well as convenient exponential transformations the infinite dimensionality of the optimization problem is overcome by approximating splines tailored to these spaces to facilitate the treatment of small samples the construction of these splines is decoupled from the sample we discuss existence and uniqueness of the estimator examine consistency under increasing hard and soft information and give rates of convergence numerical examples illustrate the value of soft information the ability to generate a family of diverse densities and the effect of misspecification of soft information

a penalized likelihood approach to image warping discussion a warping is a function that deforms images by mapping between image domains the choice of function is formulated statistically as maximum penalized likelihood where the likelihood measures the similarity between images after warping and the penalty is a measure of distortion of a warping the paper addresses two issues simultaneously of how to choose the warping function and how to assess the alignment a new fourier von mises image model is identified with phase differences between fourier transformed images having von mises distributions also new null set distortion criteria are proposed with each criterion uniquely minimized by a particular set of polynomial functions a conjugate gradient algorithm is used to estimate the warping function which is numerically approximated by a piecewise bilinear function the method is motivated by and used to solve three applied problems to register a remotely sensed image with a map to align microscope images obtained by using different optics and to discriminate between species of fish from photographic images

a semiparametric approach to density estimation abstract one method of fitting a parametric density function f x is first to estimate by maximum likelihood say and then to estimate f x by on the other hand when the parametric model does not hold the true density f x may be estimated nonparametrically as in the case of a kernel estimate the key idea proposed is to fit a combination of the parametric and nonparametric estimates namely where 0 1 is unknown the parameter is estimated from the data and its estimate is then used in as the proposed density estimate the main point is that we expect to be close to unity when the parametric model prevails and close to zero when it does not hold we show that under certain conditions converges to the parametric density when the parametric model holds and approaches the true f x when the parametric model does not hold the procedure was applied to a number of actual data sets in each case the maximum likelihood estimate was readily obtained and the semiparametric density es

spline smoothing the equivalent variable kernel method on considere une approche spline pour la regression non parametrique et l estimation de courbe on montre que dans un certain sens le lissage spline correspond approximativement au lissage par une methode du noyau avec une largeur de bande dependant de la densite locale des points du plan de regression

asymptotics of the discrete log concave maximum likelihood estimator and related applications the assumption of log concavity is a flexible and appealing non parametric shape constraint in distribution modelling in this work we study the log concave maximum likelihood estimator of a probability mass function we show that the maximum likelihood estimator is strongly consistent and we derive its pointwise asymptotic theory under both the well specified and misspecified settings our asymptotic results are used to calculate confidence intervals for the true log concave probability mass function both the maximum likelihood estimator and the associated confidence intervals may be easily computed by using the r package logcondiscr we illustrate our theoretical results by using recent data from the h1n1 pandemic in ontario canada

smoothing spline semiparametric density models density estimation plays a fundamental role in many areas of statistics and machine learning parametric nonparametric and semiparametric density estimation methods have been proposed in the lite

discretized laplacian smoothing by fourier methods abstract an approach to multidimensional smoothing is introduced that is based on a penalized likelihood with a modified discretized laplacian penalty term the choice of penalty simplifies computational difficulties associated with standard multidimensional laplacian smoothing methods yet without compromising mean squared error characteristics at least on the interior of the region of interest for linear smoothing in hyper rectangular domains which has wide application in image reconstruction and restoration problems computations are carried out using fast fourier transforms nonlinear smoothing is accomplished by iterative application of the linear smoothing technique the iterative procedure is shown to be convergent under general conditions adaptive choice of the amount of smoothing is based on approximate cross validation type scores an importance sampling technique is used to estimate the degrees of freedom of the smooth the methods are implemented in one and two dimensional settings some i

smoothing spline density estimation a dimensionless automatic algorithm abstract as a sequel to an earlier article by gu and qiu this article describes and illustrates a dimensionless automatic algorithm for nonparametric probability density estimation using smoothing splines the algorithm is designed to calculate an adaptive finite dimensional solution to the penalized likelihood problem which was shown by gu and qiu to share the same asymptotic convergence rates as the nonadaptive infinite dimensional solution the smoothing parameter is updated jointly with the estimate in a performance oriented iteration via a cross validation performance estimate where the performance is measured by proxies of the symmetrized kullback leibler distance between the true density and the estimate simulations of limited scale are conducted to examine the relative effectiveness of the automatic smoothing parameter selection procedure and to assess the practical statistical performance of the methodology in general the method is also applied to some real data sets the algorithm is implem

exponential inequalities for martingales with application to maximum likelihood estimation for counting processes we obtain an exponential probability inequality for martingales and a uniform probability inequality for the process gdn where n is a counting process and where g varies within a class of predictable functions g for the latter we use techniques from empirical process theory the uniform inequality is shown to hold under certain entropy conditions on g as an application we consider rates of convergence for nonparametric maximum likelihood estimators for counting processes a similar result for discrete time observations is also presented

cross validated spline methods for the estimation of three dimensional tumor size distributions from observations on two dimensional cross sections abstract we study the problem of estimating the distribution of the three dimensional radiuses of a collection of spheres given measurements of the two dimensional radiuses of a sample of planar cross sections this problem arises in the estimation of the tumor size distribution of spherical microtumors induced in mouse livers following injection of a carcinogen we first convert this problem to a form suitable for the application of cross validated spline methods for the solution of ill posed integral equations given noisy data then we develop special numerical techniques that will allow the spline methods to be accurately applied to integral equations like those associated with the present problem we apply the resulting method to some mouse liver data the subject mouse liver has been completely dissected allowing a rare comparison of the estimate with the truth the statistical properties of the estimate are explored via monte carlo methods the interplay between statistical and numerical analyti

estimating divergence functionals and the likelihood ratio by convex risk minimization we develop and analyze m estimation methods for divergence functionals and the likelihood ratios of two probability distributions our method is based on a nonasymptotic variational characterization of f divergences which allows the problem of estimating divergences to be tackled via convex empirical risk optimization the resulting estimators are simple to implement requiring only the solution of standard convex programs we present an analysis of consistency and convergence for these estimators given conditions only on the ratios of densities we show that our estimators can achieve optimal minimax rates for the likelihood ratio and the divergence functionals in certain regimes we derive an efficient optimization algorithm for computing our estimates and illustrate their convergence behavior and practical viability by simulations

a tabu search algorithm for the bipartite drawing problem graphs are used to represent reality in several areas of knowledge this has generated considerable interest in graph drawing algorithms arc crossing minimization is a fundamental aesthetic criterion to obtain a readable map of a graph the problem of minimizing the number of arc crossings in a bipartite graph bdp is np complete in this paper we present a tabu search ts scheme for the bdp several algorithms can be obtained with this scheme by implementing different evaluators in the move definitions in this paper we propose two variants computational results are reported on a set of 300 randomly generated test problems the two algorithms have been compared with the best heuristics published in the literature and for limited size instances with the optimal solutions

log density estimation in linear inverse problems we estimate a probability density function p which is related by a linear operator k to a density function q in sequences of regular exponential families based on a random sample from q in this paper deconvolution and positron emission tomography are considered the logarithm of the density function is approximated by basis functions consisting of singular functions of k while direct maximum likelihood or minimum kullback leibler density estimation in exponential families selects the parameters to match the moments of the basis functions to the sample moments in the inverse problem the moment of each singular function is related to a corresponding moment of the direct problem by a factor given by a singular value a of k thus an appropriate analogue of the maximum likelihood estimate is obtained by matching moments with respect to p to 1 a times the empirical moments associated with the sample from q bounds on the kullback leibler distance between the true density and the estimators are obtained and rates of convergence are established for log density functions having a measure of smoothness the density estimator converges to the unknown density in the kullback leibler sense and in the l2 sense at a rate determined not only by the order of smoothness of the log density and the dimension of data but also by the decay rate of the singular values of the operator a minimax lower bound for deconvolution is provided under certain conditions numerical examples using simulated data are provided to illustrate the finite sample performance of the proposed method for deconvolution and positron emission tomography

nonparametric transformations for both sides of a regression model one way to model heteroscedasticity and skewness of the error distribution in regression is to transform both sides of the regression equation if it is possible to transform the regression equation to result in normally distributed errors then we can obtain more efficient parameter estimates and valid prediction intervals one problem with this approach is that the choice of transformation is usually restricted to the power or shifted power family often there is no scientific basis for this model and the limited flexibility of this parametric family may miss important features of the distribution a more comprehensive approach is to estimate the transformation by using nonparametric methods based on maximizing a penalized likelihood function this maximization problem leads naturally to a spline estimate for the log derivative of the transformation an algorithm for computing the estimate is given along with results on the existence and uniqueness of the estimate this nonparametric method is illustrated by using a non normally distributed fisheries data set

penalized maximum likelihood estimation the baum welch algorithm diagonal balancing of symmetric matrices and applications to training acoustic data we study penalized maximum likelihood estimation methods for nonparametric density estimation and propose their use in training acoustic models for speech recognition several algorithms for the numerical solution of the optimization problems that we encounter are proposed and analyzed

efficient computation of smoothing splines via adaptive basis sampling summary smoothing splines provide flexible nonparametric regression estimators however the high computational cost of smoothing splines for large datasets has hindered their wide application in this article we develop a new method named adaptive basis sampling for efficient computation of smoothing splines in super large samples except for the univariate case where the reinsch algorithm is applicable a smoothing spline for a regression problem with sample size n can be expressed as a linear combination of n basis functions and its computational complexity is generally o n 3 we achieve a more scalable computation in the multivariate case by evaluating the smoothing spline using a smaller set of basis functions obtained by an adaptive sampling scheme that uses values of the response variable our asymptotic analysis shows that smoothing splines computed via adaptive basis sampling converge to the true function at the same rate as full basis smoothing splines using simulation studies and a large scale deep earth core mantle boundary imaging study we show that the proposed method outperforms a sampling method that does not use the values of response variables

on maximum likelihood estimation in infinite dimensional parameter spaces an approximate maximum likelihood estimate is known to be consistent under some compactness and integrability conditions in this paper we study its convergence rate and its asymptotic efficiency in estimating smooth functionals of the parameter we provide conditions under which the rate of convergence can be established this rate is essentially governed by the size of the space of score functions as measured by an entropy index we also show that for a large class of smooth functionals the plug in maximum likelihood estimate is asymptotically efficient that is it achieves the minimal fisher information bound the theory is illustrated by several nonparametric or semiparametric examples

maximum likelihood estimation of smooth monotone and unimodal densities we study the nonparametric estimation of univariate monotone and unimodal densities using the maximum smoothed likelihood approach the monotone estimator is the derivative of the least concave majorant of the distribution corresponding to a kernel estimator we prove that the mapping on distributions with density the derivative of the least concave majorant of is a contraction in all l p norms 1 p and some other distances such as the hellinger and kullback leibler distances the contractivity implies error bounds for monotone density estimation almost the same error bounds hold for unimodal estimation

maximum smoothed likelihood for multivariate mixtures summary we introduce an algorithm for estimating the parameters in a finite mixture of completely unspecified multivariate components in at least three dimensions under the assumption of conditionally independent coordinate dimensions we prove that this algorithm based on a majorization minimization idea possesses a desirable descent property just as any em algorithm does we discuss the similarities between our algorithm and a related one the so called nonlinearly smoothed em algorithm for the non mixture setting we also demonstrate via simulation studies that the new algorithm gives very similar results to another algorithm that has been shown empirically to be effective but that does not satisfy any descent property we provide code for implementing the new algorithm in a publicly available r package

nonlinearly smoothed em density estimation with automated smoothing parameter selection for nonparametric deconvolution problems abstract we study a nonparametric deconvolution density estimation problem the estimator is obtained by an em algorithm for a smoothed maximum likelihood estimation problem which has a unique continuous solution we present an implementation of the procedure incorporating a data driven discrepancy principle for selecting the smoothing parameter simulations illustrate the good properties of the resulting estimator when the unknown distribution is smooth and has regularly varying thin tails comparisons with a fourier kernel deconvolution method are made for the case of normal noise we show that under mild smoothness conditions the estimator based on the data driven smoothing parameter is strongly consistent

local likelihood density estimation local likelihood was introduced by tibshirani and hastie as a method of smoothing by local polynomials in non gaussian regression models in this paper an extension of these methods to density estimation is discussed and comparison with other methods of density estimation presented the local likelihood method has particularly strong advantages over kernel methods when estimating tails of densities and in multivariate settings suppose constraints are incorporated in a simple manner asymptotic properties of the estimate are discussed a method for computing the estimate is outlined c code to implement the estimation procedure described in this paper together with s interfaces for graphical display of results are available at http cm bell labs com stat project locfit index html

smoothing spline density estimation theory in this article a class of penalized likelihood probability density estimators is proposed and studied the true log density is assumed to be a member of a reproducing kernel hilbert space on a finite domain not necessarily univariate and the estimator is defined as the unique unconstrained minimizer of a penalized log likelihood functional in such a space under mild conditions the existence of the estimator and the rate of convergence of the estimator in terms of the symmetrized kullback leibler distance are established to make the procedure applicable a semiparametric approximation of the estimator is presented which sits in an adaptive finite dimensional function space and hence can be computed in principle the theory is developed in a generic setup and the proofs are largely elementary algorithms are yet to follow

item response theory with estimation of the latent population distribution using spline based densities the purpose of this paper is to introduce a new method for fitting item response theory models with the latent population distribution estimated from the data using splines a spline based density estimation system provides a flexible alternative to existing procedures that use a normal distribution or a different functional form for the population distribution a simulation study shows that the new procedure is feasible in practice and that when the latent distribution is not well approximated as normal two parameter logistic 2pl item parameter estimates and expected a posteriori scores eaps can be improved over what they would be with the normal model an example with real data compares the new method and the extant empirical histogram approach

inference for density families using functional principal component analysis we consider t 1 t samples of iid observations x1t xntt from unknown population densities ft to characterize differences and similarities of ft we assume their expansions into the first l principal components from the given observations xit we study inference on the components and on their required number l a detailed asymptotic theory is presented our method is applied in the analysis of yearly cross sectional samples of british households interpretation of the estimated principal components and their scores provides new insights into the evolution and interplay of household income and age distributions from 1968 1988 from estimating their required numbersl we draw conclusions on the dimensionality of mixture models for describing the densities

nonparametric spectral density estimation using penalized whittle likelihood abstract the penalized likelihood approach is not well developed in time series analysis even though it has been applied successfully in a number of nonparametric function estimation problems chow and grenander proposed a penalized likelihood type approach to the nonparametric estimation of the spectral density of gaussian processes in this article this estimator is extended to more general stationary processes its practical implementation is developed in some detail and some asymptotic rates of convergence are established its performance is also compared to more widely used alternatives in the field a computational algorithm involving an iterative least squares initialized by the log periodogram is first developed then motivated by an asymptotic linearization an estimator of the integrated squared error between the estimated and true log spectral densities is proposed from this a data dependent procedure for selection of the amount of smoothing is constructed the methodology is illustrated