randomized clustering forests for image classification some of the most effective recent methods for content based image classification work by quantizing image descriptors and accumulating histograms of the resulting visual word codes large numbers of descriptors and large codebooks are required for good results and this becomes slow using k means we introduce extremely randomized clustering forests ensembles of randomly created clustering trees and show that they provide more accurate results much faster training and testing and good resistance to background clutter second an efficient image classification method is proposed it combines erc forests and saliency maps very closely with the extraction of image information for a given image a classifier builds a saliency map online and uses it to classify the image we show in several state of the art image classification tasks that this method can speed up the classification process enormously finally we show that the proposed erc forests can also be used very successfully for learning distance between images the distance computation algorithm consists of learning the characteristic differences between local descriptors sampled from pairs of same or different objects these differences are vector quantized by erc forests and the similarity measure is computed from this quantization the similarity measure has been evaluated on four very different datasets and always outperforms the state of the art competitive approaches

the supervised hierarchical dirichlet process we propose the supervised hierarchical dirichlet process shdp a nonparametric generative model for the joint distribution of a group of observations and a response variable directly associated with that whole group we compare the shdp with another leading method for regression on grouped data the supervised latent dirichlet allocation slda model we evaluate our method on two real world classification problems and two real world regression problems bayesian nonparametric regression models based on the dirichlet process such as the dirichlet process generalised linear models dp glm have previously been explored these models allow flexibility in modelling nonlinear relationships however until now hierarchical dirichlet process hdp mixtures have not seen significant use in supervised problems with grouped data since a straightforward application of the hdp on the grouped data results in learnt clusters that are not predictive of the responses the shdp solves this problem by allowing for clusters to be learnt jointly from the group structure and from the label assigned to each group

high dimensional classification via nonparametric empirical bayes and maximum likelihood inference we propose new nonparametric empirical bayes methods for high dimensional classification our classifiers are designed to approximate the bayes classifier in a hypothesized hierarchical model where the prior distributions for the model parameters are estimated nonparametrically from the training data as is common with nonparametric empirical bayes the proposed classifiers are effective in high dimensional settings even when the underlying model parameters are in fact nonrandom we use nonparametric maximum likelihood estimates of the prior distributions following the elegant approach studied by kiefer wolfowitz in the 1950s however our implementation is based on a recent convex optimization framework for approximating these estimates that is well suited for large scale problems we derive new theoretical results on the accuracy of the approximate estimator which help control the misclassification rate of one of our classifiers we show that our methods outperform several existing methods in simulations and perform well when gene expression microarray data is used to classify cancer patients

rates of convergence for bayes and maximum likelihood estimation for mixture of normal densities we study the rates of convergence of the maximum likelihood estimator mle and posterior distribution in density estimation problems where the densities are location or location scale mixtures of normal distributions with the scale parameter lying between two positive numbers the true density is also assumed to lie in this class with the true mixing distribution either compactly supported or having sub gaussian tails we obtain bounds for hellinger bracketing entropies for this class and from these bounds we deduce the convergence rates of sieve mles in hellinger distance the rate turns out to be log n n where k 1 is a constant that depends on the type of mixtures and the choice of the sieve next we consider a dirichlet mixture of normals as a prior on the unknown density we estimate the prior probability of a certain kullback leibler type neighborhood and then invoke a general theorem that computes the posterior convergence rate in terms the growth rate of the hellinger entropy and the concentration rate of the prior the posterior distribution is also seen to converge at the rate log n n in where k now depends on the tail behavior of the base measure of the dirichlet process

sequential change point detection methods for nonstationary time series we present two new spectral based methods for detection of changes in autocorrelation structure in a continuous valued time series in an online process monitoring setting our methods are based on the idea that changes in the autocorrelation structure are reflected by changes in the fourier or wavelet based spectrum and can be detected by comparing estimated spectra of adjacent blocks of the series to be effective for slowly changing spectral structure the methods are extended to allow information from more than one past block to be used in determining whether a change has occurred in such a way as to minimize computational burden through simulation we evaluate the performance of our methods and find that they can provide reliable and timely detection of changes in covariance structure in an online monitoring framework we illustrate the methods using electroencephalogram traces brain waves and run time computer performance metrics

adaptive nonparametric bayesian inference using location scale mixture priors we study location scale mixture priors for nonparametric statistical problems including multivariate regression density estimation and classification we show that a rate adaptive procedure can be obtained if the prior is properly constructed in particular we show that adaptation is achieved if a kernel mixture prior on a regression function is constructed using a gaussian kernel an inverse gamma bandwidth and gaussian mixing weights

aggregation of predictors for non stationary sub linear processes and online adaptive forecasting of time varying autoregressive processes in this work we study the problem of aggregating a finite number of predictors for non stationary sub linear processes we provide oracle inequalities relying essentially on three ingredients 1 a uniform bound of the ell 1 norm of the time varying sub linear coefficients 2 a lipschitz assumption on the predictors and 3 moment conditions on the noise appearing in the linear representation two kinds of aggregations are considered giving rise to different moment conditions on the noise and more or less sharp oracle inequalities we apply this approach for deriving an adaptive predictor for locally stationary time varying autoregressive tvar processes it is obtained by aggregating a finite number of well chosen predictors each of them enjoying an optimal minimax convergence rate under specific smoothness conditions on the tvar coefficients we show that the obtained aggregated predictor achieves a minimax rate while adapting to the unknown smoothness to prove this result a lower bound is established for the minimax rate of the prediction risk for the tvar process numerical experiments complete this study an important feature of this approach is that the aggregated predictor can be computed recursively and is thus applicable in an online prediction context

model based gaussian and non gaussian clustering abstract the classification maximum likelihood approach is sufficiently general to encompass many current clustering algorithms including those based on the sum of squares criterion and on the criterion of friedman and rubin 1967 however as currently implemented it does not allow the specification of which features orientation size and shape are to be common to all clusters and which may differ between clusters also it is restricted to gaussian distributions and it does not allow for noise we propose ways of overcoming these limitations a reparameterization of the covariance matrix allows us to specify that some features but not all be the same for all clusters a practical framework for non gaussian clustering is outlined and a means of incorporating noise in the form of a poisson process is described an approximate bayesian method for choosing the number of clusters is given the performance of the proposed methods is studied by simulation with encouraging results the methods are applied to the analysis of a data set arising in the study of diabetes and the results seem better than those of previous analyses rh

map estimation of finite gray scale digital images corrupted by supremum infimum noise finite gray scale digital images are modeled as realizations of discrete random functions drf and then the estimation of realizations of drf corrupted by a supremum infimum noise model is considered it is proved that morphological operators such as openings closings supremum of openings and infimum of closings are optimal maximum a posteriori map estimators under an appropriate and minimal set of assumptions relating to the structural and statistical constraints on image drf and noise drf these results are obtained for independent identically distributed i i d noise for single and multiframe observation scenarios next the assumption of i i d noise is relaxed and the map optimality and strong consistency of morphological filters for filtering image drf degraded by morphologically smooth noise i e colored noise is proved simulations on actual image data are carried out in support of the validity of theoretical results presented

theoretical aspects of vertically invariant gray level morphological operators and their application on adaptive signal and image filtering we use vertically invariant morphological filters for time varying or adaptive signal processing the morphological filters adopted in this paper are vertically invariant openings and closings vertically invariant openings and closings have intuitive geometric interpretations and can provide different filtering scales with respect to different spatial positions hence they are suitable for adaptive signal filtering to adaptively assign structuring elements of the vertically invariant openings or closings we develop the progressive umbra filling puf procedure experimental results have shown that our approach can successfully eliminate noise without oversmoothing the important features of a signal

boosted random contextual semantic space based representation for visual recognition we propose an image representation method using boosted random contextual semantic spaces the proposed brcss can alleviate the semantic gap by using semantic space based image representation brcss uses the visual representation of images in an iterative way with random sampling and re weighting experimental results demonstrate the effectiveness and efficiency of the proposed method visual information has been widely used for image representation although proven very effective the visual representation lacks explicit semantics however how to generate a proper semantic space for image representation is still an open problem that needs to be solved to jointly model the visual and semantic representations of images we propose a boosted random contextual semantic space based image representation method images are initially represented using local feature s distribution histograms the semantic space is generated by randomly selecting training images images are then mapped into the semantic space accordingly semantic context is explored to model the correlations of different semantics which is then used for classification the classification results are used to re weight training images in a boosted way the re weighted images are used to construct new semantic space for classification in this way we are able to jointly consider the visual and semantic information of images image classification experiments on several public datasets show the effectiveness of the proposed method

hellinger consistency of certain nonparametric maximum likelihood estimators consider a class mathscr p p theta theta in theta of probability measures on a measurable space mathscr x mathscr a dominated by a sigma finite measure mu let f theta dp theta d mu theta in theta and let theta n be a maximum likelihood estimator based on n independent observations from p theta 0 theta 0 in theta we use results from empirical process theory to obtain convergence for the hellinger distance h f hat theta n f theta 0 under certain entropy conditions on the class of densities f theta theta in theta the examples we present are a model with interval censored observations smooth densities monotone densities and convolution models in most examples the convexity of the class of densities is of special importance

morphological filters part i their set theoretic analysis and relations to linear shift invariant filters this paper examines the set theoretic interpretation of morphological filters in the framework of mathematical morphology and introduces the representation of classical linear filters in terms of morphological correlations which involve supremum infimum operations and additions binary signals are classified as sets and multilevel signals as functions two set theoretic representations of signals are reviewed filters are classified as set processing sp or function processing fp conditions are provided for certain fp filters that pass binary signals to commute with signal thresholding because then they can be analyzed and implemented as sp filters the basic morphological operations of set erosion dilation opening and closing are related to minkowski set operations and are used to construct fp morphological filters emphasis is then given to analytically and geometrically quantifying the similarities and differences between morphological filtering of signals by sets and functions the latter case allows the definition of morphological convolutions and correlations toward this goal various properties of fp morphological filters are also examined linear shift invariant filters due to their translation invariance are uniquely characterized by their kernel which is a special collection of input signals increasing linear filters are represented as the supremum of erosions by their kernel functions if the filters are also discrete and have a finite extent impulse response they can be represented as the supremum of erosions only by their minimal with respect to a signal ordering kernel functions stable linear filters can be represented as the sum of at most two weighted suprema of erosions these results demonstrate the power of mathematical morphology as a unifying approach to both linear and nonlinear signal shaping strategies

posterior consistency of dirichlet mixtures in density estimation a dirichlet mixture of normal densities is a useful choice for a prior distribution on densities in the problem of bayesian density estimation in the recent years efficient markov chain monte carlo method for the computation of the posterior distribution has been developed the method has been applied to data arising from different fields of interest the important issue of consistency was however left open in this paper we settle this issue in affirmative

image classification using boosted local features with random orientation and location selection we propose an image classification method using boosted randomly selected local features we jointly consider local feature extraction codebook generation and classifier training the proposed method trains a series of classifiers using the boosting strategy experimental results demonstrate effectiveness and efficiency of the proposed method the combination of local features with sparse technique has improved image classification performance dramatically in recent years although very effective this strategy still has two shortcomings first local features are often extracted in a pre defined way e g sift with dense sampling without considering the classification task second the codebook is generated by sparse coding or its variants by minimizing the reconstruction error which has no direct relationships with the classification process to alleviate the two problems we propose a novel boosted local features method with random orientation and location selection we first extract local features with random orientation and location using a weighting strategy this randomization process makes us to extract more types of information for image representation than pre defined methods these extracted local features are then encoded by sparse representation instead of generating the codebook in a single process we construct a series of codebooks and the corresponding encoding parameters of local features using a boosting strategy the weights of local features are determined by the classification performances of learned classifiers in this way we are able to combine the local feature extraction and encoding with classifier training into a unified framework and gradually improve the image classification performance experiments on several public image datasets prove the effectiveness and efficiency of the proposed method

statistical analysis of morphological openings statistical analysis of morphological openings is carried out to study noise suppression and edge preserving properties of binary and gray scale structuring elements based on the fact that basis functions are a general representation of any morphological mapping that is translation invariant and increasing it is shown that a statistical analysis using this representation is feasible and more general than the threshold decomposition approach

pattern spectrum and multiscale shape representation the results of a study on multiscale shape description smoothing and representation are reported multiscale nonlinear smoothing filters are first developed using morphological opening and closings g matheron 1975 used openings and closings to obtain probabilistic size distributions of euclidean space sets continuous binary images these distributions are used to develop a concept of pattern spectrum a shape size descriptor a pattern spectrum is introduced for continuous graytone images and arbitrary multilevel signals as well as for discrete images by developing a discrete size family of patterns large jumps in the pattern spectrum at a certain scale indicate the existence of major protruding or intruding substructures of the signal at the scale an entropy like shape size complexity measure is also developed based on the pattern spectrum for shape representation a reduced morphological skeleton transform is introduced for discrete binary and graytone images this transform is a sequence of skeleton components sparse images which represent the original shape at various scales it is shown that the partially reconstructed images from the inverse transform on subsequences of skeleton components are the openings of the image at a scale determined by the number of eliminated components in addition two way correspondences are established among the degree of shape smoothing via multiscale openings or closings the pattern spectrum zero values and the elimination or nonexistence of skeleton components at certain scales

rates of contraction of posterior distributions based on gaussian process priors we derive rates of contraction of posterior distributions on nonparametric or semiparametric models based on gaussian processes the rate of contraction is shown to depend on the position of the true parameter relative to the reproducing kernel hilbert space of the gaussian process and the small ball probabilities of the gaussian process we determine these quantities for a range of examples of gaussian priors and in several statistical settings for instance we consider the rate of contraction of the posterior distribution based on sampling from a smooth density model when the prior models the log density as a fractionally integrated brownian motion we also consider regression with gaussian errors and smooth classification under a logistic or probit link function combined with various priors

statistical behavior and consistency of classification methods based on convex risk minimization we study how closely the optimal bayes error rate can be approximately reached using a classification algorithm that computes a classifier by minimizing a convex upper bound of the classification error function the measurement of closeness is characterized by the loss function used in the estimation we show that such a classification scheme can be generally regarded as a nonmaximum likelihood conditional in class probability estimate and we use this analysis to compare various convex loss functions that have appeared in the literature furthermore the theoretical insight allows us to design good loss functions with desirable properties another aspect of our analysis is to demonstrate the consistency of certain classification methods using convex risk minimization this study sheds light on the good performance of some recently proposed linear classification methods including boosting and support vector machines it also shows their limitations and suggests possible improvements

randomboost simplified multiclass boosting through randomization we propose a novel boosting approach to multiclass classification problems in which multiple classes are distinguished by a set of random projection matrices in essence the approach uses random projections to alleviate the proliferation of binary classifiers typically required to perform multiclass classification the result is a multiclass classifier with a single vector valued parameter irrespective of the number of classes involved two variants of this approach are proposed the first method randomly projects the original data into new spaces while the second method randomly projects the outputs of learned weak classifiers these methods are not only conceptually simple but also effective and easy to implement a series of experiments on synthetic machine learning and visual recognition data sets demonstrate that our proposed methods could be compared favorably with existing multiclass boosting algorithms in terms of both the convergence rate and classification accuracy

image classification by search with explicitly and implicitly semantic representations we model the explicitly and implicitly semantic representations for image classification explicitly semantic representations are obtained using learned models with supervision implicitly semantic representations are obtained with information from the internet the explicitly and implicitly semantic based method achieves good performances image classification refers to the task of automatically classifying the categories of images based on the contents this task is typically solved using visual features with the histogram based classification scheme although effective this strategy has two drawbacks on one hand histogram based representation often disregards the object layout which is very important for classification on the other hand visual features are unable to fully separate different images due to the semantic gap to solve these two problems in this paper we propose a novel image classification method by explicitly and implicitly representing the images with searching strategy first to make use of object layouts we randomly select a number of regions and then use these regions for image representations second we generate the explicitly semantic representations using a number of pre learned semantic models third we measure the visual similarities with the internet images and use the text information for implicitly semantic representations since internet images are contaminated with noise the resulting representations only implicitly reflect the contents of images finally both the explicitly and implicitly semantic representations are jointly modeled for image classifications by training bi linear classifiers we evaluate the effectiveness of the proposed image classification by search with explicitly and implicitly semantic representations method eisr on the scene 15 dataset the mit indoor dataset the uiuc sports dataset and the pascal voc 2007 dataset the experimental results prove the usefulness of the proposed method

convergence rate of sieve estimates in this paper we develop a general theory for the convergence rate of sieve estimates maximum likelihood estimates mle s and related estimates obtained by optimizing certain empirical criteria in general parameter spaces in many cases especially when the parameter space is infinite dimensional maximization over the whole parameter space is undesirable in such cases one has to perform maximization over an approximating space sieve of the original parameter space and allow the size of the approximating space to grow as the sample size increases this method is called the method of sieves in the case of the maximum likelihood estimation an mle based on a sieve is called a sieve mle we found that the convergence rate of a sieve estimate is governed by a the local expected values variances and l 2 entropy of the criterion differences and b the approximation error of the sieve a robust nonparametric regression problem a mixture problem and a nonparametric regression problem are discussed as illustrations of the theory we also found that when the underlying space is too large the estimate based on optimizing over the whole parameter space may not achieve the best possible rates of convergence whereas the sieve estimate typically does not suffer from this difficulty

on rates of convergence for posterior distributions in infinite dimensional models this paper introduces a new approach to the study of rates of convergence for posterior distributions it is a natural extension of a recent approach to the study of bayesian consistency in particular we improve on current rates of convergence for models including the mixture of dirichlet process model and the random bernstein polynomial model

rates of contraction for posterior distributions in lr metrics 1 r the frequentist behavior of nonparametric bayes estimates more specifically rates of contraction of the posterior distributions to shrinking lr norm neighborhoods 1 r of the unknown parameter are studied a theorem for nonparametric density estimation is proved under general approximation theoretic assumptions on the prior the result is applied to a variety of common examples including gaussian process wavelet series normal mixture and histogram priors the rates of contraction are minimax optimal for 1 r 2 but deteriorate as r increases beyond 2 in the case of gaussian nonparametric regression a gaussian prior is devised for which the posterior contracts at the optimal rate in all lr norms 1 r

an efficient calculation of the moments of matched and mismatched hidden markov models streit 1990 analysed the classification of an unknown hidden markov model hmm using a set of prescribed hmms he proposed a suboptimal test statistic that can be approximated by certain moments for this classification in the present paper the algorithm given by streit to derive these moments is reformulated in a matrix algebra setting that gives a better insight into the algorithm also an asymptotic analysis of the algorithm is derived using the reformulation

a study of type i intermittency of a circular differential equation under a discontinuous right hand side abstract in this paper we study a circular differential equation under a discontinuous periodic input developing a quadratic differential equations system on s 1 and a linear differential equations system in the minkowski space m 3 the symmetry groups of these two systems are respectively pso o 2 1 and so o 2 1 the poincare circle map is constructed exactly and a critical value c of the parameter is identified depending on of the input amplitude the equation may exhibit periodic subharmonic or quasiperiodic motions when varies from c to c there undergoes an inverse tangent bifurcation consequently the resultant poincare circle map offers one route to the quasiperiodicity via a type i intermittency in the parameter range of c the orbit generated by the poincare circle map is either m periodic or quasiperiodic when n m is a rational or an irrational number

a fast learning algorithm for deep belief nets we show how to use complementary priors to eliminate the explaining away effects that make inference difficult in densely connected belief nets that have many hidden layers using complementary priors we derive a fast greedy algorithm that can learn deep directed belief networks one layer at a time provided the top two layers form an undirected associative memory the fast greedy algorithm is used to initialize a slower learning procedure that fine tunes the weights using a contrastive version of the wake sleep algorithm after fine tuning a network with three hidden layers forms a very good generative model of the joint distribution of handwritten digit images and their labels this generative model gives better digit classification than the best discriminative learning algorithms the low dimensional manifolds on which the digits lie are modeled by long ravines in the free energy landscape of the top level associative memory and it is easy to explore these ravines by using the directed connections to display what the associative memory has in mind

semiparametric estimation of normal mixture densities a semiparametric method for estimating densities of normal mean mixtures is presented this consistent data driven method of estimation is based on probability spacings the estimation technique involves iteratively fixing the standard deviation of the normal kernel that serves as a smoothing parameter and then maximizing a function of the probability spacings over all mixing distributions based on the distribution of uniform spacings a distribution free goodness of fit criterion is developed to guide the selection of the smoothing parameter the result is a set of consistent estimators indexed by a range of smoothing parameters empirical process results are used to prove consistency

a multiscanning approach based on morphological filtering it is argued that the mathematical morphology method seems to be more reasonable and powerful in studying certain multiscaling vision problems than the approach that uses derivatives of gaussian shaped filters of different sizes to show the validity of this method the authors concentrated on an application that involves forming scale space image of a 2 d shape using morphological opening filtering a proof is given to show that morphological opening filtering has a property of not introducing additional zero crossings as one moves to a coarser scale this is a different result from the conclusion by a l yuille and t a poggio ibid vol pami 8 jan 1986 that the gaussian filter is the only filter with this property in addition opening filtering is computationaly simpler than the gaussian filter

a representation theory for morphological image and signal processing a unifying theory for many concepts and operations encountered in or related to morphological image and signal analysis is presented the unification requires a set theoretic methodology where signals are modeled as sets systems signal transformations are viewed as set mappings and translational invariant systems are uniquely characterized by special collections of input signals this approach leads to a general representation theory in which any translation invariant increasing upper semicontinuous system can be presented exactly as a minimal nonlinear superposition of morphological erosions or dilations the theory is used to analyze some special cases of image signal analysis systems such as morphological filters median and order statistic filters linear filters and shape recognition transforms although the developed theory is algebraic its prototype operations are well suited for shape analysis hence the results also apply to systems that extract information about the geometrical structure of signals

structural break estimation for nonstationary time series models this article considers the problem of modeling a class of nonstationary time series using piecewise autoregressive ar processes the number and locations of the piecewise ar segments as well as the orders of the respective ar processes are assumed unknown the minimum description length principle is applied to compare various segmented ar fits to the data the goal is to find the best combination of the number of segments the lengths of the segments and the orders of the piecewise ar processes such a best combination is implicitly defined as the optimizer of an objective function and a genetic algorithm is implemented to solve this difficult optimization problem numerical results from simulation experiments and real data analyses show that the procedure has excellent empirical properties the segmentation of multivariate time series is also considered assuming that the true underlying model is a segmented autoregression this procedure is shown to be consistent for estimating the location of

second order statistics of morphological dilation and erosion of a memoryless source the paper shows that morphological dilation and erosion preserve strict sense stationarity derives the 2 d probability distribution and density functions of dilated and eroded stationary memoryless sources and presents numerical results for a memoryless uniform source which shows identical closed form autocovariance functions of dilated and eroded sources

probability inequalities for likelihood ratios and convergence rates of sieve mles let y 1 y n be independent identically distributed with density p 0 and let f be a space of densities we show that the supremum of the likelihood ratios i 1 n p y i p 0 y i where the supremum is over p f with p l 2 p 0 1 2 2 e is exponentially small with probability exponentially close to 1 the exponent is proportional to ne 2 the only condition required for this to hold is that e exceeds a value determined by the bracketing hellinger entropy of f a similar inequality also holds if we replace f by f n and p 0 by q n where q n is an approximation to p 0 in a suitable sense these results are applied to establish rates of convergence of sieve mles furthermore weak conditions are given under which the optimal rate e n defined by h e n f ne n 2 where h f is the hellinger entropy of f is nearly achievable by sieve estimators

sparse representation for color image restoration sparse representations of signals have drawn considerable interest in recent years the assumption that natural signals such as images admit a sparse decomposition over a redundant dictionary leads to efficient algorithms for handling such sources of data in particular the design of well adapted dictionaries for images has been a major challenge the k svd has been recently proposed for this task and shown to perform very well for various grayscale image processing tasks in this paper we address the problem of learning dictionaries for color images and extend the k svd based grayscale image denoising algorithm that appears in this work puts forward ways for handling nonhomogeneous noise and missing information paving the way to state of the art results in applications such as color image denoising demosaicing and inpainting as demonstrated in this paper

statistical analysis of space varying morphological openings with flat structuring elements the statistical behavior of a special type of morphological operation the space varying opening is analyzed and investigated the major distinction between the space invariant morphological operation and the space varying one is that the structuring element corresponding to each position in the space invariant operation is the same but the structuring element corresponding to each position in the space varying operation may be different a regular statistical analysis method which was originally designed for the analysis of the space invariant opening is applied and generalized to the analysis of the space varying one for the filtering of a noisy step edge some comparative analytic results are obtained revealing that edge preserving ability can be improved by the use of the space varying opening

renewal sequences and intermittency in this paper we examine the generating function z of a renewal sequence arising from the distribution of return times in the turbulent region for a class of piecewise affine interval maps introduced by gaspard and wang and studied by several authors we prove that it admits a meromorphic continuation to the entire complex z plane with a branch cut along the ray 1 moreover we compute the asymptotic behavior of the coefficients of its taylor expansion at z 0 from this we obtain the exact polynomial asympotics for the rate of mixing when the invariant measure is finite and of the scaling rate when it is infinite

on choosing mixture components via non local priors choosing the number of mixture components remains an elusive challenge model selection criteria can be either overly liberal or conservative and return poorly separated components of limited practical use we formalize non local priors nlps for mixtures and show how they lead to well separated components with non negligible weight interpretable as distinct subpopulations we also propose an estimator for posterior model probabilities under local priors and nlps showing that bayes factors are ratios of posterior to prior empty cluster probabilities the estimator is widely applicable and helps to set thresholds to drop unoccupied components in overfitted mixtures we suggest default prior parameters based on multimodality for normal t mixtures and minimal informativeness for categorical outcomes we characterize theoretically the nlp induced sparsity derive tractable expressions and algorithms we fully develop normal binomial and product binomial mixtures but the theory computation and principles hold more generally we observed a serious lack of sensitivity of the bayesian information criterion insufficient parsimony of the akaike information criterion and a local prior and a mixed behaviour of the singular bayesian information criterion we also considered overfitted mixtures their performance was competitive but depended on tuning parameters under our default prior elicitation nlps offered a good compromise between sparsity and power to detect meaningfully separated components

modeling unobserved sources of heterogeneity in animal abundance using a dirichlet process prior in surveys of natural populations of animals a sampling protocol is often spatially replicated to collect a representative sample of the population in these surveys differences in abundance of animals among sample locations may induce spatial heterogeneity in the counts associated with a particular sampling protocol for some species the sources of heterogeneity in abundance may be unknown or unmeasurable leading one to specify the variation in abundance among sample locations stochastically however choosing a parametric model for the distribution of unmeasured heterogeneity is potentially subject to error and can have profound effects on predictions of abundance at unsampled locations in this article we develop an alternative approach wherein a dirichlet process prior is assumed for the distribution of latent abundances this approach allows for uncertainty in model specification and for natural clustering in the distribution of abundances in a data adaptive way we apply this approach in an analysis of counts based on removal samples of an endangered fish species the okaloosa darter results of our data analysis and simulation studies suggest that our implementation of the dirichlet process prior has several attractive features not shared by conventional fully parametric alternatives

a dirichlet process gaussian state machine model for change detection in transient processes abstractthe ability to detect incipient and critical changes in real world process esessential for system integrity assurance is currently impeded by the mismatch between the key assumption of stationarity underlying most change detection methods and the nonlinear and nonstationary transient dynamics of most real world processes the current approaches are slow or outright unable to detect qualitative changes in the behaviors that lead to anomalies we present a dirichlet process gaussian state machine dpgsm model to represent dynamic intermittency which is one of the most ubiquitous real world transient behaviors the dpgsm model treats a signal as a random walk among a dirichlet process mixture of gaussian clusters hypothesis tests and a numerical scheme based on this nonparametric representation were developed to detect subtle changes in the transient intermittent dynamics experimental investigations suggest that the dpgsm approach can consistently detect incipient critical changes in intermi

general maximum likelihood empirical bayes estimation of normal means we propose a general maximum likelihood empirical bayes gmleb method for the estimation of a mean vector based on observations with i i d normal errors we prove that under mild moment conditions on the unknown means the average mean squared error mse of the gmleb is within an infinitesimal fraction of the minimum average mse among all separable estimators which use a single deterministic estimating function on individual observations provided that the risk is of greater order than log n 5 n we also prove that the gmleb is uniformly approximately minimax in regular and weak l p balls when the order of the length normalized norm of the unknown means is between log n k1 n 1 p 2 and n log n k2 simulation experiments demonstrate that the gmleb outperforms the james stein and several state of the art threshold estimators in a wide range of settings without much down side

adaptive bayesian multivariate density estimation with dirichlet mixtures summary we show that rate adaptive multivariate density estimation can be performed using bayesian methods based on dirichlet mixtures of normal kernels with a prior distribution on the kernel s covariance matrix parameter we derive sufficient conditions on the prior specification that guarantee convergence to a true density at a rate that is minimax optimal for the smoothness class to which the true density belongs no prior knowledge of smoothness is assumed the sufficient conditions are shown to hold for the dirichlet location mixture of normals prior with a gaussian base measure and an inverse wishart prior on the covariance matrix parameter locally h older

bayesian estimation of the spectral density of a time series this article describes a bayesian approach to estimating the spectral density of a stationary time series a nonparametric prior on the spectral density is described through bernstein polynomials because the actual likelihood is very complicated a pseudoposterior distribution is obtained by updating the prior using the whittle likelihood a markov chain monte carlo algorithm for sampling from this posterior distribution is described that is used for computing the posterior mean variance and other statistics a consistency result is established for this pseudoposterior distribution that holds for a short memory gaussian time series and under some conditions on the prior to prove this asymptotic result a general consistency theorem of schwartz is extended for a triangular array of independent nonidentically distributed observations this extension is also of independent interest a simulation study is conducted to compare the proposed method with some existing methods the method is illustrated with

bayesian density estimation and inference using mixtures abstract we describe and illustrate bayesian inference in models for density estimation using mixtures of dirichlet processes these models provide natural settings for density estimation and are exemplified by special cases where data are modeled as a sample from mixtures of normal distributions efficient simulation methods are used to approximate various prior posterior and predictive distributions this allows for direct inference on a variety of practical issues including problems of local versus global smoothing uncertainty about density estimates assessment of modality and the inference on the numbers of components also convergence results are established for a general class of normal mixture models

bayesian repulsive gaussian mixture model we develop a general class of bayesian repulsive gaussian mixture models that encourage well separated clusters aiming at reducing potentially redundant components produced by independent priors f

the consistency of posterior distributions in nonparametric problems we give conditions that guarantee that the posterior probability of every hellinger neighborhood of the true distribution tends to 1 almost surely the conditions are 1 a requirement that the prior not put high mass near distributions with very rough densities and 2 a requirement that the prior put positive mass in kullback leibler neighborhoods of the true distribution the results are based on the idea of approximating the set of distributions with a finite dimensional set of distributions with sufficiently small hellinger bracketing metric entropy we apply the results to some examples

image level classification by hierarchical structure learning with visual and semantic similarities abstract image classification methods often use class level information without considering the distinctive character of each image images of the same class may have varied appearances besides visually similar images may not be semantically correlated to solve these problems in this paper we propose a novel image classification method by automatically learning the image level hierarchical structure ilhs using both visual and semantic similarities we try to generate new representations by exploring both visual and semantic similarities of images images are clustered hierarchically to explore their correlations we then use them for image representations the diversity of image classes within each cluster is used to re weight visual similarities the re weighted similarities are aggregated to generate new image representations we conduct image classification experiments on the caltech 256 dataset the pascal voc 2007 dataset and the pascal voc 2012 dataset experimental results demonstrate the effectiveness of the proposed method

practical bayesian density estimation using mixtures of normals abstract mixtures of normals provide a flexible model for estimating densities in a bayesian framework there are some difficulties with this model however first standard reference priors yield improper posteriors second the posterior for the number of components in the mixture is not well defined if the reference prior is used third posterior simulation does not provide a direct estimate of the posterior for the number of components we present some practical methods for coping with these problems finally we give some results on the consistency of the method when the maximum number of components is allowed to grow with the sample size

posterior convergence rates of dirichlet mixtures at smooth densities we study the rates of convergence of the posterior distribution for bayesian density estimation with dirichlet mixtures of normal distributions as the prior the true density is assumed to be twice continuously differentiable the bandwidth is given a sequence of priors which is obtained by scaling a single prior by an appropriate order in order to handle this problem we derive a new general rate theorem by considering a countable covering of the parameter space whose prior probabilities satisfy a summability condition together with certain individual bounds on the hellinger metric entropy we apply this new general theorem on posterior convergence rates by computing bounds for hellinger bracketing entropy numbers for the involved class of densities the error in the approximation of a smooth density by normal mixtures and the concentration rate of the prior the best obtainable rate of convergence of the posterior turns out to be equivalent to the well known frequentist rate for integrated mean squared error n 2 5 up to a logarithmic factor

quickest change detection of a markov process across a sensor array recent attention in quickest change detection in the multisensor setting has been on the case where the densities of the observations change at the same instant at all the sensors due to the disruption in this work a more general scenario is considered where the change propagates across the sensors and its propagation can be modeled as a markov process a centralized bayesian version of this problem is considered with a fusion center that has perfect information about the observations and a priori knowledge of the statistics of the change process the problem of minimizing the average detection delay subject to false alarm constraints is formulated in a dynamic programming framework insights into the structure of the optimal stopping rule are presented in the limiting case of rare disruptions it is shown that the structure of the optimal test reduces to thresholding the a posteriori probability of the hypothesis that no change has happened under a certain condition on the kullback leibler k l divergence between the post and the pre change densities it is established that the threshold test is asymptotically optimal in the vanishing false alarm probability regime it is shown via numerical studies that this low complexity threshold test results in a substantial improvement in performance over naive tests such as a single sensor test or a test that incorrectly assumes that the change propagates instantaneously

fitting timeseries by continuous time markov chains a quadratic programming approach construction of stochastic models that describe the effective dynamics of observables of interest is an useful instrument in various fields of application such as physics climate science and finance we present a new technique for the construction of such models from the timeseries of an observable we construct a discrete in time markov chain and calculate the eigenspectrum of its transition probability or stochastic matrix as a next step we aim to find the generator of a continuous time markov chain whose eigenspectrum resembles the observed eigenspectrum as closely as possible using an appropriate norm the generator is found by solving a minimization problem the norm is chosen such that the object function is quadratic and convex so that the minimization problem can be solved using quadratic programming techniques the technique is illustrated on various toy problems as well as on datasets stemming from simulations of molecular dynamics and of atmospheric flows

morphological filters part ii their relations to median order statistic and stack filters this paper extends the theory of median order statistic os and stack filters by using mathematical morphology to analyze them and by relating them to those morphological erosions dilations openings closings and open closings that commute with thresholding the max min representation of os filters is introduced by showing that any median or other os filter is equal to a maximum of erosions moving local minima and also to a minimum of dilations moving local maxima thus os filters can be computed by a closed formula that involves a max min on prespecified sets of numbers and no sorting stack filters are established as the class of filters that are composed exactly of a finite number of max min operations the kernels of median os and stack filters are collections of input signals that uniquely represent these filters due to their translation invariance the max min functional definitions of these nonlinear iliters is shown to be equivalent to a maximum of erosions by minimal with respect to a signal ordering kernel elements and also to a minimum of dilations by minimal kernel elements of dual filters the representation of stack filters based on their minimal kernel elements is proven to be equivalent to their representation based on irreducible sum of products expressions of boolean functions it is also shown that median filtering and its iterations of any signal by convex 1 d windows is bounded below by openings and above by closings a signal is a root fixed point of the median iff it is a root of both an opening and a closing the open closing and clos opening yield median roots in one pass suppress impulse noise similarly to the median can discriminate between positive and negative noise impulses and are computationally less complex than the median some similar results are obtained for 2 d median filtering

asymptotics of discrete mdl for online prediction minimum description length mdl is an important principle for induction and prediction with strong relations to optimal bayesian learning this paper deals with learning processes which are independent and identically distributed i i d by means of two part mdl where the underlying model class is countable we consider the online learning framework i e observations come in one by one and the predictor is allowed to update its state of mind after each time step we identify two ways of predicting by mdl for this setup namely a static and a dynamic one a third variant hybrid mdl will turn out inferior we will prove that under the only assumption that the data is generated by a distribution contained in the model class the mdl predictions converge to the true values almost surely this is accomplished by proving finite bounds on the quadratic the hellinger and the kullback leibler loss of the mdl learner which are however exponentially worse than for bayesian prediction we demonstrate that these bounds are sharp even for model classes containing only bernoulli distributions we show how these bounds imply regret bounds for arbitrary loss functions our results apply to a wide range of setups namely sequence prediction pattern classification regression and universal induction in the sense of algorithmic information theory among others

multivariate online kernel density estimation with gaussian kernels we propose a novel approach to online estimation of probability density functions which is based on kernel density estimation kde the method maintains and updates a non parametric model of the observed data from which the kde can be calculated we propose an online bandwidth estimation approach and a compression revitalization scheme which maintains the kde s complexity low we compare the proposed online kde to the state of the art approaches on examples of estimating stationary and non stationary distributions and on examples of classification the results show that the online kde outperforms or achieves a comparable performance to the state of the art and produces models with a significantly lower complexity while allowing online adaptation

convergence rates of posterior distributions we consider the asymptotic behavior of posterior distributions and bayes estimators for infinite dimensional statistical models we give general results on the rate of convergence of the posterior measure these are applied to several examples including priors on finite sieves log spline models dirichlet processes and interval censoring

nonparametric maximum likelihood estimation by the method of sieves maximum likelihood estimation often fails when the parameter takes values in an infinite dimensional space for example the maximum likelihood method cannot be applied to the completely nonparametric estimation of a density function from an operatorname iid sample the maximum of the likelihood is not attained by any density in this example as in many other examples the parameter space positive functions with area one is too big but the likelihood method can often be salvaged if we first maximize over a constrained subspace of the parameter space and then relax the constraint as the sample size grows this is grenander s method of sieves application of the method sometimes leads to new estimators for familiar problems or to a new motivation for an already well studied technique we will establish some general consistency results for the method and then we will focus on three applications

asymptotic convergence rate of the em algorithm for gaussian mixtures it is well known that the convergence rate of the expectation maximization em algorithm can be faster than those of convention first order iterative algorithms when the overlap in the given mixture is small but this argument has not been mathematically proved yet this article studies this problem asymptotically in the setting of gaussian mixtures under the theoretical framework of xu and jordan 1996 it has been proved that the asymptotic convergence rate of the em algorithm for gaussian mixtures locally around the true solution is o e0 5 e where e 0 is an arbitrarily small number o x means that it is a higher order infinitesimal as x 0 and e is a measure of the average overlap of gaussians in the mixture in other words the large sample local convergence rate for the em algorithm tends to be asymptotically superlinear when e tends to zero

rates of convergence for the gaussian mixture sieve gaussian mixtures provide a convenient method of densityestimation that lies somewhere between parametric models and kernel densityestimators when the number of components of the mixture is allowed to increase as sample size increases the model is called a mixture sieve we establish a bound on the rate of convergence in hellinger distance for densityestimation using the gaussian mixture sieve assuming that the true density is itself a mixture of gaussians the underlying mixing measure of the true densityis not necessarilyassumed to have finite support computing the rate involves some delicate calculations since the size of the sieve as measured bybracketing entropy and the saturation rate cannot be found using standard methods when the mixing measure has compact support using kn n 2 3 log n 1 3 components in the mixture yields a rate of order log n 1 6 n1 6 for every 0 the rates depend heavilyon the tail behavior of the true density the sensitivity to the tail behavior is diminished byusing a robust sieve which includes a long tailed component in the

convergence rates of posterior distributions for non i i d observations we consider the asymptotic behavior of posterior distributions and bayes estimators based on observations which are required to be neither independent nor identically distributed we give general results on the rate of convergence of the posterior measure relative to distances derived from a testing criterion we then specialize our results to independent nonidentically distributed observations markov processes stationary gaussian time series and the white noise model we apply our general results to several examples of infinite dimensional statistical models including nonparametric regression with normal errors binary regression poisson regression an interval censoring model whittle estimation of the spectral density of a time series and a nonlinear autoregressive model

rates of convergence of posterior distributions we compute the rate at which the posterior distribution concentrates around the true parameter value the spaces we work in are quite general and include infinite dimensional cases the rates are driven by two quantities the size of the space as measured by bracketing entropy and the degree to which the prior concentrates in a small ball around the true parameter we consider two examples