the supervised hierarchical dirichlet process we propose the supervised hierarchical dirichlet process shdp a nonparametric generative model for the joint distribution of a group of observations and a response variable directly associated with that whole group we compare the shdp with another leading method for regression on grouped data the supervised latent dirichlet allocation slda model we evaluate our method on two real world classification problems and two real world regression problems bayesian nonparametric regression models based on the dirichlet process such as the dirichlet process generalised linear models dp glm have previously been explored these models allow flexibility in modelling nonlinear relationships however until now hierarchical dirichlet process hdp mixtures have not seen significant use in supervised problems with grouped data since a straightforward application of the hdp on the grouped data results in learnt clusters that are not predictive of the responses the shdp solves this problem by allowing for clusters to be learnt jointly from the group structure and from the label assigned to each group

order based dependent dirichlet processes in this article we propose a new framework for bayesian nonparametric modeling with continuous covariates in particular we allow the nonparametric distribution to depend on covariates through ordering the random variables building the weights in the stick breaking representation we focus mostly on the class of random distributions that induces a dirichlet process at each covariate value we derive the correlation between distributions at different covariate values and use a point process to implement a practically useful type of ordering two main constructions with analytically known correlation structures are proposed practical and efficient computational methods are introduced we apply our framework through mixtures of these processes to regression modeling the modeling of stochastic volatility in time series data and spatial geostatistical modeling

on the number of limit cycles for a class of discontinuous quadratic differential systems abstract the present paper is devoted to the study of the maximum number of limit cycles bifurcated from the periodic orbits of the quadratic isochronous center x y 16 3 x 2 4 3 y 2 y x 8 3 x y by the averaging method of first order when it is perturbed inside a class of discontinuous quadratic polynomial differential systems the chebyshev criterion is used to show that this maximum number is 5 and can be realizable in some sense the result and that in paper 6 also answer the questions left in the paper 9

ideal spatial adaptation by wavelet shrinkage summary with ideal spatial adaptation an oracle furnishes information about how best to adapt a spatially variable estimator whether piecewise constant piecewise polynomial variable knot spline or variable bandwidth kernel to the unknown function estimation with the aid of an oracle offers dramatic advantages over traditional linear estimation by nonadaptive kernels however it is a priori unclear whether such performance can be obtained by a procedure relying on the data alone we describe a new principle for spatially adaptive estimation selective wavelet reconstruction we show that variable knot spline fits and piecewise polynomial fits when equipped with an oracle to select the knots are not dramatically more powerful than selective wavelet reconstruction with an oracle we develop a practical spatially adaptive method riskshrink which works by shrinkage of empirical wavelet coefficients riskshrink mimics the performance of an oracle for selective wavelet reconstruction as well as it is possible to do so a new inequality in multivariate normal decision theory which we call the oracle inequality shows that attained performance differs from ideal performance by at most a factor of approximately 2 log n where n is the sample size moreover no estimator can give a better guarantee than this within the class of spatially adaptive procedures riskshrink is essentially optimal relying only on the data it comes within a factor log 2 n of the performance of piecewise polynomial and variableknot spline methods equipped with an oracle in contrast it is unknown how or if piecewise polynomial methods could be made to function this well when denied access to an oracle and forced to rely on data alone

on the comparisons of decorrelation approaches for non gaussian neutral vector variables as a typical non gaussian vector variable a neutral vector variable contains nonnegative elements only and its l norm equals one in addition its neutral properties make it significantly different from the commonly studied vector variables e g the gaussian vector variables due to the aforementioned properties the conventionally applied linear transformation approaches e g principal component analysis pca and independent component analysis ica are not suitable for neutral vector variables as pca cannot transform a neutral vector variable which is highly negatively correlated into a set of mutually independent scalar variables and ica cannot preserve the bounded property after transformation in recent work we proposed an efficient nonlinear transformation approach i e the parallel nonlinear transformation pnt for decorrelating neutral vector variables in this article we extensively compare pnt with pca and ica through both theoretical analysis and experimental evaluations the results of our investigations demonstrate the superiority of pnt for decorrelating the neutral vector variables

quadratic systems with center and their perturbations abstract we study the quadratic systems on a plane z i z az2 b z 2 cz2 z x iy we give a simple algebraic proof of the center conditions 0 and b 0 or a 1 2 b 1 or a b 1 c c or a 2 b 1 c 1 and of the theorem of bautin that the number of small amplitude limit cycles bifurcating from a center or a focus is not greater than 3 we present the bifurcational diagrams and phase portraits of the systems with center and describe the cyclicity precisely in each case here one mistake in bautin s work is revealed and some questions concerning focus numbers are answered we study also global not small limit cycles for a small perturbation of the center case with two invariant lines the lotka volterra system using abelian integrals we show that the number of zeroes of the corresponding abelian integral is 0 1 or 2 some attention is devoted to the problem of limit cycles for a small perturbation of a system with a two fold center when two center conditions hold the problem turns out to be equivalent to the problem of zeroes of some integral with polynomial coefficients

estimating normal means with a dirichlet process prior abstract in this article the dirichlet process prior is used to provide a nonparametric bayesian estimate of a vector of normal means in the past there have been computational difficulties with this model this article solves the computational difficulties by developing a gibbs sampler algorithm the estimator developed in this article is then compared to parametric empirical bayes estimators peb and nonparametric empirical bayes estimators npeb in a monte carlo study the monte carlo study demonstrates that in some conditions the peb is better than the npeb and in other conditions the npeb is better than the peb the monte carlo study also shows that the estimator developed in this article produces estimates that are about as good as the peb when the peb is better and produces estimates that are as good as the npeb estimator when that method is better

the cyclicity of period annuli of a class of quintic hamiltonian systems abstract in this paper we prove that the cyclicity of period annuli for system d x d t y d y d t a x b x 3 c x 5 under perturbations of polynomials with degree n is not more than 54 n 13 taking into account the multiplicity where a b c r with c 0 such that the unperturbed systems have at least one center

generalized spatial dirichlet process models summary many models for the study of point referenced data explicitly introduce spatial random effects to capture residual spatial association these spatial effects are customarily modelled as a zeromean stationary gaussian process the spatial dirichlet process introduced by gelfand et al 2005 produces a random spatial process which is neither gaussian nor stationary rather it varies about a process that is assumed to be stationary and gaussian the spatial dirichlet process arises as a probability weighted collection of random surfaces this can be limiting for modelling and inferential purposes since it insists that a process realization must be one of these surfaces we introduce a random distribution for the spatial effects that allows different surface selection at different sites moreover we can specify the model so that the marginal distribution of the effect at each site still comes from a dirichlet process the development is offered constructively providing a multivariate extension of the stick breaking representation of the weights we then introduce mixing using this generalized spatial dirichlet process we illustrate with a simulated dataset of independent replications and note that we can embed the generalized process within a dynamic model specification to eliminate the independence assumption

bayesian semiparametric wiener system identification we present a novel method for wiener system identication the method relies on a semiparametric i e a mixed parametric nonparametric model of a wiener system we use a state space model for the linear dynamical system and a nonparametric gaussian process model for the static nonlinearity we avoid making strong assumptions such as monotonicity on the nonlinear mapping stochastic disturbances entering both as measurement noise and as process noise are handled in a systematic manner the nonparametric nature of the gaussian process allows us to handle a wide range of nonlinearities without making problem specic parameterizations we also consider sparsity promoting priors based on generalized hyperbolic distributions to automatically infer the order of the underlying dynamical system we derive an inference algorithm based on an ecient particle markov chain monte carlo method referred to as particle gibbs with ancestor sampling the method is proled on two challenging identication problems with good results blind wiener system identication is handled as a special case

online model selection based on the variational bayes the bayesian framework provides a principled way of model selection this framework estimates a probability distribution over an ensemble of models and the prediction is done by averaging over the ensemble of models accordingly the uncertainty of the models is taken into account and complex models with more degrees of freedom are penalized however integration over model parameters is often intractable and some approximation scheme is needed recently a powerful approximation scheme called the variational bayes vb method has been proposed this approach defines the free energy for a trial probability distribution which approximates a joint posterior probability distribution over model parameters and hidden variables the exact maximization of the free energy gives the true posterior distribution the vb method uses factorized trial distributions the integration over model parameters can be done analytically and an iterative expectation maximization like algorithm whose convergence is guaranteed is derived in this article we derive an online version of the vb algorithm and prove its convergence by showing that it is a stochastic approximation for finding the maximum of the free energy by combining sequential model selection procedures the online vb method provides a fully online learning method with a model selection mechanism in preliminary experiments using synthetic data the online vb method was able to adapt the model structure to dynamic environments

scale space and edge detection using anisotropic diffusion a new definition of scale space is suggested and a class of algorithms used to realize a diffusion process is introduced the diffusion coefficient is chosen to vary spatially in such a way as to encourage intraregion smoothing rather than interregion smoothing it is shown that the no new maxima should be generated at coarse scales property of conventional scale space is preserved as the region boundaries in the approach remain sharp a high quality edge detector which successfully exploits global information is obtained experimental results are shown on a number of images parallel hardware implementations are made feasible because the algorithm involves elementary local operations replicated over the image

on the consistency of bayes estimates on etudie les proprietes de frequence des regles de bayes avec une attention particuliere a la consistence

the cyclicity of period annuli of a class of quadratic reversible systems with two centers abstract quadratic perturbations of a class of quadratic reversible systems are studied the associated systems can have at most four limit cycles in any compact region of the period annuli and if four limit cycles exist then their distribution is 3 1 in other words three of them are nested and surrounding one focus the other limit cycle is surrounding the different focus

multiple imputation of missing categorical and continuous values via bayesian mixture models with local dependence abstractwe present a nonparametric bayesian joint model for multivariate continuous and categorical variables with the intention of developing a flexible engine for multiple imputation of missing values the model fuses dirichlet process mixtures of multinomial distributions for categorical variables with dirichlet process mixtures of multivariate normal distributions for continuous variables we incorporate dependence between the continuous and categorical variables by 1 modeling the means of the normal distributions as component specific functions of the categorical variables and 2 forming distinct mixture components for the categorical and continuous data with probabilities that are linked via a hierarchical model this structure allows the model to capture complex dependencies between the categorical and continuous data with minimal tuning by the analyst we apply the model to impute missing values due to item nonresponse in an evaluation of the redesign of the survey of income and program partic

limit cycles bifurcated from a class of quadratic reversible center of genus one abstract we investigate the bifurcation of limit cycles in a class of planar quadratic reversible non hamiltonian systems this systems has a center of genus one the exact upper bound of the number of limit cycles is given

multivariate adaptive regression splines a new method is presented for flexible regression modeling of high dimensional data the model takes the form of an expansion in product spline basis functions where the number of basis functions as well as the parameters associated with each one product degree and knot locations are automatically determined by the data this procedure is motivated by the recursive partitioning approach to regression and shares its attractive properties unlike recursive partitioning however this method produces continuous models with continuous derivatives it has more power and flexibility to model relationships that are nearly additive or involve interactions in at most a few variables in addition the model can be represented in a form that separately identifies the additive contributions and those associated with the different multivariable interactions

a recursive local polynomial approximation method using dirichlet clouds and radial basis functions we present a recursive function approximation technique that does not require the storage of the arrival data stream our work is motivated by algorithms in stochastic optimization which require approximating functions in a recursive setting such as a stochastic approximation algorithm the unique collection of these features in this technique is essential for nonlinear modeling of large data sets where the storage of the data becomes prohibitively expensive and in circumstances where our knowledge about a given query point increases as new information arrives the algorithm presented here employs radial basis functions rbfs to provide locally adaptive parametric models such as linear models the local models are updated using recursive least squares and only store the statistical representative of the local approximations the resulting scheme is very fast and memory efficient without compromising accuracy in comparison to methods well accepted as the standard and some advanced techniques used for fu

nonconjugate bayesian estimation of covariance matrices and its use in hierarchical models abstract the problem of estimating a covariance matrix in small samples has been considered by several authors following early work by stein this problem can be especially important in hierarchical models where the standard errors of fixed and random effects depend on estimation of the covariance matrix of the distribution of the random effects we propose a set of hierarchical priors hps for the covariance matrix that produce posterior shrinkage toward a specified structure here we examine shrinkage toward diagonality we then address the computational difficulties raised by incorporating these priors and nonconjugate priors in general into hierarchical models we apply a combination of approximation gibbs sampling possibly with a metropolis step and importance reweighting to fit the models and compare this hybrid approach to alternative markov chain monte carlo methods our investigation involves three alternative hps the first works with the spectral decomposition of the covariance matrix an

dirichlet process mixtures of generalized linear models we propose dirichlet process mixtures of generalized linear models dp glm a new class of methods for nonparametric regression given a data set of input response pairs the dp glm produces a global model of the joint distribution through a mixture of local generalized linear models dp glms allow both continuous and categorical inputs and can model the same class of responses that can be modeled with a generalized linear model we study the properties of the dp glm and show why it provides better predictions and density estimates than existing dirichlet process mixture regression models we give conditions for weak consistency of the joint distribution and pointwise consistency of the regression estimate

bounding the number of zeros of certain abelian integrals abstract in this paper we prove a criterion that provides an easy sufficient condition in order for any nontrivial linear combination of n abelian integrals to have at most n k 1 zeros counted with multiplicities this condition involves the functions in the integrand of the abelian integrals and it can be checked in many cases in a purely algebraic way

rodeo sparse greedy nonparametric regression we present a greedy method for simultaneously performing local bandwidth selection and variable selection in nonparametric regression the method starts with a local linear estimator with large bandwidths and incrementally decreases the bandwidth of variables for which the gradient of the estimator with respect to bandwidth is large the method called rodeo regularization of derivative expectation operator conducts a sequence of hypothesis tests to threshold derivatives and is easy to implement under certain assumptions on the regression function and sampling density it is shown that the rodeo applied to local linear smoothing avoids the curse of dimensionality achieving near optimal minimax rates of convergence in the number of relevant variables as if these variables were isolated in advance

complete hyperelliptic integrals of the first kind and their non oscillation let p x be a real polynomial of degree 2g 1 h y 2 p x and h be an oval contained in the level set h h we study complete abelian integrals of the form i h h 0 1 x g 1 x g 1 y dx y h where i are real and e r is a maximal open interval on which a continuous family of ovals h exists we show that the g dimensional real vector space of these integrals is not chebyshev in general for any g 1 there are hyperelliptic hamiltonians h and continuous families of ovals h h h h e such that the abelian integral i h can have at least 3 2g 1 zeros in our main result is theorem 1 in which we show that when g 2 exceptional families of ovals h exist such that the corresponding vector space is still chebyshev

bayesian covariance selection in generalized linear mixed models repeated measures and longitudinal data are commonly collected for analysis in epidemiology clinical trials biology sociology and economic sciences in such studies a response is measured repeatedly over time for each subject under study and the number and timing of the observations often varies among subjects in contrast to cross sectional studies that collect a single measurement per subject longitudinal studies have the extra complication of within subject dependence in the repeated measures such dependence can be thought to arise due to the impact of unmeasured predictors main effects of unmeasured predictors lead to variation in the average level of response among subjects while interactions with measured predictors lead to heterogeneity in the regression coefficients this justification has motivated random effects models which allow the intercept and slopes in a regression model to be subject specific random effects models are broadly useful for modeling of dependence not only for longitudinal data but also in multicenter studies meta analysis and functional data analysis

practical poissonian gaussian noise modeling and fitting for single image raw data we present a simple and usable noise model for the raw data of digital imaging sensors this signal dependent noise model which gives the pointwise standard deviation of the noise as a function of the expectation of the pixel raw data output is composed of a poissonian part modeling the photon sensing and gaussian part for the remaining stationary disturbances in the output data we further explicitly take into account the clipping of the data over and under exposure faithfully reproducing the nonlinear response of the sensor we propose an algorithm for the fully automatic estimation of the model parameters given a single noisy image experiments with synthetic images and with real raw data from various sensors prove the practical applicability of the method and the accuracy of the proposed model

bayesian nonparametric generative models for causal inference with missing at random covariates we propose a general bayesian nonparametric bnp approach to causal inference in the point treatment setting the joint distribution of the observed data outcome treatment and confounders is modeled using an enriched dirichlet process the combination of the observed data model and causal assumptions allows us to identify any type of causal effect differences ratios or quantile effects either marginally or for subpopulations of interest the proposed bnp model is well suited for causal inference problems as it does not require parametric assumptions about the distribution of confounders and naturally leads to a computationally efficient gibbs sampling algorithm by flexibly modeling the joint distribution we are also able to impute via data augmentation values for missing covariates within the algorithm under an assumption of ignorable missingness obviating the need to create separate imputed data sets this approach for imputing the missing covariates has the additional advantage of guaranteeing congeniality between the imputation model and the analysis model and because we use a bnp approach parametric models are avoided for imputation the performance of the method is assessed using simulation studies the method is applied to data from a cohort study of human immunodeficiency virus hepatitis c virus co infected patients

the infinitesimal 16th hilbert problem in the quadratic case let h x y be a real cubic polynomial with four distinct critical values in a complex domain and let x h h y frac partial partial x h x frac partial partial y be the corresponding hamiltonian vector field we show that there is a neighborhood of x h in the space of all quadratic plane vector fields such that any x has at most two limit cycles

posterior consistency of dirichlet mixtures in density estimation a dirichlet mixture of normal densities is a useful choice for a prior distribution on densities in the problem of bayesian density estimation in the recent years efficient markov chain monte carlo method for the computation of the posterior distribution has been developed the method has been applied to data arising from different fields of interest the important issue of consistency was however left open in this paper we settle this issue in affirmative

nonparametric multiscale blind estimation of intensity frequency dependent noise the camera calibration parameters and the image processing chain which generated a given image are generally not available to the receiver this happens for example with scanned photographs and for most jpeg images these images have undergone various nonlinear contrast changes and also linear and nonlinear filters to deal with remnant noise in such images we introduce a general nonparametric intensity and frequency dependent noise model we demonstrate by simulated and experiments with real images that this model which requires the estimation of more than 1000 parameters performs an efficient noise estimation the proposed noise model is a patch model its estimation can therefore be used as a preliminary step to any patch based denoising method our noise estimation method introduces several new tools for performing this complex estimation one of them is a new sparse patch distance function permitting to find noisy patches with similar underlying geometry a validation of the noise model and of its estimation method is obtained by comparing its results to ground truth noise curves for both raw and jpeg encoded images and by visual inspection of the denoising results of real images a fair comparison with the state of the art is also performed

flexible link functions in nonparametric binary regression with gaussian process priors in many scientific fields it is a common practice to collect a sequence of 0 1 binary responses from a subject across time space or a collection of covariates researchers are interested in finding out how the expected binary outcome is related to covariates and aim at better prediction in the future 0 1 outcomes gaussian processes have been widely used to model nonlinear systems in particular to model the latent structure in a binary regression model allowing nonlinear functional relationship between covariates and the expectation of binary outcomes a critical issue in modeling binary response data is the appropriate choice of link functions commonly adopted link functions such as probit or logit links have fixed skewness and lack the flexibility to allow the data to determine the degree of the skewness to address this limitation we propose a flexible binary regression model which combines a generalized extreme value link function with a gaussian process prior on the latent structure bayesian computation is employed in model estimation posterior consistency of the resulting posterior distribution is demonstrated the flexibility and gains of the proposed model are illustrated through detailed simulation studies and two real data examples empirical results show that the proposed model outperforms a set of alternative models which only have either a gaussian process prior on the latent regression function or a dirichlet prior on the link function

asymptotic normality of semiparametric and nonparametric posterior distributions in this article it is shown that the marginal semiparametric and nonparametric posterior distributions for a parameter of interest behave like an ordinary parametric posterior distribution this in practice provides support of the utility of marginal semiparametric and nonparametric posterior distributions in particular the marginal semiparametric and nonparametric posterior distributions are asymptotically normal and centered at the corresponding maximum likelihood estimates mles or posterior means with covariance matrix the inverse of the fisher information additionally the semiparametric and nonparametric mles for the parameter of interest and the marginal posterior means are asymptotically normal and centered at the true parameter with the same covariance matrix the results are a semiparametric version and a nonparametric version of the parametric bayesian central limit theorem that establish a connection between the semiparametric and the nonparametric bayesian inference and their frequentis

on rates of convergence for posterior distributions in infinite dimensional models this paper introduces a new approach to the study of rates of convergence for posterior distributions it is a natural extension of a recent approach to the study of bayesian consistency in particular we improve on current rates of convergence for models including the mixture of dirichlet process model and the random bernstein polynomial model

data driven bandwidth selection in local polynomial fitting variable bandwidth and spatial adaptation when estimating a mean regression function and its derivatives locally weighted least squares regression has proven to be a very attractive technique the present paper focuses on the important issue of how to select the smoothing parameter or bandwidth in the case of estimating curves with a complicated structure a variable bandwidth is desirable furthermore the bandwidth should be indicated by the data themselves recent developments in nonparametric smoothing techniques inspired us to propose such a data driven bandwidth selection procedure which can be used to select both constant and variable bandwidths the idea is based on a residual squares criterion along with a good approximation of the bias and variance of the estimator the procedure can be applied to select bandwidths not only for estimating the regression curve but also for estimating its derivatives the resulting estimation procedure has the necessary flexibility for capturing complicated shapes of curves this is illustrated via a large variety of testing examples including examples with a large spatial variability the results are also compared with wavelet thresholding techniques and it seems that our results are at least comparable i e local polynomial regression using our data driven variable bandwidth has spatial adaptation properties that are similar to wavelets

an anova model for dependent random measures we consider dependent nonparametric models for related random probability distributions for example the random distributions might be indexed by a categorical covariate indicating the treatment levels in a clinical trial and might represent random effects distributions under the respective treatment combinations we propose a model that describes dependence across random distributions in an analysis of variance anova type fashion we define a probability model in such a way that marginally each random measure follows a dirichlet process dp and use the dependent dirichlet process to define the desired dependence across the related random measures the resulting probability model can alternatively be described as a mixture of anova models with a dp prior on the unknown mixing measure the main features of the proposed approach are ease of interpretation and computational simplicity because the model follows the standard anovastructure interpretation and inference parallels conventions for anova mode

gaussian process models for computer experiments with qualitative and quantitative factors modeling experiments with qualitative and quantitative factors is an important issue in computer modeling we propose a framework for building gaussian process models that incorporate both types of factors the key to the development of these new models is an approach for constructing correlation functions with qualitative and quantitative factors an iterative estimation procedure is developed for the proposed models modern optimization techniques are used in the estimation to ensure the validity of the constructed correlation functions the proposed method is illustrated with an example involving a known function and a real example for modeling the thermal distribution of a data center

the cyclicity of period annuli for a class of cubic hamiltonian systems with nilpotent singular points abstract this paper deals with the limit cycles of a class of cubic hamiltonian systems under polynomial perturbations we suppose that the corresponding hamiltonian system which has at least one center has finite singular points and is symmetrical with respect to both x axis and y axis and also the origin is a nilpotent singular point hence the hamiltonian h x y can be written as h x y x 2 a x 4 b x 2 y 2 c y 4 a b c r 3 c 0 for the above h x y the corresponding vector field is x 2 y b x 2 2 c y 2 y 2 x 1 2 a x 2 b y 2 we first obtain that the above vector fields with at least one center can be divided into 8 classes by its topological phase portraits for the following perturbed system x 2 y b x 2 2 c y 2 e f x y y 2 x 1 2 a x 2 b y 2 e g x y where 0 e 1 f x y and g x y are polynomials in x y with degree n we give an estimation of the number of isolated zeros of the corresponding abelian integral the number of limit cycles follows from poincare pontryagin theorem

estimation of gaussian poissonian gaussian and processed visual noise and its level function we propose a method for estimating the image and video noises of different types white gaussian signal independent mixed poissonian gaussian signal dependent or processed non white our method also estimates the noise level function nlf of these types we do so by classifying image patches based on their intensity and variance in order to find homogeneous regions that best represent the noise we assume that the noise variance is a piecewise linear function of intensity in each intensity class to find noise representative regions noisy signal free patches are first nominated in each intensity class next clusters of connected patches are weighted where the weights are calculated based on the degree of similarity to the noise model the highest ranked cluster defines the peak noise variance and other selected clusters are used to approximate the nlf the more information we incorporate such as temporal data and camera settings the more reliable the estimation becomes to account for the processed noise i e remaining after in camera processing we consider the ratio of low to high frequency energies we address noise variations along video signals using a temporal stabilization of the estimated noise objective and subjective simulations demonstrate that the proposed method outperforms other noise estimation techniques both in accuracy and speed

hierarchical dirichlet processes we consider problems involving groups of data where each observation within a group is a draw from a mixture model and where it is desirable to share mixture components between groups we assume that the number of mixture components is unknown a priori and is to be inferred from the data in this setting it is natural to consider sets of dirichlet processes one for each group where the well known clustering property of the dirichlet process provides a nonparametric prior for the number of mixture components within each group given our desire to tie the mixture models in the various groups we consider a hierarchical model specifically one in which the base measure for the child dirichlet processes is itself distributed according to a dirichlet process such a base measure being discrete the child dirichlet processes necessarily share atoms thus as desired the mixture models in the different groups necessarily share mixture components we discuss representations of hierarchical dirichlet processes

automatic parameter selection for denoising algorithms using a no reference measure of image content across the field of inverse problems in image and video processing nearly all algorithms have various parameters which need to be set in order to yield good results in practice usually the choice of such parameters is made empirically with trial and error if no ground truth reference is available some analytical methods such as cross validation and stein s unbiased risk estimate sure have been successfully used to set such parameters however these methods tend to be strongly reliant on restrictive assumptions on the noise and also computationally heavy in this paper we propose a no reference metric q which is based upon singular value decomposition of local image gradient matrix and provides a quantitative measure of true image content i e sharpness and contrast as manifested in visually salient geometric features such as edges in the presence of noise and other disturbances this measure 1 is easy to compute 2 reacts reasonably to both blur and random noise and 3 works well even when the noise is not gaussian the proposed measure is used to automatically and effectively set the parameters of two leading image denoising algorithms ample simulated and real data experiments support our claims furthermore tests using the tid2008 database show that this measure correlates well with subjective quality evaluations for both blur and noise distortions

an em approach for time variant poisson gaussian model parameter estimation the problem of estimating the parameters of a poisson gaussian model from experimental data has recently raised much interest in various applications for instance in confocal fluorescence microscopy in this context a field of independent random variables is observed which is varying both in time and space each variable is a sum of two components one following a poisson and the other a gaussian distribution in this paper a general formulation is considered where the associated poisson process is nonstationary in space and also exhibits an exponential decay in time whereas the gaussian component corresponds to a stationary white noise with arbitrary mean to solve the considered parametric estimation problem we follow an iterative expectation maximization em approach the parameter update equations involve deriving finite approximation of infinite sums expressions for the maximum error incurred in the process are also given since the problem is non convex we pay attention to the em initialization using a moment based method where recent optimization tools come into play we carry out a performance analysis by computing the cramer rao bounds on the estimated variables the practical performance of the proposed estimation procedure is illustrated on both synthetic data and real fluorescence macroscopy image sequences the algorithm is shown to provide reliable estimates of the mean variance of the gaussian noise and of the scale parameter of the poisson component as well as of its exponential decay rate in particular the mean estimate of the poisson component can be interpreted as a good quality denoised version of the data

reducing multiclass to binary a unifying approach for margin classifiers we present a unifying framework for studying the solution of multiclass categorization problems by reducing them to multiple binary problems that are then solved using a margin based binary learning algorithm the proposed framework unifies some of the most popular approaches in which each class is compared against all others or in which all pairs of classes are compared to each other or in which output codes with error correcting properties are used we propose a general method for combining the classifiers generated on the binary problems and we prove a general empirical multiclass loss bound given the empirical loss of the individual binary learning algorithms the scheme and the corresponding bounds apply to many popular classification learning algorithms including support vector machines adaboost regression logistic regression and decision tree algorithms we also give a multiclass generalization error analysis for general output codes with adaboost as the binary learner experimental results with svm and adaboost show that our scheme provides a viable alternative to the most commonly used multiclass algorithms

homogeneity localization using particle filters with application to noise estimation this paper proposes a method for localizing homogeneity and estimating additive white gaussian noise awgn variance in images the proposed method uses spatially and sparsely scattered initial seeds and utilizes particle filtering techniques to guide their spatial movement towards homogeneous locations this way the proposed method avoids the need to perform the full search associated with block based noise estimation methods to achieve this the paper proposes for the particle filter a dynamic model and a homogeneity observation model based on laplacian structure detectors the variance of awgn is robustly estimated from the variances of blocks in the detected homogeneous areas a proposed adaptive trimmed mean based robust estimator is used to account for the reduction in estimation samples from the full search approach our results show that the proposed method reduces the number of homogeneity measurements required by block based methods while achieving more accuracy

on the classification of elliptic foliations induced by real quadratic fields with center abstract related to the study of hilbert s infinitesimal problem is the problem of determining the existence and estimating the number of limit cycles of the linear perturbation of hamiltonian fields a classification of the elliptic foliations in the projective plane induced by the fields obtained by quadratic fields with center was already studied by several authors in this work we devise a unified proof of the classification of elliptic foliations induced by quadratic fields with center this technique involves using a formula due to cerveau lins neto to calculate the genus of the generic fiber of a first integral of foliations of these kinds furthermore we show that these foliations induce several examples of linear families of foliations which are not bimeromorphically equivalent to certain remarkable examples given by lins neto

bayesian density regression summary the paper considers bayesian methods for density regression allowing a random probability distribution to change flexibly with multiple predictors the conditional response distribution is expressed as a non parametric mixture of regression models with the mixture distribution changing with predictors a class of weighted mixture of dirichlet process priors is proposed for the uncountable collection of mixture distributions it is shown that this specification results in a generalized polya urn scheme which incorporates weights that are dependent on the distance between subjects predictor values to allow local dependence in the mixture distributions we propose a kernel based weighting scheme a gibbs sampling algorithm is developed for posterior computation the methods are illustrated by using simulated data examples and an epidemiologic application

infinite max margin factor analysis via data augmentation this paper addresses the bayesian estimation of the discriminative probabilistic latent models especially the mixture models we develop the max margin factor analysis mmfa model which utilizes the latent variable support vector machine lvsvm as the classification criterion in the latent space to learn a discriminative subspace with max margin constraint furthermore to deal with multimodally distributed data we further extend mmfa to infinite gaussian mixture model and develop the infinite max margin factor analysis immfa model via the consideration of dirichlet process mixtures dpm it jointly learns clustering max margin classifiers and the discriminative latent space in a united framework to improve the prediction performance moreover both of mmfa and immfa are natural to handle outlier rejection problem since the observations are described by a single or a mixture of gaussian distributions additionally thanks to the conjugate property the parameters in the two models can be inferred efficiently via the simple gibbs sampler finally we implement our models on synthesized and real world data including multimodally distributed datasets and measured radar echo data to validate the classification and rejection performance of the proposed models jointly learning fa and svm mmfa is proposed to get a discriminative subspace clustering the dataset in the subspace by dpm mmfa is extended to immfa thanks to the jointly learning framework they gain good prediction performance having the data description ability the proposed models can reject outlier samples in bayesian framework parameters can be inferred efficiently by the gibbs sampler

dirichlet process mixed generalized linear models abstract although generalized linear models glm s are an attractive and widely used class of models they are limited in the range of density shapes that they can provide for instance they are unimodal exponential families of densities in the response variable with tail behavior determined by the implicit mean variance relationship dirichlet process dp mixing adds considerable flexibility to these models using such mixing we develop models that we call dpmglm s which still retain the glm character with regard to the mean overdispersed glm s oglm s provide an alternative class of models to cope with extra variability in samples we show that how oglm s may be dp mixed leading to what we call dpmoglm s these models are extremely rich moreover recent computational advances enable them to be fitted straightforwardly we illustrate this with both simulated and real datasets we also address the question of choosing between the glm oglm dpmglm and dpmoglm finally we consider extensions by

improving prediction from dirichlet process mixtures via enrichment flexible covariate dependent density estimation can be achieved by modelling the joint density of the response and covariates as a dirichlet process mixture an appealing aspect of this approach is that computations are relatively easy in this paper we examine the predictive performance of these models with an increasing number of covariates even for a moderate number of covariates we find that the likelihood for x tends to dominate the posterior of the latent random partition degrading the predictive performance of the model to overcome this we suggest using a different nonparametric prior namely an enriched dirichlet process our proposal maintains a simple allocation rule so that computations remain relatively simple advantages are shown through both predictive equations and examples including an application to diagnosis alzheimer s disease

em algorithms for ml factor analysis the details of em algorithms for maximum likelihood factor analysis are presented for both the exploratory and confirmatory models the algorithm is essentially the same for both cases and involves only simple least squares regression operations the largest matrix inversion required is for aq q symmetric matrix whereq is the matrix of factors the example that is used demonstrates that the likelihood for the factor analysis model may have multiple modes that are not simply rotations of each other such behavior should concern users of maximum likelihood factor analysis and certainly should cast doubt on the general utility of second derivatives of the log likelihood as measures of precision of estimation

a dirichlet process gaussian state machine model for change detection in transient processes abstractthe ability to detect incipient and critical changes in real world process esessential for system integrity assurance is currently impeded by the mismatch between the key assumption of stationarity underlying most change detection methods and the nonlinear and nonstationary transient dynamics of most real world processes the current approaches are slow or outright unable to detect qualitative changes in the behaviors that lead to anomalies we present a dirichlet process gaussian state machine dpgsm model to represent dynamic intermittency which is one of the most ubiquitous real world transient behaviors the dpgsm model treats a signal as a random walk among a dirichlet process mixture of gaussian clusters hypothesis tests and a numerical scheme based on this nonparametric representation were developed to detect subtle changes in the transient intermittent dynamics experimental investigations suggest that the dpgsm approach can consistently detect incipient critical changes in intermi

pure let image deconvolution we propose a non iterative image deconvolution algorithm for data corrupted by poisson or mixed poisson gaussian noise many applications involve such a problem ranging from astronomical to biological imaging we parameterize the deconvolution process as a linear combination of elementary functions termed as linear expansion of thresholds this parameterization is then optimized by minimizing a robust estimate of the true mean squared error the poisson unbiased risk estimate each elementary function consists of a wiener filtering followed by a pointwise thresholding of undecimated haar wavelet coefficients in contrast to existing approaches the proposed algorithm merely amounts to solving a linear system of equations which has a fast and exact solution simulation experiments over different types of convolution kernels and various noise levels indicate that the proposed method outperforms the state of the art techniques in terms of both restoration quality and computational complexity finally we present some results on real confocal fluorescence microscopy images and demonstrate the potential applicability of the proposed method for improving the quality of these images

nonparametric bayes inference on conditional independence in many application areas a primary focus is on assessing evidence in the data refuting the assumption of independence of y and x conditionally on z with y response variables x predictors of interest and z covariates ideally one would have methods available that avoid parametric assumptions allow y x z to be random variables on arbitrary spaces with arbitrary dimension and accommodate rapid consideration of different candidate predictors as a formal decision theoretic approach has clear disadvantages in this context we instead rely on an encompassing nonparametric bayes model for the joint distribution of y x and z with conditional mutual information used as a summary of the strength of conditional dependence we construct a functional of the encompassing model and empirical measure for estimation of conditional mutual information the implementation relies on a single markov chain monte carlo run under the encompassing model with conditional mutual information for candidate models calculated as a byproduct we provide an asymptotic theory supporting the approach and apply the method to variable selection the methods are illustrated through simulations and criminology applications

bayesian cart model search abstract in this article we put forward a bayesian approach for finding classification and regression tree cart models the two basic components of this approach consist of prior specification and stochastic search the basic idea is to have the prior induce a posterior distribution that will guide the stochastic search toward more promising cart models as the search proceeds such models can then be selected with a variety of criteria such as posterior probability marginal likelihood residual sum of squares or misclassification rates examples are used to illustrate the potential superiority of this approach over alternative methods

bayesian non parametric models for spatially indexed data of mixed type we develop bayesian nonparametric models for spatially indexed data of mixed type our work is motivated by challenges that occur in environmental epidemiology where the usual presence of several confounding variables that exhibit complex interactions and high correlations makes it difficult to estimate and under stand the effects of risk factors on health outcomes of interest the modeling approach we adopt assumes that responses and confounding variables are manifestations of continuous latent variables and uses mul tivariate gaussians to jointly model these responses and confounding variables are not treated equally as relevant parameters of the distributions of the responses only are modeled in terms of explanatory variables or risk factors spatial dependence is introduced by allowing the weights of the nonparametric process priors to be location specific obtained as probit transformations of gaussian markov random fields con founding variables and spatial configuration have a similar role in the model in that they only in uence along with the responses the allocation probabilities of the areas into the mixture components thereby allowing for exible adjustment of the effects of observed confounders while allowing for the possibility of residual spatial structure possibly occurring due to unmeasured or undiscovered spatially varying factors aspects of the model are illustrated in simulation studies and an application to a real data set

bayesian nonparametric spatial modeling with dirichlet process mixing customary modeling for continuous point referenced data assumes a gaussian process that is often taken to be stationary when such models are fitted within a bayesian framework the unknown parameters of the process are assumed to be random so a random gaussian process results here we propose a novel spatial dirichlet process mixture model to produce a random spatial process that is neither gaussian nor stationary we first develop a spatial dirichlet process model for spatial data and discuss its properties because of familiar limitations associated with direct use of dirichlet process models we introduce mixing by convolving this process with a pure error process we then examine properties of models created through such dirichlet process mixing in the bayesian framework we implement posterior inference using gibbs sampling spatial prediction raises interesting questions but these can be handled finally we illustrate the approach using simulated data as well as a dataset involving precipitati

bayesian density estimation and inference using mixtures abstract we describe and illustrate bayesian inference in models for density estimation using mixtures of dirichlet processes these models provide natural settings for density estimation and are exemplified by special cases where data are modeled as a sample from mixtures of normal distributions efficient simulation methods are used to approximate various prior posterior and predictive distributions this allows for direct inference on a variety of practical issues including problems of local versus global smoothing uncertainty about density estimates assessment of modality and the inference on the numbers of components also convergence results are established for a general class of normal mixture models

image denoising by sparse 3 d transform domain collaborative filtering we propose a novel image denoising strategy based on an enhanced sparse representation in transform domain the enhancement of the sparsity is achieved by grouping similar 2d image fragments e g blocks into 3d data arrays which we call groups collaborative altering is a special procedure developed to deal with these 3d groups we realize it using the three successive steps 3d transformation of a group shrinkage of the transform spectrum and inverse 3d transformation the result is a 3d estimate that consists of the jointly filtered grouped image blocks by attenuating the noise the collaborative filtering reveals even the finest details shared by grouped blocks and at the same time it preserves the essential unique features of each individual block the filtered blocks are then returned to their original positions because these blocks are overlapping for each pixel we obtain many different estimates which need to be combined aggregation is a particular averaging procedure which is exploited to take advantage of this redundancy a significant improvement is obtained by a specially developed collaborative wiener filtering an algorithm based on this novel denoising strategy and its efficient implementation are presented in full detail an extension to color image denoising is also developed the experimental results demonstrate that this computationally scalable algorithm achieves state of the art denoising performance in terms of both peak signal to noise ratio and subjective visual quality

the consistency of posterior distributions in nonparametric problems we give conditions that guarantee that the posterior probability of every hellinger neighborhood of the true distribution tends to 1 almost surely the conditions are 1 a requirement that the prior not put high mass near distributions with very rough densities and 2 a requirement that the prior put positive mass in kullback leibler neighborhoods of the true distribution the results are based on the idea of approximating the set of distributions with a finite dimensional set of distributions with sufficiently small hellinger bracketing metric entropy we apply the results to some examples

posterior convergence rates of dirichlet mixtures at smooth densities we study the rates of convergence of the posterior distribution for bayesian density estimation with dirichlet mixtures of normal distributions as the prior the true density is assumed to be twice continuously differentiable the bandwidth is given a sequence of priors which is obtained by scaling a single prior by an appropriate order in order to handle this problem we derive a new general rate theorem by considering a countable covering of the parameter space whose prior probabilities satisfy a summability condition together with certain individual bounds on the hellinger metric entropy we apply this new general theorem on posterior convergence rates by computing bounds for hellinger bracketing entropy numbers for the involved class of densities the error in the approximation of a smooth density by normal mixtures and the concentration rate of the prior the best obtainable rate of convergence of the posterior turns out to be equivalent to the well known frequentist rate for integrated mean squared error n 2 5 up to a logarithmic factor

new approaches to bayesian consistency we use martingales to study bayesian consistency we derive sufficient conditions for both hellinger and kullback leibler consistency which do not rely on the use of a sieve alternative sufficient conditions for hellinger consistency are also found and demonstrated on examples

noise parameter estimation for poisson corrupted images using variance stabilization transforms noise is present in all images captured by real world image sensors poisson distribution is said to model the stochastic nature of the photon arrival process and agrees with the distribution of measured pixel values we propose a method for estimating unknown noise parameters from poisson corrupted images using properties of variance stabilization with a significantly lower computational complexity and improved stability the proposed estimation technique yields noise parameters that are comparable in accuracy to the state of art methods

automatic estimation and removal of noise from a single image image denoising algorithms often assume an additive white gaussian noise awgn process that is independent of the actual rgb values such approaches cannot effectively remove color noise produced by today s ccd digital camera in this paper we propose a unified framework for two tasks automatic estimation and removal of color noise from a single image using piecewise smooth image models we introduce the noise level function nlf which is a continuous function describing the noise level as a function of image brightness we then estimate an upper bound of the real nlf by fitting a lower envelope to the standard deviations of per segment image variances for denoising the chrominance of color noise is significantly removed by projecting pixel values onto a line fit to the rgb values in each segment then a gaussian conditional random field gcrf is constructed to obtain the underlying clean image from the noisy input extensive experiments are conducted to test the proposed algorithm which is shown to outperform state of the art denoising algorithms

a unifying view of sparse approximate gaussian process regression we provide a new unifying view including all existing proper probabilistic sparse approximations for gaussian process regression our approach relies on expressing the effective prior which the methods are using this allows new insights to be gained and highlights the relationship between existing methods it also allows for a clear theoretically justified ranking of the closeness of the known approximations to the corresponding full gps finally we point directly to designs of new better sparse approximations combining the best of the existing strategies within attractive computational constraints

the cyclicity of quadratic reversible systems with a center of genus one and non morsean point in this paper we investigate a quadratic reversible system with non morsean point it is proved that the cyclicity of the period annulus under quadratic perturbations is less or equal to four

nonlinear models using dirichlet process mixtures we introduce a new nonlinear model for classification in which we model the joint distribution of response variable y and covariates x non parametrically using dirichlet process mixtures we keep the relationship between y and x linear within each component of the mixture the overall relationship becomes nonlinear if the mixture contains more than one component with different regression coefficients we use simulated data to compare the performance of this new approach to alternative methods such as multinomial logit mnl models decision trees and support vector machines we also evaluate our approach on two classification problems identifying the folding class of protein sequences and detecting parkinson s disease our model can sometimes improve predictive accuracy moreover by grouping observations into sub populations i e mixture components our model can sometimes provide insight into hidden structure in the data

bayesian nonparametric functional data analysis through density estimation in many modern experimental settings observations are obtained in the form of functions and interest focuses on inferences on a collection of such functions we propose a hierarchical model that allows us to simultaneously estimate multiple curves nonparametrically by using dependent dirichlet process mixtures of gaussians to characterize the joint distribution of predictors and outcomes function estimates are then induced through the conditional distribution of the outcome given the predictors the resulting approach allows for flexible estimation and clustering while borrowing information across curves we also show that the function estimates we obtain are consistent on the space of integrable functions as an illustration we consider an application to the analysis of conductivity and temperature at depth data in the north atlantic

convergence rates of posterior distributions we consider the asymptotic behavior of posterior distributions and bayes estimators for infinite dimensional statistical models we give general results on the rate of convergence of the posterior measure these are applied to several examples including priors on finite sieves log spline models dirichlet processes and interval censoring

a class of reversible quadratic systems with piecewise polynomial perturbations abstract this paper investigates a class of reversible quadratic systems perturbed inside piecewise polynomial differential systems of arbitrary degree n all possible phase portraits of the reversible quadratic systems on the plane are obtained by analytic techniques then we present the algebraic structure of its corresponding first order melnikov function which can be used to study the limit cycle bifurcation problem in this direction we use it to investigate the hopf bifurcation of the perturbed systems and obtain an upper bound for the hopf cyclicity meanwhile we employ it to discuss the maximal number of limit cycles emerging from the periodic orbits around the origin for a special case and also provide its upper bound

image noise level estimation by principal component analysis the problem of blind noise level estimation arises in many image processing applications such as denoising compression and segmentation in this paper we propose a new noise level estimation method on the basis of principal component analysis of image blocks we show that the noise variance can be estimated as the smallest eigenvalue of the image block covariance matrix compared with 13 existing methods the proposed approach shows a good compromise between speed and accuracy it is at least 15 times faster than methods with similar accuracy and it is at least two times more accurate than other methods our method does not assume the existence of homogeneous areas in the input image and hence can successfully process images containing only textures

an effective bandwidth selector for local least squares regression abstract local least squares kernel regression provides an appealing solution to the nonparametric regression or scatterplot smoothing problem as demonstrated by fan for example the practical implementation of any scatterplot smoother is greatly enhanced by the availability of a reliable rule for automatic selection of the smoothing parameter in this article we apply the ideas of plug in bandwidth selection to develop strategies for choosing the smoothing parameter of local linear squares kernel estimators our results are applicable to odd degree local polynomial fits and can be extended to other settings such as derivative estimation and multiple nonparametric regression an implementation in the important case of local linear fits with univariate predictors is shown to perform well in practice a by product of our work is the development of a class of nonparametric variance estimators based on local least squares ideas and plug in rules for their implementation

bayesian curve fitting using multivariate normal mixtures summary problems of regression smoothing and curve fitting are addressed via predictive inference in a flexible class of mixture models multidimensional density estimation using dirichlet mixture models provides the theoretical basis for semi parametric regression methods in which fitted regression functions may be deduced as means of conditional predictive distributions these bayesian regression functions have features similar to generalised kernel regression estimates but the formal analysis addresses problems of multivariate smoothing parameter estimation and the assessment of uncertainties about regression functions naturally computations are based on multidimensional versions of existing markov chain simulation analysis of univariate dirichlet mixture models

estimation of conditional densities and sensitivity measures in nonlinear dynamical systems using locally polynomial regression we develop nonparametric estimators for the conditional density function and its square root and their partial derivatives two measures of sensitivity to initial conditions in nonlinear stochastic dynamic systems are proposed one of which relates fisher information with initial value sensitivity in dynamical systems we propose estimators for these and show asymptotic normality for one of them we further propose a simple method for choosing the bandwidth the methods are illustrated by simulation of two well known models in dynamical systems

rates of convergence of posterior distributions we compute the rate at which the posterior distribution concentrates around the true parameter value the spaces we work in are quite general and include infinite dimensional cases the rates are driven by two quantities the size of the space as measured by bracketing entropy and the degree to which the prior concentrates in a small ball around the true parameter we consider two examples

quadratic perturbations of quadratic codimension four centers abstract we study the stratum in the set of all quadratic differential systems x p 2 x y y q 2 x y with a center known as the codimension four case q 4 it has a center and a node and a rational first integral the limit cycles under small quadratic perturbations in the system are determined by the zeros of the first poincare pontryagin melnikov integral i we show that the orbits of the unperturbed system are elliptic curves and i is a complete elliptic integral then using picard fuchs equations and the petrov s method based on the argument principle we set an upper bound of eight for the number of limit cycles produced from the period annulus around the center