partial non gaussian state space summary in this paper we suggest the use of simulation techniques to extend the applicability of the usual gaussian state space filtering and smoothing techniques to a class of nongaussian time series models this allows a fully bayesian or maximum likelihood analysis of some interesting models including outlier models discrete markov chain components multiplicative models and stochastic variance models finally we discuss at some length the use of a non gaussian model to seasonally adjust the published money supply figures

learning nonlinear image manifolds by global alignment of local linear models appearance based methods based on statistical models of the pixel values in an image region rather than geometrical object models are increasingly popular in computer vision in many applications the number of degrees of freedom dof in the image generating process is much lower than the number of pixels in the image if there is a smooth function that maps the dof to the pixel values then the images are confined to a low dimensional manifold embedded in the image space we propose a method based on probabilistic mixtures of factor analyzers to 1 model the density of images sampled from such manifolds and 2 recover global parameterizations of the manifold a globally nonlinear probabilistic two way mapping between coordinates on the manifold and images is obtained by combining several locally valid linear mappings we propose a parameter estimation scheme that improves upon an existing scheme and experimentally compare the presented approach to self organizing maps generative topographic mapping and mixtures of factor analyzers in addition we show that the approach also applies to finding mappings between different embeddings of the same manifold

scalable estimation and regularization for the logistic normal multinomial model clustered multinomial data are prevalent in a variety of applications such as microbiome studies where metagenomic sequencing data are summarized as multinomial counts for a large number of bacterial taxa per subject count normalization with ad hoc zero adjustment tends to result in poor estimates of abundances for taxa with zero or small counts to account for heterogeneity and overdispersion in such data we suggest using the logistic normal multinomial lnm model with an arbitrary correlation structure to simultaneously estimate the taxa compositions by borrowing information across subjects we overcome the computational difficulties in high dimensions by developing a stochastic approximation em algorithm with hamiltonian monte carlo sampling for scalable parameter estimation in the lnm model the ill conditioning problem due to unstructured covariance is further mitigated by a covariance regularized estimator with a condition number constraint the advantages of the proposed methods are illustrated through simulations and an application to human gut microbiome data

on kakeya nikodym type maximal inequalities we show that for any dimension d 3 one can obtain wolff s l d 2 2 bound on kakeya nikodym maximal function in r d for d 3 without the induction on scales argument the key ingredient is to reduce to a 2 dimensional l 2 estimate with an auxiliary maximal function we also prove that the same l d 2 2 bound holds for nikodym maximal function for any manifold m d g with constant curvature which generalizes sogge s results for d 3 to any d 3 as in the 3 dimensional case we can handle manifolds of constant curvature due to the fact that in this case two intersecting geodesics uniquely determine a 2 dimensional totally geodesic submanifold which allows the use of the auxiliary maximal function

gtm the generative topographic mapping latent variable models represent the probability density of data in a space of several dimensions in terms of a smaller number of latent or hidden variables a familiar example is factor analysis which is based on a linear transformations between the latent space and the data space in this paper we introduce a form of non linear latent variable model called the generative topographic mapping for which the parameters of the model can be determined using the em algorithm gtm provides a principled alternative to the widely used self organizing map som of kohonen 1982 and overcomes most of the significant limitations of the som we demonstrate the performance of the gtm algorithm on a toy problem and on simulated data from flow diagnostics for a multi phase oil pipeline

on the correct convergence of the em algorithm for gaussian mixtures it is well known that the em algorithm generally converges to a local maximum likelihood estimate however there have been many evidences to show that the em algorithm can converge correctly to the true parameters as long as the overlap of gaussians in the sample data is small enough this paper studies this correct convergence problem asymptotically on the em algorithm for gaussian mixtures it has been proved that the em algorithm becomes a contraction mapping of the parameters within a neighborhood of the consistent solution of the maximum likelihood when the measure of average overlap among gaussians in the original mixture is small enough and the number of samples is large enough that is if the initial parameters are set within the neighborhood the em algorithm will always converge to the consistent solution i e the expected result moreover the simulation results further demonstrate that this correct convergence neighborhood becomes larger as the average overlap becomes smaller

variational bayesian strategies for high dimensional stochastic design problems this paper is concerned with a lesser studied problem in the context of model based uncertainty quantification uq that of optimization design control under uncertainty the solution of such problems is hindered not only by the usual difficulties encountered in uq tasks e g the high computational cost of each forward simulation the large number of random variables but also by the need to solve a nonlinear optimization problem involving large numbers of design variables and potentially constraints we propose a framework that is suitable for a class of such problems and is based on the idea of recasting them as probabilistic inference tasks to that end we propose a variational bayesian vb formulation and an iterative vb expectation maximization scheme that is capable of identifying a local maximum as well as a low dimensional set of directions in the design space along which the objective exhibits the largest sensitivity we demonstrate the validity of the proposed approach in the context of two numerical examples involving thousands of random and design variables in all cases considered the cost of the computations in terms of calls to the forward model was of the order of 100 or less the accuracy of the approximations provided is assessed by information theoretic metrics

identifying dynamical systems with bifurcations from noisy partial observation we propose a statistical machine learning approach to derive low dimensional models by integrating noisy time series data from partial observation of high dimensional systems aiming to utilize quantitative data on biological phenomena in the cell in particular the method estimates a model from data at different values of a bifurcation parameter in order to characterize biological functions as bifurcation types that are insensitive to system details and experimental errors the method is tested using artificial data generated from two cell cycle control system models that exhibit different bifurcations and the learned systems are shown to robustly inherit the bifurcation types

model based analyses of bioequivalence crossover trials using the stochastic approximation expectation maximisation algorithm in this work we develop a bioequivalence analysis using nonlinear mixed effects models nlmem that mimics the standard noncompartmental analysis nca we estimate nlmem parameters including between subject and within subject variability and treatment period and sequence effects we explain how to perform a wald test on a secondary parameter and we propose an extension of the likelihood ratio test for bioequivalence we compare these nlmem based bioequivalence tests with standard nca based tests we evaluate by simulation the nca and nlmem estimates and the type i error of the bioequivalence tests for nlmem we use the stochastic approximation expectation maximisation saem algorithm implemented in monolix we simulate crossover trials under h 0 using different numbers of subjects and of samples per subject we simulate with different settings for between subject and within subject variability and for the residual error variance the simulation study illustrates the accuracy of nlmem based geometric means estimated with the saem algorithm whereas the nca estimates are biased for sparse design nca based bioequivalence tests show good type i error except for high variability for a rich design type i errors of nlmem based bioequivalence tests wald test and likelihood ratio test do not differ from the nominal level of 5 type i errors are inflated for sparse design we apply the bioequivalence wald test based on nca and nlmem estimates to a three way crossover trial showing that omnitrope sandoz gmbh kundl austria powder and solution are bioequivalent to genotropin pfizer pharma gmbh karlsruhe germany nlmem based bioequivalence tests are an alternative to standard nca based tests however caution is needed for small sample size and highly variable drug

stability of stochastic approximation under verifiable conditions in this paper we address the problem of the stability and convergence of the stochastic approximation procedure theta n 1 theta n gamma n 1 h theta n xi n 1 the stability of such sequences theta n is known to heavily rely on the behavior of the mean field h at the boundary of the parameter set and the magnitude of the stepsizes used the conditions typically required to ensure convergence and in particular the boundedness or stability of theta n are either too difficult to check in practice or not satisfied at all this is the case even for very simple models the most popular technique for circumventing the stability problem consists of constraining theta n to a compact subset mathcal k in the parameter space this is obviously not a satisfactory solution as the choice of mathcal k is a delicate one in this paper we first prove a deterministic stability result which relies on simple conditions on the sequences xi n and gamma n we then propose and analyze an algorithm based on projections on adaptive truncation sets which ensures that the aforementioned conditions required for stability are satisfied we focus in particular on the case where xi n is a so called markov state dependent noise we establish both the stability and convergence with probability 1 w p 1 of the algorithm under a set of simple and verifiable assumptions we illustrate our results with an example related to adaptive markov chain monte carlo algorithms

propagation of singularities and maximal functions in the plane in this work we mainly generalize bourgain s circular maximal function to include variable coefficient averages our techniques involve a combination of bourgain s basic ideas plus microlocal analysis in particular to see the role of curvature we rely heavily on methods used in studying propogation of singularities for hyperbolic differential equations we also show that forp 2 there is local smoothing inl p for solutions to the wave equation

a few endpoint geodesic restriction estimates for eigenfunctions we prove a couple of new endpoint geodesic restriction estimates for eigenfunctions in the case of general 3 dimensional compact manifolds after a tt argument simply by using the l 2 boundedness of the hilbert transform on mathbb r we are able to improve the corresponding l 2 restriction bounds of burq gerard and tzvetkov duke math j 138 445 486 2007 and hu forum math 6 1021 1052 2009 also in the case of 2 dimensional compact manifolds with nonpositive curvature we obtain improved l 4 estimates for restrictions to geodesics which by holder s inequality and interpolation implies improved l p bounds for all exponents p 2 we do this by using oscillatory integral theorems of hormander ark mat 11 1 11 1973 greenleaf and seeger j reine angew math 455 35 56 1994 and phong and stein int math res notices 4 49 60 1991 along with a simple geometric lemma lemma 3 2 about properties of the mixed hessian of the riemannian distance function restricted to pairs of geodesics in riemannian surfaces we are also able to get further improvements beyond our new results in three dimensions under the assumption of constant nonpositive curvature by exploiting the fact that in this case there are many totally geodesic submanifolds

convergence of a stochastic approximation version of the em algorithm the expectation maximization em algorithm is a powerful computational technique for locating maxima of functions it is widely used in statistics for maximum likelihood or maximum a posteriori estimation in incomplete data models in certain situations however this method is not applicable because the expectation step cannot be performed in closed form to deal with these problems a novel method is introduced the stochastic approximation em saem which replaces the expectation step of the em algorithm by one iteration of a stochastic approximation procedure the convergence of the saem algorithm is established under conditions that are applicable to many practical situations moreover it is proved that under mild additional conditions the attractive stationary points of the saem algorithm correspond to the local maxima of the function presented to support our findings

kalman filter implementation of self organizing feature maps the self organizing learning algorithm of kohonen and most of its extensions are controlled by two learning parameters the learning coefficient and the width of the neighborhood function which have to be chosen empirically because neither rules nor methods for their calculation exist consequently often time consuming parameter studies precede neighborhood preserving feature maps of the learning data to circumvent those lengthy numerical studies this article describes the learning process by a state space model in order to use the linear kalman filter algorithm training the feature maps then the kalman filter equations calculate the learning coefficient online during the training while the width of the neighborhood function needs to be estimated by a second extended kalman filter for the process of neighborhood preservation the performance of the kalman filter implementation is demonstrated on toy problems as well as on a crab classification problem the results of crab classification are compared to those of generative topographic mapping an alternative method to the self organizing feature map

probabilistic self organizing maps for continuous data the original self organizing feature map did not define any probability distribution on the input space however the advantages of introducing probabilistic methodologies into self organizing map models were soon evident this has led to a wide range of proposals which reflect the current emergence of probabilistic approaches to computational intelligence the underlying estimation theories behind them derive from two main lines of thought the expectation maximization methodology and stochastic approximation methods here we present a comprehensive view of the state of the art with a unifying perspective of the involved theoretical frameworks in particular we examine the most commonly used continuous probability distributions self organization mechanisms and learning schemes special emphasis is given to the connections among them and their relative advantages depending on the characteristics of the problem at hand furthermore we evaluate their performance in two typical applications of self organizing maps classification and visualization

stochastic approximation em for large scale exploratory irt factor analysis a stochastic approximation em algorithm saem is described for exploratory factor analysis of dichotomous or ordinal variables the factor structure is obtained from sufficient statistics that are updated during iterations with the robbins monro procedure two large scale simulations are reported that compare accuracy and cpu time of the proposed saem algorithm to the metropolis hasting robbins monro procedure and to a generalized least squares analysis of the polychoric correlation matrix a smaller scale application to real data is also reported including a method for obtaining standard errors of rotated factor loadings a simulation study based on the real data analysis is conducted to study bias and error estimates the saem factor algorithm requires minimal lines of code no derivatives and no large matrix inversion it is programmed entirely in r

model based clustering by probabilistic self organizing maps in this paper we consider the learning process of a probabilistic self organizing map pbsom as a model based data clustering procedure that preserves the topological relationships between data clusters in a neural network based on this concept we develop a coupling likelihood mixture model for the pbsom that extends the reference vectors in kohonen s self organizing map som to multivariate gaussian distributions we also derive three expectation maximization em type algorithms called the socem soem and sodaem algorithms for learning the model pbsom based on the maximum likelihood criterion socem is derived by using the classification em cem algorithm to maximize the classification likelihood soem is derived by using the em algorithm to maximize the mixture likelihood and sodaem is a deterministic annealing da variant of socem and soem moreover by shrinking the neighborhood size socem and soem can be interpreted respectively as da variants of the cem and em algorithms for gaussian model based clustering the experimental results show that the proposed pbsom learning algorithms achieve comparable data clustering performance to that of the deterministic annealing em daem approach while maintaining the topology preserving property

on line em algorithm for the normalized gaussian network a normalized gaussian network ngnet moody darken 1989 is a network of local linear regression units the model softly partitions the input space by normalized gaussian functions and each local unit linearly approximates the output within the partition in this article we propose a new on line em algorithm for the ngnet which is derived from the batch em algorithm xu jordan hinton 1995 by introducing a discount factor we show that the on line em algorithm is equivalent to the batch em algorithm if a specific scheduling of the discount factor is employed in addition we show that the on line em algorithm can be considered as a stochastic approximation method to find the maximum likelihood estimator a new regularization method is proposed in order to deal with a singular input distribution in order to manage dynamic environments where the input output distribution of data changes over time unit manipulation mechanisms such as unit production unit deletion and unit division are also introduced based on probabilistic interpretation experimental results show that our approach is suitable for function approximation problems in dynamic environments we also apply our on line em algorithm to robot dynamics problems and compare our algorithm with the mixtures of experts family

growing hierarchical probabilistic self organizing graphs since the introduction of the growing hierarchical self organizing map much work has been done on self organizing neural models with a dynamic structure these models allow adjusting the layers of the model to the features of the input dataset here we propose a new self organizing model which is based on a probabilistic mixture of multivariate gaussian components the learning rule is derived from the stochastic approximation framework and a probabilistic criterion is used to control the growth of the model moreover the model is able to adapt to the topology of each layer so that a hierarchy of dynamic graphs is built this overcomes the limitations of the self organizing maps with a fixed topology and gives rise to a faithful visualization method for high dimensional data

weighted rule based adaptive algorithm for simultaneously extracting generalized eigenvectors in this brief we consider extracting generalized eigenvectors in parallel for the generalized eigendecomposition problem the problem is formulated as an optimization problem of minimizing an unconstrained quartic cost function based on the weighted rule it is shown that the proposed weighted cost function has a unique global minimum which corresponds to the principal generalized eigenvectors in order to estimate the principal generalized eigenvector matrix efficiently we simplify the quartic cost function as a quadric one by making an appropriate approximation and then derive a fast algorithm for extracting the principal generalized eigenvector in parallel we also show the application of the proposed algorithm in blind source separation numerical simulations are performed and the results demonstrate the performance of the proposed algorithm

solutions for semilinear parabolic equations in lp and regularity of weak solutions of the navier stokes system abstract we construct a unique local regular solution in l q 0 t l p for a class of semilinear parabolic equations which includes the semilinear heat equation u t u u u 0 and the navier stokes system here p and q are so chosen that the norm of l q 0 t l p is dimensionless or scaling invariant the main relation between p and q for the semilinear heat equation is 1 q 1 p 1 q n 2 provided that initial data are in l r with r n 2 1 where n is the space dimension applying our regular solutions to the navier stokes system we show that the k 2 dimensional hausdorff measure of possible time singularities of a turbulent solution is zero if the turbulent solution is in l q 0 t l p where k 2 q nq p p n 1 q we show moreover that a turbulent solution is regular if it is in c 0 t l n

a multiple imputation metropolis version of the em algorithm summary in this paper we introduce a new stochastic variant of the em algorithm the algorithm combines the principle of multiple imputation and the theory of simulated annealing to deal with cases where the e step and the m step can be intractable or numerically inefficient

parallel proximal algorithm for image restoration using hybrid regularization regularization approaches have demonstrated their effectiveness for solving ill posed problems however in the context of variational restoration methods a challenging question remains namely how to find a good regularizer while total variation introduces staircase effects wavelet domain regularization brings other artefacts e g ringing however a tradeoff can be made by introducing a hybrid regularization including several terms not necessarily acting in the same domain e g spatial and wavelet transform domains while this approach was shown to provide good results for solving deconvolution problems in the presence of additive gaussian noise an important issue is to efficiently deal with this hybrid regularization for more general noise models to solve this problem we adopt a convex optimization framework where the criterion to be minimized is split in the sum of more than two terms for spatial domain regularization isotropic or anisotropic total variation definitions using various gradient filters are considered an accelerated version of the parallel proximal algorithm is proposed to perform the minimization some difficulties in the computation of the proximity operators involved in this algorithm are also addressed in this paper numerical experiments performed in the context of poisson data recovery show the good behavior of the algorithm as well as promising results concerning the use of hybrid regularization techniques

design evaluation and optimisation in crossover pharmacokinetic studies analysed by nonlinear mixed effects models bioequivalence or interaction trials are commonly studied in crossover design and can be analysed by nonlinear mixed effects models as an alternative to noncompartmental approach we propose an extension of the population fisher information matrix in nonlinear mixed effects models to design crossover pharmacokinetic trials using a linearisation of the model around the random effect expectation including within subject variability and discrete covariates fixed or changing between periods we use the expected standard errors of treatment effect to compute the power for the wald test of comparison or equivalence and the number of subjects needed for a given power we perform various simulations mimicking crossover two period trials to show the relevance of these developments we then apply these developments to design a crossover pharmacokinetic study of amoxicillin in piglets and implement them in the new version 3 2 of the r function pfim

flexible longitudinal linear mixed models for multiple censored responses data in biomedical studies and clinical trials repeated measures are often subject to some upper and or lower limits of detection hence the responses are either left or right censored a complication arises when more than one series of responses is repeatedly collected on each subject at irregular intervals over a period of time and the data exhibit tails heavier than the normal distribution the multivariate censored linear mixed effect mlmec model is a frequently used tool for a joint analysis of more than one series of longitudinal data in this context we develop a robust generalization of the mlmec based on the scale mixtures of normal distributions to take into account the autocorrelation existing among irregularly observed measures a damped exponential correlation structure is considered for this complex longitudinal structure we propose an exact estimation procedure to obtain the maximum likelihood estimates of the fixed effects and variance components using a stochastic approximation of the em algorithm this approach allows us to estimate the parameters of interest easily and quickly as well as to obtain the standard errors of the fixed effects the predictions of unobservable values of the responses and the log likelihood function as a byproduct the proposed method is applied to analyze a set of aids data and is examined via a simulation study

entropy based kernel mixture modeling for topographic map formation a new information theoretic learning algorithm for kernel based topographic map formation is introduced in the one dimensional case the algorithm is aimed at uniformizing the cumulative distribution of the kernel mixture densities by maximizing its differential entropy a nonparametric differential entropy estimator is used on which normalized gradient ascent is performed both differentiable and nondifferentiable kernels are in principle supported such as gaussian and rectangular on off kernels the relation is shown with joint entropy maximization of the kernel outputs the learning algorithm s performance is assessed and compared with the theoretically optimal performance a fixed point rule is derived for the case of heterogeneous kernel mixtures finally an extension of the algorithm to the multidimensional case is suggested

case deletion measures for models with incomplete data summary this paper proposes several case deletion measures for assessing the influence of an observation for complicated models with real missing data or hypothetical missing data corresponding to latent random variables the idea is to generalise cook s 1977 approach to the conditional expectation of the complete data loglikelihood function in the em algorithm on the basis of the diagnostic measures a procedure is proposed for detecting influential observations two examples illustrate our methodology we show that the method can be applied efficiently to a wide variety of complicated problems that are difficult to handle by existing methods

the ecme algorithm a simple extension of em and ecm with faster monotone convergence a generalisation of the ecm algorithm meng rubin 1993 which is itselfan extension of the em algorithm dempster laird rubin 1977 can be obtained by replacing some cm steps of ecm which maximise the constrained expected complete data loglikelihood function with steps that maximise the correspondingly constrained actual likelihood function this algorithm which we call ecme algorithm for expectation conditional maximisation either shares with both em and ecm their stable monotone convergence and basic simplicity of implementation relative to competing faster converging methods moreover ecme can have a substantially faster convergence rate than either em or ecm measured using either the number of iterations or actual computer time

a stochastic approximation algorithm with varying bounds many optimization problems that are intractable with conventional approaches will yield to stochastic approximation algorithms this is because these algorithms can be used to optimize functions that cannot be evaluated analytically but have to be estimated for instance through simulation or measured thus stochastic approximation algorithms can be used for optimization in simulation unfortunately the classical stochastic approximation algorithm sometimes diverges because of unboundedness problems we study the convergence of a variant of stochastic approximation defined over a growing sequence of compact sets we show that this variant converges under more general conditions on the objective function than the classical algorithm while maintaining the same asymptotic convergence rate we also present empirical evidence that shows that this algorithm sometimes converges much faster than the classical algorithm

a correlated probit model for joint modeling of clustered binary and continuous responses a difficulty in joint modeling of continuous and discrete response variables is the lack of a natural multivariate distribution for joint modeling of clustered observations on binary and continuous responses we study a correlated probit model that has an underlying normal latent variable for the binary responses catalano and ryan have factored the model into a marginal and a conditional component and used generalized estimating equations methodology to estimate the effects we propose a monte carlo expectation conditional maximization algorithm for finding maximum likelihood estimates of the mixed model itself extending and accelerating an algorithm for models with binary responses we demonstrate the methodology with a developmental toxicity study measuring fetal weight and a binary malformation status for several litters of mice a simulation study suggests that efficiency gains of joint fittings over separate fittings of the response variables occur mainly for small datasets with strong correlation

concerning nikodym type sets in 3 dimensional curved spaces in this paper we shall give some natural partial extensions to the curved space setting of results of bourgain 1 and wolff 21 concerning lower bounds for the dimension of compliments of nikodym sets in euclidean space recall that a classical nikodym set is a subset of 1 1 x 1 1 of lebesgue measure one which has the property that for each x e f there is a line yx so that x nf x because of this the relative compliment q 1 1 x 1 1 f must be a set of measure zero with the property that if q is the set of points x for which there is a line segment y through x with iq n axf a then iq ji 0 for every 0 7 3 if iq i 0 here as before tyx denotes a line segment through x of length ae 0 this lower bound was later improved by wolff 21 to dim q 5 2 if iq i o in both works the lower bounds on the dimension were obtained for somewhat more general sets lower bounds for analogous sets in higher dimensions were also obtained in 1 and 21 the strongest to date are those of wolff 21 showing that if q c ri and iq i 0 for some ae 0 then dim q n 2 2 let us now consider extensions of this result to the curved 3 dimensional setting to this end we shall let m3 g denote a paracompact 3 dimensional riemannian manifold with metric ds2 zgjk x dxjdxk given x e m3 we let ya denote the set of all geodesics containing x with arclength ae that is i y a abusing the classical terminology somewhat we now define nikodym type subsets of m3

hypoelliptic diffusions filtering and inference from complete and partial observations the statistical problem of parameter estimation in partially observed hypoelliptic diffusion processes is naturally occurring in many applications however because of the noise structure where the noise components of the different co ordinates of the multi dimensional process operate on different timescales standard inference tools are ill conditioned we propose to use a higher order scheme to approximate the likelihood such that the different timescales are appropriately accounted for we show consistency and asymptotic normality with non typical convergence rates when only partial observations are available we embed the approximation in a filtering algorithm for the unobserved co ordinates and use this as a building block in a stochastic approximation expectation maximization algorithm we illustrate on simulated data from three models the harmonic oscillator the fitzhugh nagumo model used to model membrane potential evolution in neuroscience and the synaptic inhibition and excitation model used for determination of neuronal synaptic input

mixtures of probabilistic principal component analyzers principal component analysis pca is one of the most popular techniques for processing compressing and visualizing data although its effectiveness is limited by its global linearity while nonlinear variants of pca have been proposed an alternative paradigm is to capture data complexity by a combination of local linear pca projections however conventional pca does not correspond to a probability density and so there is no unique way to combine pca models therefore previous attempts to formulate mixture models for pca have been ad hoc to some extent in this article pca is formulated within a maximum likelihood framework based on a specific form of gaussian latent variable model this leads to a well defined mixture model for probabilistic principal component analyzers whose parameters can be determined using an expectationmaximization algorithm we discuss the advantages of this model in the context of clustering density modeling and local dimensionality reduction and we demonstrate its application to image compression and handwritten digit recognition

robust clustering by deterministic agglomeration em of mixtures of multivariate t distributions abstract this paper presents new robust clustering algorithms which significantly improve upon the noise and initialization sensitivity of traditional mixture decomposition algorithms and simplify the determination of the optimal number of clusters in the data set the algorithms implement maximum likelihood mixture decomposition of multivariate t distributions a robust parametric extension of gaussian mixture decomposition we achieve improved convergence capability relative to the expectation maximization em approach by deriving deterministic annealing em daem algorithms for this mixture model and turning them into agglomerative algorithms going through a monotonically decreasing number of components an approach we term deterministic agglomeration em dagem two versions are derived based on two variants of daem for mixture models simulation studies demonstrate the algorithms performance for mixtures with isotropic and non isotropic covariances in two and 10 dimensions with known or unknown levels of outlier contamination

blow up set for a semilinear heat equation with small diffusion abstract we consider the blow up problem for a semilinear heat equation t u u u p in 0 t u x t 0 on 0 t if u x 0 x 0 in where is a domain in r n n 1 0 p 1 and t 0 in this paper under suitable assumptions on we prove that if the family of the solutions u satisfies a uniform type i blow up estimate with respect to then the solution u blows up only near the maximum points of the initial datum for any sufficiently small 0 this is proved without any conditions on the exponent p and the domain such as n 2 p n 2 and the convexity of the domain

a bilinear approach to the restriction and kakeya conjectures the purpose of this paper is to investigate bilinear variants of the restriction and kakeya conjectures to relate them to the standard formulations of these conjectures and to give applications of this bilinear approach to existing conjectures the methods used are based on several observations and results of bourgain see 2 6 together with some refinements by moyua vargas and vega 17 18 this paper is organized as follows in the first section we discuss bilinear restriction estimates and show how one can pass back and forth between these estimates and the standard restriction estimates we also generalize the 12 7 bilinear restriction estimate of 18 to higher dimensions in the second section we give analogues of the above results for the kakeya operator in particular we give a bilinear improvement to wolff s kakeya theorem in arbitrary dimension in the third section we give applications of these bilinear estimates in three dimensions for example we are able to improve the 42 11 exponent in wolff s restriction theorem to 34 9 we are also able to prove a sharp lp lq restriction theorem which improves on the classical l2 l4 tomas stein theorem and also give some concrete progress on a bilinear restriction conjecture of klainerman and machedon we also give a non bilinear approach to these estimates which gives weaker results but is more direct and probably has a wider range of application

self organizing mixture networks for probability density estimation a self organizing mixture network somn is derived for learning arbitrary density functions the network minimizes the kullback leibler information metric by means of stochastic approximation methods the density functions are modeled as mixtures of parametric distributions a mixture needs not to be homogenous i e it can have different density profiles the first layer of the network is similar to kohonen s self organizing map som but with the parameters of the component densities as the learning weights the winning mechanism is based on maximum posterior probability and updating of the weights is limited to a small neighborhood around the winner the second layer accumulates the responses of these local nodes weighted by the learned mixing parameters the network possesses a simple structure and computational form yet yields fast and robust convergence the network has a generalization ability due to the relative entropy criterion used applications to density profile estimation and pattern classification are presented the somn can also provide an insight to the role of neighborhood function used in the som

recursive maximum likelihood algorithm for dependent observations a recursive maximum likelihood algorithm rml is proposed that can be used when both the observations and the hidden data have continuous values and are statistically dependent between different time samples the algorithm recursively approximates the probability density functions of the observed and hidden data by analytically computing the integrals with respect to the state variables where the parameters are updated using gradient steps a full convergence proof is given based on the ordinary differential equation approach which shows that the algorithm converges to a local minimum of the kullback leibler divergence between the true and the estimated parametric probability density functions a result that is useful even for a miss specified parametric model compared to other rml algorithms proposed in the literature this contribution extends the state space model and provides a theoretical analysis in a nontrivial statistical model that was not analyzed so far we further extend the rml analysis to constrained parameter estimation problems two examples including nonlinear state space models are given to highlight this contribution

near peak scattering and the corrected kirchhoff approximation for a convex obstacle abstract the uniform asympotic behavior of the scattering amplitude near the forward peak in the case of classical scattering of waves by a convex obstacle is derived a microlocal model is obtained for the scattering operator this is achieved by use of a parametrix for diffractive boundary problems and by a new study of a class of fourier integral operators those with folding canonical relations a crucial ingredient consists of putting a fourier integral operator with folding canonical relation into a normal form the analysis also gives the asymptotic behavior of the normal derivative of the scattered wave on a neighbourhood of the shadow boundary thus providing a corrected version of the kirchhoff approximation

convergence asymptotic periodicity and finite point blow up in one dimensional semilinear heat equations abstract we consider the initial value problem for the semilinear heat equation u t u xx f u t 0 x l t 0 under the dirichlet the neumann or the periodic boundary conditions we show that each solution whether it exists globally for t 0 or blows up in a finite time possesses an asymptotic profile in a certain sense and tends to this profile as time increases in the special case where f u t t f u t for some t 0 among other things the above statement is interpreted as saying that any bounded global solution converges as t to a time t periodic solution having some specific spatial structure in the case where the solution blows up in a finite time say at t t 0 assuming simply that f is a smooth function satisfying some growth conditions and that the initial data is a nonconstant bounded function we prove that the blow up set is a finite set and that lim t t 0 u x t x exists with being a smooth function having at most finitely many singular points

kernel based topographic map formation by local density modeling we introduce a new learning algorithm for kernel based topographic map formation the algorithm generates a gaussian mixture density model by individually adapting the gaussian kernels centers and radii to the assumed gaussian local input densities

blow up for a semilinear parabolic equation with large diffusion on rn ii we are concerned with the cauchy problem for a semilinear heat equation p tu d u u p 1u x rn t 0 u x 0 x x rn where d 0 p 1 n 3 0 and l rn l1 rn 1 x 2dx in the paper of fujishima and ishige 2011 8 the authors of this paper studied the behavior of the blow up time and the blow up set of the solution of p as d for the case rn x dx 0 in this paper as a continuation of fujishima and ishige 2011 8 we consider the case rn x dx 0 and study the behavior of the blow up time and the blow up set of the solution of p as d the behavior in the case rn x dx 0 is completely different from the one in the case rn x dx 0

some alternatives to asymptotic tests for the analysis of pharmacogenetic data using nonlinear mixed effects models nonlinear mixed effects models allow investigating individual differences in drug concentration profiles pharmacokinetics and responses pharmacogenetics focuses on the genetic component of this variability two tests often used to detect a gene effect on a pharmacokinetic parameter are 1 the wald test assessing whether estimates for the gene effect are significantly different from 0 and 2 the likelihood ratio test comparing models with and without the genetic effect because those asymptotic tests show inflated type i error on small sample size and or with unevenly distributed genotypes we develop two alternatives and evaluate them by means of a simulation study first we assess the performance of the permutation test using the wald and the likelihood ratio statistics second for the wald test we propose the use of the f distribution with four different values for the denominator degrees of freedom we also explore the influence of the estimation algorithm using both the first order conditional estimation with interaction linearization based algorithm and the stochastic approximation expectation maximization algorithm we apply these methods to the analysis of the pharmacogenetics of indinavir in hiv patients recruited in the cophar2 anrs 111 trial results of the simulation study show that the permutation test seems appropriate but at the cost of an additional computational burden one of the four f distribution based approaches provides a correct type i error estimate for the wald test and should be further investigated

a unified model for probabilistic principal surfaces principal curves and surfaces are nonlinear generalizations of principal components and subspaces respectively they can provide insightful summary of high dimensional data not typically attainable by classical linear methods solutions to several problems such as proof of existence and convergence faced by the original principal curve formulation have been proposed in the past few years nevertheless these solutions are not generally extensible to principal surfaces the mere computation of which presents a formidable obstacle consequently relatively few studies of principal surfaces are available we previously 2000 proposed the probabilistic principal surface pps to address a number of issues associated with current principal surface algorithms pps uses a manifold oriented covariance noise model based on the generative topographical mapping gtm which can be viewed as a parametric formulation of kohonen s self organizing map building on the pps we introduce a unified covariance model that implements pps 0 1 by varying the clamping parameter spl alpha then we comprehensively evaluate the empirical performance of pps gtm and the manifold aligned gtm on three popular benchmark data sets it is shown in two different comparisons that the pps outperforms the gtm under identical parameter settings convergence of the pps is found to be identical to that of the gtm and the computational overhead incurred by the pps decreases to 40 percent or less for more complex manifolds these results show that the generalized pps provides a flexible and effective way of obtaining principal surfaces

the zero set of a solution of a parabolic equation on etudie l ensemble nul d une solution u t x de l equation u t a x t u xx b x t u x c x t u sous des hypotheses tres generales sur les coefficients a b et c

unbiased estimators and multilevel monte carlo multilevel monte carlo mlmc and recently proposed unbiased estimators are closely related this connection is elaborated by presenting a new general class of unbiased estimators which admits previous debiasing schemes as special cases new lower variance estimators are proposed which are stratified versions of earlier unbiased schemes under general conditions essentially when mlmc admits the canonical square root monte carlo error rate the proposed new schemes are shown to be asymptotically as efficient as mlmc both in terms of variance and cost the experiments demonstrate that the variance reduction provided by the new schemes can be substantial

hierarchical gtm constructing localized nonlinear projection manifolds in a principled way it has been argued that a single two dimensional visualization plot may not be sufficient to capture all of the interesting aspects of complex data sets and therefore a hierarchical visualization system is desirable in this paper we extend an existing locally linear hierarchical visualization system phivis in several directions 1 we allow for nonlinear projection manifolds the basic building block is the generative topographic mapping gtm 2 we introduce a general formulation of hierarchical probabilistic models consisting of local probabilistic models organized in a hierarchical tree general training equations are derived regardless of the position of the model in the tree 3 using tools from differential geometry we derive expressions for local directional curvatures of the projection manifold like phivis our system is statistically principled and is built interactively in a top down fashion using the em algorithm it enables the user to interactively highlight those data in the ancestor visualization plots which are captured by a child model we also incorporate into our system a hierarchical locally selective representation of magnification factors and directional curvatures of the projection manifolds such information is important for further refinement of the hierarchical visualization plot as well as for controlling the amount of regularization imposed on the local models we demonstrate the principle of the approach on a toy data set and apply our system to two more complex 12 and 18 dimensional data sets

maximum likelihood topographic map formation we introduce a new unsupervised learning algorithm for kernel based topographic map formation of heteroscedastic gaussian mixtures that allows for a unified account of distortion error vector quantization log likelihood and kullback leibler divergence

a gradient algorithm locally equivalent to the em algorithm in many problems of maximum likelihood estimation it is impossible to carry out either the e step or the m step of the em algorithm the present paper introduces a gradient algorithm that is closely related to the em algorithm this em gradient algorithm approximately solves the m step of the em algorithm by one iteration of newton s method since newton s method converges quickly the local properties of the em gradient algorithm are almost identical with those of the em algorithm any strict local maximum point of the observed likelihood locally attracts the em and em gradient algorithm at the same rate of convergence and near the maximum point the em gradient algorithm always produces an increase in the likelihood with proper modification the em gradient algorithm also exhibits global convergence properties that are similar to those of the em algorithm our proof of global convergence applies and improves existing theory for the em algorithm these theoretical points are reinforced by a discussion of three realistic examples illustrating how the em gradient algorithm can succeed where the em algorithm is intractable

new approach to assess bioequivalence parameters using generalized gamma mixed effect model model based asymptotic bioequivalence test in the pharmacokinetic pk study under a 2x2 crossover design that involves both the test and reference drugs we propose a mixed effects model for the drug concentration time profiles obtained from subjects who receive different drugs at different periods in the proposed model the drug concentrations repeatedly measured from the same subject at different time points are distributed according to a multivariate generalized gamma distribution and the drug concentration time profiles are described by a compartmental pk model with between subject and within subject variations we then suggest a bioequivalence test based on the estimated bioavailability parameters in the proposed mixed effects model the results of a monte carlo study further show that the proposed model based bioequivalence test is not only better on maintaining its level but also more powerful for detecting the bioequivalence of the two drugs than the conventional bioequivalence test based on a non compartmental analysis or the one based on a mixed effects model with a normal error variable the application of the proposed model and test is finally illustrated by using data sets in two pk studies copyright 2013 john wiley sons ltd

stochastic approximation in monte carlo computation the wang landau wl algorithm is an adaptive markov chain monte carlo algorithm used to calculate the spectral density for a physical system a remarkable feature of the wl algorithm is that it is not trapped by local energy minima which is very important for systems with rugged energy landscapes this feature has led to many successful applications of the algorithm in statistical physics and biophysics however there does not exist rigorous theory to support its convergence and the estimates produced by the algorithm can reach only a limited statistical accuracy in this article we propose the stochastic approximation monte carlo samc algorithm which overcomes the shortcomings of the wl algorithm we establish a theorem concerning its convergence the estimates produced by samc can be improved continuously as the simulation proceeds samc also extends applications of the wl algorithm to continuum systems the potential uses of samc in statistics are discussed through two classes of applications i

constrained monte carlo maximum likelihood for dependent data maximum likelihood estimates mles in autologistic models and other exponential family models for dependent data can be calculated with markov chain monte carlo methods the metropolis algorithm or the gibbs sampler which simulate ergodic markov chains having equilibrium distributions in the model from one realization of such a markov chain a monte carlo approximant to the whole likelihood function can be constructed the parameter value if any maximizing this function approximates the mle

a fast improvement to the em algorithm on its own terms the em algorithm is a numerical technique for the evaluation of maximum likelihood estimates for parameters describing incomplete data models it is easy to apply in many problems and is stable but slow the algorithm fails to provide a consistent estimator of the standard errors of the maximum likelihood estimates unless the additional analysis required by the louis method is performed newton type or other gradient methods are faster and provide error estimates but tend to be unstable and require the analytical evaluation of likelihoods to derive expressions for the score function and at least approximations to the fisher information matrix the purpose of this paper is to expand on a result by fisher that permits a unification of em methodology and newton methods the evaluation of the individual observation by observation score functions of the incomplete data is a byproduct of the application of the e step of the em algorithm once these become available the fisher information matrix may be consistently estimated and the m step may be replaced by a fast newton type step

mixed effect statistics for group analysis in fmri a nonparametric maximum likelihood approach this technical note describes a collection of test statistics accounting for estimation uncertainties at the within subject level that can be used as alternatives to the standard t statistic in one sample random effect analyses i e when testing the mean effect of a population we build such test statistics by estimating the across subject distribution of the effects using maximum likelihood under a nonparametric mixed effect model for inference purposes the statistics are calibrated using permutation tests to achieve exact false positive control under a symmetry assumption regarding the across subject distribution the new tests are implemented in a freely available toolbox for spm called distance

high dimensional odes coupled with mixed effects modeling techniques for dynamic gene regulatory network identification gene regulation is a complicated process the interaction of many genes and their products forms an intricate biological network identification of this dynamic network will help us understand the biological processes in a systematic way however the construction of a dynamic network is very challenging for a high dimensional system in this article we propose to use a set of ordinary differential equations ode coupled with dimensional reduction by clustering and mixed effects modeling techniques to model the dynamic gene regulatory network grn the ode models allow us to quantify both positive and negative gene regulation as well as feedback effects of genes in a functional module on the dynamic expression changes of genes in another functional module which results in a directed graph network a five step procedure clustering smoothing regulation identification parameter estimates refining and function enrichment analysis csief is developed to identify the ode based dynamic grn in the prop

joint entropy maximization in kernel based topographic maps a new learning algorithm for kernel based topographic map formation is introduced the kernel parameters are adjusted individually so as to maximize the joint entropy of the kernel outputs this is done by maximizing the differential entropies of the individual kernel outputs given that the map s output redundancy due to the kernel overlap needs to be minimized the latter is achieved by minimizing the mutual information between the kernel outputs as a kernel the radial incomplete gamma distribution is taken since for a gaussian input density the differential entropy of the kernel output will be maximal since the theoretically optimal joint entropy performance can be derived for the case of nonoverlapping gaussian mixture densities a new clustering algorithm is suggested that uses this optimum as its null distribution finally it is shown that the learning algorithm is similar to one that performs stochastic gradient descent on the kullback leibler divergence for a heteroskedastic gaussian mixture density model

local behavior near blow up points for semilinear parabolic equations abstract this paper is concerned with solutions of u t u xx u p p 1 defined in a cylinder qr x t r 0 which blowup at t 1 more precisely we are interested in the asymptotic behaviour of such solutions near a blow up point as t 1 we show that whenever the blow up set can be confined to the interior of the interval r r asymptotics are the same as those corresponding to solutions defined in the whole line which assume bounded initial values however if the blow up set reaches the boundary of the interval r r a different behaviour may appear as indicated by theorem 2 in the introduction

a flexible state space model for learning nonlinear dynamical systems we consider a nonlinear state space model with the state transition and observation functions expressed as basis function expansions the coefficients in the basis function expansions are learned f

convergence of the monte carlo expectation maximization for curved exponential families the monte carlo expectation maximization mcem algorithm is a versatile tool for inference in incomplete data models especially when used in combination with markov chain monte carlo simulation methods in this contribution the almost sure convergence of the mcem algorithm is established it is shown using uniform versions of ergodic theorems for markov chains that mcem converges under weak conditions on the simulation kernel practical illustrations are presented using a hybrid random walk metropolis hastings sampler and an independence sampler the rate of convergence is studied showing the impact of the simulation schedule on the fluctuation of the parameter estimate at the convergence a novel averaging procedure is then proposed to reduce the simulation variance and increase the rate of convergence

asymptotic convergence rate of the em algorithm for gaussian mixtures it is well known that the convergence rate of the expectation maximization em algorithm can be faster than those of convention first order iterative algorithms when the overlap in the given mixture is small but this argument has not been mathematically proved yet this article studies this problem asymptotically in the setting of gaussian mixtures under the theoretical framework of xu and jordan 1996 it has been proved that the asymptotic convergence rate of the em algorithm for gaussian mixtures locally around the true solution is o e0 5 e where e 0 is an arbitrarily small number o x means that it is a higher order infinitesimal as x 0 and e is a measure of the average overlap of gaussians in the mixture in other words the large sample local convergence rate for the em algorithm tends to be asymptotically superlinear when e tends to zero

high dimensional exploratory item factor analysis by a metropolis hastings robbins monro algorithm a metropolis hastings robbins monro mh rm algorithm for high dimensional maximum marginal likelihood exploratory item factor analysis is proposed the sequence of estimates from the mh rm algorithm converges with probability one to the maximum likelihood solution details on the computer implementation of this algorithm are provided the accuracy of the proposed algorithm is demonstrated with simulations as an illustration the proposed algorithm is applied to explore the factor structure underlying a new quality of life scale for children it is shown that when the dimensionality is high mh rm has advantages over existing methods such as numerical quadrature based em algorithm extensions of the algorithm to other modeling frameworks are discussed

combining monte carlo and mean field like methods for inference in hidden markov random fields issues involving missing data are typical settings where exact inference is not tractable as soon as nontrivial interactions occur between the missing variables approximations are required and most of them are based either on simulation methods or on deterministic variational methods while variational methods provide fast and reasonable approximate estimates in many scenarios simulation methods offer more consideration of important theoretical issues such as accuracy of the approximation and convergence of the algorithms but at a much higher computational cost in this work we propose a new class of algorithms that combine the main features and advantages of both simulation and deterministic methods and consider applications to inference in hidden markov random fields hmrfs these algorithms can be viewed as stochastic perturbations of variational expectation maximization vem algorithms which are not tractable for hmrf we focus more specifically on one of these perturbations and we prove their almost sure convergence to the same limit set as the limit set of vem in addition experiments on synthetic and real world images show that the algorithm performance is very close and sometimes better than that of other existing simulation based and variational em like algorithms

finding the observed information matrix when using the em algorithm a procedure is derived for extracting the observed information matrix when the em algorithm is used to find maximum likelihood estimates in incomplete data problems the technique requires computation of a complete data gradient vector or second derivative matrix but not those associated with the incomplete data likelihood in addition a method useful in speeding up the convergence of the em algorithm is developed two examples are presented

solution of a nonlinear heat equation with arbitrarily given blow up points we consider the equation where i u is scalar valued and p 1 it has been proven that if u t blows up at time t the blow up points are finite in number and located in i our aim is to prove that this result is optimal that is for any given points x1 xk in i there is a solution u such that its blow up points are exactly x1 xk

single point blow up for a semilinear initial value problem abstract the initial value problem on r r is considered u t t x u xx t x u t x u t r 0 u 0 x x where 0 and is a fixed large number it is known that for some initial values the solution u t x exists only up to some finite time t and that u t as t t for the specific initial value k where 0 xx 0 r 0 k is sufficiently large it is shown that if x 0 then lim t t u t x and lim t t u x t x exist and are finite in other words blow up occurs only at the point x 0

multivariate analysis of longitudinal ordinal data with mixed effects models with application to clinical outcomes in osteoarthritis our objective was to evaluate the efficacy of robenacoxib in osteoarthritic dogs using four ordinal responses measured repeatedly over time we propose a multivariate probit mixed effects model to describe the joint evolution of endpoints and to evidence the intrinsic correlations between responses that are not due to treatment effect maximum likelihood computation is intractable within reasonable time frames we therefore use a pairwise modeling approach in combination with a stochastic em algorithm multidimensional ordinal responses with longitudinal measurements are a common feature in clinical trials however the standard methods for data analysis use unidimensional models resulting in a loss of information our methodology provides substantially greater insight than these methods for the evaluation of treatment effects and shows a good performance at low computational cost we thus believe that it could be used in routine practice to optimize the evaluation of treatment efficacy