implicit finite difference approximation for time fractional diffusion equations time fractional diffusion equations are used when attempting to describe transport processes with long memory where the rate of diffusion is inconsistent with the classical brownian motion model in this paper we develop an implicit unconditionally stable numerical method to solve the one dimensional linear time fractional diffusion equation formulated with caputo s fractional derivative on a finite slab several numerical examples of interest are also included

the em algorithm an old folk song sung to a fast new tune celebrating the 20th anniversary of the presentation of the paper by dempster laird and rubin which popularized the em algorithm we investigate after a brief historical account strategies that aim to make the em algorithm converge faster while maintaining its simplicity and stability e g automatic monotone convergence in likelihood first we introduce the idea of a working parameter to facilitate the search for efficient data augmentation schemes and thus fast em implementations second summarizing various recent extensions of the em algorithm we formulate a general alternating expectation conditional maximization algorithm aecm that couples flexible data augmentation schemes with model reduction schemes to achieve efficient computations we illustrate these methods using multivariate t models with known or unknown degrees of freedom and poisson models for image reconstruction we show through both empirical and theoretical evidence the potential for a dramatic reduction in computational time with little increase in human effort we also discuss the intrinsic connection between em type algorithms and the gibbs sampler and the possibility of using the techniques presented here to speed up the latter the main conclusion of the paper is that with the help of statistical considerations it is possible to construct algorithms that are simple stable and fast

parameter expansion to accelerate em the px em algorithm summary the em algorithm and its extensions are popular tools for modal estimation but ar e often criticised for their slow convergence we propose a new method that can often make em much faster the intuitive idea is to use a covariance adjustment to correct the analysis of the m step capitalising on extra information captured in the imputed complete data the way we accomplish this is by parameter expansion we expand the complete data model while preserving the observed data model and use the expanded complete data model to generate em this parameter expanded ei m px em algorithm shares the simplicity and stability of ordinary em but has a faster rate of convergence since its m step performs a more efficient analysis the px em algorithm is illustrated for the multivariate t distribution a random effects model factor analysis probit regression and a poisson imaging model

compact finite difference method for the fractional diffusion equation high order compact finite difference scheme for solving one dimensional fractional diffusion equation is considered in this paper after approximating the second order derivative with respect to space by the compact finite difference we use the grunwald letnikov discretization of the riemann liouville derivative to obtain a fully discrete implicit scheme we analyze the local truncation error and discuss the stability using the fourier method then we prove that the compact finite difference scheme converges with the spatial accuracy of fourth order using matrix analysis numerical results are provided to verify the accuracy and efficiency of the proposed algorithm

the approximate solutions of some stochastic differential equations using transformations in this paper the transformation technique together with a numerical technique is used to obtain an approximate probability density function pdf for the solution process of some stochastic differential equations the approach is applied on some first order and second order differential equations with making comparisons with the exact solution if possible

a new fractional numerical differentiation formula to approximate the caputo fractional derivative and its applications in the present work first a new fractional numerical differentiation formula called the l1 2 formula to approximate the caputo fractional derivative of order 0 1 is developed it is established by means of the quadratic interpolation approximation using three points t j 2 f t j 2 t j 1 f t j 1 and t j f t j for the integrand f t on each small interval t j 1 t j j 2 while the linear interpolation approximation is applied on the first small interval t 0 t 1 as a result the new formula can be formally viewed as a modification of the classical l1 formula which is obtained by the piecewise linear approximation for f t both the computational efficiency and numerical accuracy of the new formula are superior to that of the l1 formula the coefficients and truncation errors of this formula are discussed in detail two test examples show the numerical accuracy of l1 2 formula second by the new formula two improved finite difference schemes with high order accuracy in time for solving the time fractional sub diffusion equations on a bounded spatial domain and on an unbounded spatial domain are constructed respectively in addition the application of the new formula into solving fractional ordinary differential equations is also presented several numerical examples are computed the comparison with the corresponding results of finite difference methods by the l1 formula demonstrates that the new l1 2 formula is much more effective and more accurate than the l1 formula when solving time fractional differential equations numerically

joint state and parameter estimation for a target directed nonlinear dynamic system model we present a new approach to joint state and parameter estimation for a target directed nonlinear dynamic system model with switching states the model recently proposed for representing speech dynamics is called the hidden dynamic model hdm the model parameters subject to statistical estimation consist of the target vector and the system matrix also called time constants as well as parameters characterizing the nonlinear mapping from the hidden state to the observation we implement these parameters as the weights of a three layer feedforward multilayer perceptron mlp network the new estimation approach is based on the extended kalman filter ekf and its performance is compared with the traditional expectation maximization em based approach extensive simulation results are presented using both approaches and under typical hdm speech modeling conditions the ekf based algorithm demonstrates superior convergence performance compared with the em algorithm but the former suffers from excessive computational loads when adopted for training the mlp weights in all cases the simulated model output converges to the given observation sequence however only in the case where the mlp weights or the target vector are assumed known do the time constant parameters converge to their true values we also show that the mlp weights never converge to their true values thus demonstrating the many to one mapping property of the feedforward mlp we conclude that for the system to be identifiable restrictions on the parameter space are needed

maximum likelihood estimation via the ecm algorithm a general framework two major reasons for the popularity of the em algorithm are that its maximum step involves only complete data maximum likelihood estimation which is often computationally simple and that its convergence is stable with each iteration increasing the likelihood when the associated complete data maximum likelihood estimation itself is complicated em is less attractive because the m step is computationally unattractive in many cases however complete data maximum likelihood estimation is relatively simple when conditional on some function of the parameters being estimated

compact difference scheme for the fractional sub diffusion equation with neumann boundary conditions an effective finite difference scheme is considered for solving the time fractional sub diffusion equation with neumann boundary conditions a difference scheme combining the compact difference approach the spatial discretization and l 1 approximation for the caputo fractional derivative is proposed and analyzed although the spatial approximation order at the neumann boundary is one order lower than that for interior mesh points the unconditional stability and the global convergence order o t 2 a h 4 in discrete l 2 norm of the compact difference scheme are proved rigorously where t is the temporal grid size and h is the spatial grid size numerical experiments are included to support the theoretical results and comparison with the related works are presented to show the effectiveness of our method

mixture densities maximum likelihood and the em algorithm the problem of estimating the parameters which determine a mixture density has been the subject of a large diverse body of literature spanning nearly ninety years during the last two decades the

recursive parameter estimation using incomplete data abstract stochastic approximation procedures are considered for the estimation of parameters using incomplete data one procedure is stated and illustrated which often leads to asymptotically efficient estimators others are developed which although possibly not optimal in the above sense will be very much easier to apply this will be particularly advantageous when quick recursive estimates are required examples are given and a link is made between one of the sub optimal methods and the em algorithm author

second order linear differential equations with analytic uncertainties stochastic analysis via the computation of the probability density function abstract this paper concerns the analysis of random second order linear differential equations usually solving these equations consists of computing the first statistics of the response process and that task has been an essential goal in the literature a more ambitious objective is the computation of the solution probability density function we present advances on these two aspects in the case of general random non autonomous second order linear differential equations with analytic data processes the frobenius method is employed to obtain the stochastic solution in the form of a mean square convergent power series we demonstrate that the convergence requires the boundedness of the random input coefficients further the mean square error of the frobenius method is proved to decrease exponentially with the number of terms in the series although not uniformly in time regarding the probability density function of the solution at a given time which is the focus of the paper we rely on the law of total probability to express it in closed form as an expectation for the computation of this expectation a sequence of approximating density functions is constructed by reducing the dimensionality of the problem using the truncated power series of the fundamental set we prove several theoretical results regarding the pointwise convergence of the sequence of density functions and the convergence in total variation the pointwise convergence turns out to be exponential under a lipschitz hypothesis as the density functions are expressed in terms of expectations we propose a crude monte carlo sampling algorithm for their estimation this algorithm is implemented and applied on several numerical examples designed to illustrate the theoretical findings of the paper after that the efficiency of the algorithm is improved by employing the control variates method numerical examples corroborate the variance reduction of the monte carlo approach

generalized linear latent variable models for repeated measures of spatially correlated multivariate data summary observations of multiple response variables across space and over time occur often in environmental and ecological studies compared to purely spatial models for a single response variable in the exponential family of distributions fewer statistical tools are available for multiple response variables that are not necessarily gaussian an exception is a common factor model developed for multivariate spatial data by wang and wall 2003 biostatistics 4 569 582 the purpose of this article is to extend this multivariate space only model and develop a flexible class of generalized linear latent variable models for multivariate spatial temporal data for statistical inference maximum likelihood estimates and their standard deviations are obtained using a monte carlo em algorithm we also use a novel way to automatically adjust the monte carlo sample size which facilitates the convergence of the monte carlo em algorithm the methodology is illustrated by an ecological study of red pine trees in response to bark beetle challenges in a forest stand of wisconsin

large sample joint posterior approximations when full conditionals are approximately normal modern bayesian statistical methods such as gibbs and metropolis hastings sampling were developed to liberate statisticians from the necessity of making large sample assumptions and to facilitate the numerical approximation of problems that had previously been analytically intractable counter to this trend we develop a method for constructing asymptotic joint posterior approximations based on models with k blocks of parameters and where the corresponding properly normalized full conditionals are themselves asymptotically normal we illustrate these techniques by applying them to particular linear and generalized linear mixed models glmms we also consider the relevance of different parameterizations with regard to our asymptotics recent work has indicated that gibbs samplers based on so called centering parameterizations result in better convergence properties for the resulting markov chains our results for the one way random effects model shed some light on this issue for this example we also

on the generalization of probabilistic transformation method abstract the probabilistic transformation method with the finite element analysis is a new technique to solve random differential equation the advantage of this technique is finding the exact expression of the probability density function of the solution when the probability density function of the input is known however the disadvantage is due to the characteristics geometrics and materials of the analyzed structure included in the random differential equation in this paper a developed formula is used to generalize this technique by obtaining the exact joint probability density function of the solution in any situations as well as the proposed technique for the non linear case

convergence of a stochastic approximation version of the em algorithm the expectation maximization em algorithm is a powerful computational technique for locating maxima of functions it is widely used in statistics for maximum likelihood or maximum a posteriori estimation in incomplete data models in certain situations however this method is not applicable because the expectation step cannot be performed in closed form to deal with these problems a novel method is introduced the stochastic approximation em saem which replaces the expectation step of the em algorithm by one iteration of a stochastic approximation procedure the convergence of the saem algorithm is established under conditions that are applicable to many practical situations moreover it is proved that under mild additional conditions the attractive stationary points of the saem algorithm correspond to the local maxima of the function presented to support our findings

solution of the stochastic transport equation of neutral particles with anisotropic scattering using rvt technique much of the literatures are directed toward the development of a mathematical formalism for a rigorous estimation of the ensemble average of the solution process of a stochastic differential equation sde the random variable transformation technique rvt is a powerful technique to get the complete solution for the sde represented by the probability density function of the solution process in this paper the rvt technique together with a simple integral transformation to the input stochastic process are implemented to get the complete solution of the one speed transport equation for neutral particles in a semi infinite stochastic medium with linear anisotropic scattering the extinction function of the medium input stochastic process is assumed to be a continuous random function of position the probability density function and hence the higher order statistical moments of the solution process are presented numerical results are given for different distributions of the input stochastic process

compact difference schemes for the modified anomalous fractional sub diffusion equation and the fractional diffusion wave equation in this paper compact finite difference schemes for the modified anomalous fractional sub diffusion equation and fractional diffusion wave equation are studied schemes proposed previously can at most achieve temporal accuracy of order which depends on the order of fractional derivatives in the equations and is usually less than two based on the idea of weighted and shifted grunwald difference operator we establish schemes with temporal and spatial accuracy order equal to two and four respectively

a monte carlo implementation of the em algorithm and the poor man s data augmentation algorithms abstract the first part of this article presents the monte carlo implementation of the e step of the em algorithm given the current guess to the maximizer of the posterior distribution latent data patterns are generated from the conditional predictive distribution the expected value of the augmented log posterior is then updated as a mixture of augmented log posteriors mixed over the generated latent data patterns multiple imputations in the m step of the algorithm this mixture is maximized to obtain the update to the maximizer of the observed posterior the gradient and hessian of the observed log posterior are also expressed as mixtures mixed over the multiple imputations the relation between the monte carlo em mcem algorithm and the data augmentation algorithm is noted two modifications to the mcem algorithm the poor man s data augmentation algorithms which allow for the calculation of the entire posterior are then presented these approximations serve as diagnostics for the validity o

computing probabilistic solutions of the bernoulli random differential equation the random variable transformation technique is a powerful method to determine the probabilistic solution for random differential equations represented by the first probability density function of the solution stochastic process in this paper that technique is applied to construct a closed form expression of the solution for the bernoulli random differential equation in order to account for the general scenario all the input parameters coefficients and initial condition are assumed to be absolutely continuous random variables with an arbitrary joint probability density function the analysis is split into two cases for which an illustrative example is provided finally a fish weight growth model is considered to illustrate the usefulness of the theoretical results previously established using real data

applying the random variable transformation method to solve a class of random linear differential equation with discrete delay abstract we randomize the following class of linear differential equations with delay x t a x t b x t t 0 and initial condition x t g t t 0 by assuming that coefficients a and b are random variables and the initial condition g t is a stochastic process we consider two cases depending on the functional form of the stochastic process g t and then we solve from a probabilistic point of view both random initial value problems by determining explicit expressions to the first probability density function f x t of the corresponding solution stochastic processes afterwards we establish sufficient conditions on the involved random input parameters in order to guarantee that f x t converges as 0 to the first probability density function say f x t of the corresponding associated random linear problem without delay 0 the paper concludes with several numerical experiments illustrating our theoretical findings

an algorithm based on discrete response regression models suitable to correct the bias of non response in surveys with several capture tries the use of survey plans which contemplate several tries or call backs when endeavouring to capture individual data may supply unarguable information in certain sampling situations with non ignorable non response this paper presents an algorithm whose final aim is the estimation of the individual non response probabilities from a general perspective of discrete response regression models which includes the well known probit and logit models it will be assumed that the respondents supply all the variables of interest when they are captured nevertheless the call backs continue even after previous captures for a small number of tries r which has been fixed beforehand only for estimating purposes the different retries or call backs are supposed to be carried out with different capture intensities as mentioned above the response probabilities which may vary from one individual to another are sought by discrete response regression models whose parameters are estimated from conditioned likelihoods evaluated on the respondents only the algorithm quick and easy to implement may be used even when the capture indicator matrix has been partially recorded finally the practical performance of the proposed procedure is tested and evaluated from empirical simulations whose results are undoubtedly encouraging

a discontinuous petrov galerkin method for time fractional diffusion equations we propose and analyze a time stepping discontinuous petrov galerkin method combined with the continuous conforming finite element method in space for the numerical solution of time fractional subdiffusion problems we prove the existence uniqueness and stability of approximate solutions and derive error estimates to achieve high order convergence rates from the time discretizations the time mesh is graded appropriately near t 0 to compensate for the singular temporal behavior of the exact solution near t 0 caused by the weakly singular kernel but the spatial mesh is quasi uniform in the l infty 0 t l 2 omega norm 0 t is the time domain and omega is the spatial domain for sufficiently graded time meshes a global convergence of order k m alpha 2 h r 1 is shown where 0 alpha 1 is the fractional exponent k is the maximum time step h is the maximum diameter of the elements of the spatial mesh and m and r are the degrees of approximate solutions in time an

latent variable models for mixed discrete and continuous outcomes we propose a latent variable model for mixed discrete and continuous outcomes the model accommodates any mixture of outcomes from an exponential family and allows for arbitrary covariate effects as well as direct modelling of covariates on the latent variable an em algorithm is proposed for parameter estimation and estimates of the latent variables are produced as a by product of the analysis a generalized likelihood ratio test can be used to test the significance of covariates affecting the latent outcomes this method is applied to birth defects data where the outcomes of interest are continuous measures of size and binary indicators of minor physical anomalies infants who were exposed in utero to anticonvulsant medications are compared with controls

a proposed technique of sfem on solving ordinary random differential equation the stochastic finite element sfem is a new method to solve random differential equation using the known deterministic finite element technique fem adapted to stochastic techniques for solving stochastic differential equations in this paper a proposed technique of fem and random variable transformation rvt is used to solve a differential equation with random excitation the technique shows high accuracy when solving a case study compared with the exact solution

error estimates for a semidiscrete finite element method for fractional order parabolic equations we consider the initial boundary value problem for a homogeneous time fractional diffusion equation with an initial condition v x and a homogeneous dirichlet boundary condition in a bounded convex polygonal domain omega we study two semidiscrete approximation schemes i e the galerkin finite element method fem and lumped mass galerkin fem using piecewise linear functions we establish almost optimal with respect to the data regularity error estimates including the cases of smooth and nonsmooth initial data i e v in h 2 omega cap h 1 0 omega and v in l 2 omega for the lumped mass method the optimal l 2 norm error estimate is valid only under an additional assumption on the mesh which in two dimensions is known to be satisfied for symmetric meshes finally we present some numerical results that give insight into the reliability of the theoretical study

a developed solution of the stochastic milne problem using probabilistic transformations abstract this paper considers the solution of the milne problem of radiative transfer with isotropic scattering in a continuous stochastic medium properties of the medium are assumed to be continuous random functions of the spatial dimensions the available solutions in literature for this stochastic integro differential equation side are represented only by the ensemble average of the radiant energy density in this paper a developed algorithm based on the implementation of the random variable transformation technique together with an integral transformation to the stochastic properties is introduced a complete stochastic solution represented by the probability density function p d f of the radiant energy density is obtained using the closed form of the p d f the nth moment of the stochastic solution is evaluated in realization of this work exponential and gaussian statistics for the medium properties are assumed results are physically acceptable and found to be compatible with those in the literature

an approximate probabilistic solution of a random sir type epidemiological model using rvt technique abstract this paper adapts a multi dimensional random variable transformation technique to derive a comprehensive stochastic description of the susceptible infected recovered epidemiological model an approximate solution of a system of nonlinear differential equations that characterizes this model is deterministically obtained and from this approximation we derive the first probability density functions for the solution processes of susceptible infected and recovered percentages these probability density functions are used to find the approximate mean and variance functions as well as the confidence intervals taking a general situation the infection contact rate the recovery rate and the initial conditions are taken to be random variables with arbitrary distributions to test the validity of the theoretical findings associated to the proposed random epidemiological model some numerical results are tabulated and graphically presented through an illustrative example

on use of the em algorithm for penalized likelihood estimation summary the em algorithm is a popular approach to maximum likelihood estimation but has not been much used for penalized likelihood or maximum a posteriori estimation this paper discusses properties of the em algorithm in such contexts concentrating on rates of conver gence and presents an alternative that is usually more practical and converges at least as quickly the em algorithm is a general approach to maximum likelihood estimation rather than a specific algorithm dempster et al 1977 discussed the method and derived basic properties demonstrating that a variety of procedures previously developed rather informally could be unified the common strand to problems where the approach is applicable is a notion of incomplete data this includes the conventional sense of missing data but is much broader than that the em algorithm demon strates its strength in situations where some hypothetical experiment yields data from which estimation is particularly convenient and economical the incomplete data actually at hand are regarded as observable functions of these complete data the resulting algorithms while usually slow to converge are often extremely simple and remain practical in large problems where no other approaches may be feasible dempster et al 1977 briefly refer to the use of the same approach to the problem of finding the posterior mode maximum a posteriori estimate in a bayesian estima

low rank structure learning via nonconvex heuristic recovery in this paper we propose a nonconvex framework to learn the essential low rank structure from corrupted data different from traditional approaches which directly utilizes convex norms to measure the sparseness our method introduces more reasonable nonconvex measurements to enhance the sparsity in both the intrinsic low rank structure and the sparse corruptions we will respectively introduce how to combine the widely used lp norm 0 p 1 and log sum term into the framework of low rank structure learning although the proposed optimization is no longer convex it still can be effectively solved by a majorization minimization mm type algorithm with which the nonconvex objective function is iteratively replaced by its convex surrogate and the nonconvex problem finally falls into the general framework of reweighed approaches we prove that the mm type algorithm can converge to a stationary point after successive iterations the proposed model is applied to solve two typical problems robust principal component analysis and low rank representation experimental results on low rank structure learning demonstrate that our nonconvex heuristic methods especially the log sum heuristic recovery algorithm generally perform much better than the convex norm based method 0 p 1 for both data with higher rank and with denser corruptions

robust statistical modeling using the t distribution abstract the t distribution provides a useful extension of the normal for statistical modeling of data sets involving errors with longer than normal tails an analytical strategy based on maximum likelihood for a general model with multivariate t errors is suggested and applied to a variety of problems including linear and nonlinear regression robust estimation of the mean and covariance matrix with missing data unbalanced multivariate repeated measures data multivariate modeling of pedigree data and multivariate nonlinear regression the degrees of freedom parameter of the t distribution provides a convenient dimension for achieving robust statistical inference with moderate increases in computational complexity for many models estimation of precision from asymptotic theory and the bootstrap is discussed and graphical methods for checking the appropriateness of the t distribution are presented

novel numerical methods for solving the time space fractional diffusion equation in two dimensions in this paper a time space fractional diffusion equation in two dimensions tsfde 2d with homogeneous dirichlet boundary conditions is considered the tsfde 2d is obtained from the standard diffusion equation by replacing the first order time derivative with the caputo fractional derivative t d gamma gamma in 0 1 and the second order space derivatives with the fractional laplacian delta alpha 2 alpha in 1 2 using the matrix transfer technique proposed by ilic et al m ilic f liu i turner and v anh fract calc appl anal 9 2006 pp 333 349 the tsfde 2d is transformed into a time fractional differential system as t d gamma mathbf u k alpha mathbf a alpha 2 mathbf u where mathbf a is the approximate matrix representation of delta traditional approximation of mathbf a alpha 2 requires diagonalization of mathbf a which is very time consuming for large sparse matrices the novelty of our proposed numerical schemes is that using either the finite difference method or the laplace transform to handle the caputo time fractional derivative the solution of the tsfde 2d is written in terms of a matrix function vector product f mathbf a mathbf b at each time step where mathbf b is a suitably defined vector depending on the method used to generate the matrix mathbf a the product f mathbf a mathbf b can be approximated using either the preconditioned lanczos method when mathbf a is symmetric or the mathbf m lanzcos method when mathbf a is nonsymmetric which are powerful techniques for solving large linear systems we give error bounds for the new methods and illustrate their roles in solving the tsfde 2d we also derive the analytical solution of the tsfde 2d in terms of the mittag leffler function finally numerical results are presented to verify the proposed numerical solution strategies

variational learning for switching state space models we introduce a new statistical model for time series that iteratively segments data into regimes with approximately linear dynamics and learns the parameters of each of these linear regimes this model combines and generalizes two of the most widely used stochastic time series models hidden markov models and linear dynamical systems and is closely related to models that are widely used in the control and econometrics literatures it can also be derived by extending the mixture of experts neural network jacobs jordan nowlan hinton 1991 to its fully dynamical version in which both expert and gating networks are recurrent inferring the posterior probabilities of the hidden states of this model is computationally intractable and therefore the exact expectation maximization em algorithm cannot be applied however we present a variational approximation that maximizes a lower bound on the log likelihood and makes use of both the forward and backward recursions for hidden markov models and the kalman filter recursions for linear dynamical systems we tested the algorithm on artificial data sets and a natural data set of respiration force from a patient with sleep apnea the results suggest that variational approximations are a viable method for inference and learning in switching state space models

high order finite element methods for time fractional partial differential equations the aim of this paper is to develop high order methods for solving time fractional partial differential equations the proposed high order method is based on high order finite element method for space and finite difference method for time optimal convergence rate o dt 2 a n r is proved for the r 1 th order finite element method r 2

solving second order linear differential equations with random analytic coefficients about ordinary points a full probabilistic solution by the first probability density function abstract this paper deals with the approximate computation of the first probability density function of the solution stochastic process to second order linear differential equations with random analytic coefficients about ordinary points under very general hypotheses our approach is based on considering approximations of the solution stochastic process via truncated power series solution obtained from the random regular power series method together with the so called random variable transformation technique the validity of the proposed method is shown through several illustrative examples

error estimates of crank nicolson type difference schemes for the subdiffusion equation a crank nicolson type difference scheme is proposed for solving the subdiffusion equation with fractional derivative and the truncation error is analyzed in detail at each temporal level only a tridiagonal linear system needs to be solved and the thomas algorithm may be used the solvability unconditional stability and h 1 norm convergence are proved the convergence order is rm min 2 gamma 2 1 gamma in the temporal direction and two in the spatial direction by the sobolev embedding inequality we obtain the maximum norm error estimate a spatial compact scheme based on the crank nicolson type difference scheme is also presented and similar results are given the convergence order is mathcal o tau rm min 2 gamma 2 1 gamma h 4 numerical experiments are included to support the theoretical results and comparisons with the related works are presented to show the effectiveness of our method

an efficient numerical solver for anisotropic subdiffusion problems abstract this paper presents an efficient and robust numerical solver for anisotropic subdiffusion problems which are important but not addressed directly in the literature the chebyshev spectral collocation method is utilized for discretization of the spatial laplacian whereas a linear interpolation is used for discretizing the fractional order caputo temporal derivative this solver is stable and catches the main features of subdiffusion numerical experiments are presented to demonstrate the accuracy and efficiency of this new solver

using fem rvt technique for solving a randomly excited ordinary differential equation with a random operator the technique of stochastic finite element sfem which is the finite element technique fem adapted to stochastic problems can be re described to use random variable transformation technique rvt a new fem rvt technique was successfully used in solving stochastic problems with random excitation m el tawil w el tahhan a hussein a proposed technique of sfem on solving ordinary random differential equation j appl math comput 161 2005 35 47 in this paper the technique is adapted to solve a randomly excited differential equation with a random operator the technique shows high accuracy when solving a case study compared with the exact solution finally a problem with unknown exact solution is solved using this technique

constrained maximum likelihood estimation of two level covariance structure model via em type algorithms in this paper the constrained maximum likelihood estimation of a two level covariance structure model with unbalanced designs is considered the two level model is reformulated as a single level model by treating the group level latent random vectors as hypothetical missing data then the popular em algorithm is extended to obtain the constrained maximum likelihood estimates for general nonlinear constraints the multiplier method is used at them step to find the constrained minimum of the conditional expectation an accelerated em gradient procedure is derived to handle linear constraints the empirical performance of the proposed em type algorithms is illustrated by some artifical and real examples

a space time spectral method for the time fractional diffusion equation in this paper we consider the numerical solution of the time fractional diffusion equation essentially the time fractional diffusion equation differs from the standard diffusion equation in the time derivative term in the former case the first order time derivative is replaced by a fractional derivative making the problem global in time we propose a spectral method in both temporal and spatial discretizations for this equation the convergence of the method is proven by providing a priori error estimate numerical tests are carried out to confirm the theoretical results thanks to the spectral accuracy in both space and time of the proposed method the storage requirement due to the global time dependence can be considerably relaxed and therefore calculation of the long time solution becomes possible

a chebyshev pseudospectral method to solve the space time tempered fractional diffusion equation a chebyshev pseudospectral method to solve the space time tempered fractional diffusion equation

finite difference spectral approximations for the time fractional diffusion equation in this paper we consider the numerical resolution of a time fractional diffusion equation which is obtained from the standard diffusion equation by replacing the first order time derivative with a fractional derivative of order a with 0 a 1 the main purpose of this work is to construct and analyze stable and high order scheme to efficiently solve the time fractional diffusion equation the proposed method is based on a finite difference scheme in time and legendre spectral methods in space stability and convergence of the method are rigourously established we prove that the full discretization is unconditionally stable and the numerical solution converges to the exact one with order o dt 2 a n m where dt n and m are the time step size polynomial degree and regularity of the exact solution respectively numerical experiments are carried out to support the theoretical claims

vector l_0 sparse variable pca principal component analysis pca achieves dimension reduction by replacing the original measured variables with a smaller set of derived variables called the principal components sparse pca improves this with sparsity there are two kinds of sparse pca sparse loading pca slpca which keeps all the measured variables but zeroes out some of their loadings and sparse variable pca svpca which removes some measured variables completely by simultaneously zeroing out all their loadings because it zeroes out some measured variables completely svpca is capable of huge additional dimension reduction beyond pca while slpca keeps all measured variables and does not have this capability here we consider a vector l0 penalized likelihood approach to svpca and develop a penalized expectation maximization pem algorithm which remarkably in an l0 setting leads to a closed form m step and we provide a convergence analysis

maximum likelihood estimation of nonlinear structural equation models the existing maximum likelihood theory and its computer software in structural equation modeling are established based on linear relationships among manifest variables and latent variables however models with nonlinear relationships are often encountered in social and behavioral sciences in this article an em type algorithm is developed for maximum likelihood estimation of a general nonlinear structural equation model to avoid computation of the complicated multiple integrals involved the e step is completed by a metropolis hastings algorithm it is shown that the m step can be completed efficiently by simple conditional maximization standard errors of the maximum likelihood estimates are obtained via louis s formula the methodology is illustrated with results from a simulation study and two real examples

weighted breslow type and maximum likelihood estimation in semiparametric transformation models a semiparametric transformation model comprises a parametric component for covariate effects and a nonparametric component for the baseline hazard intensity the breslow type estimator has been proposed for estimating the nonparametric component in some inefficient estimation procedures we show that introducing weights into this estimator leads to nonparametric maximum likelihood estimation with the weights depending on the martingale residuals the weighted breslow type estimator suggests an iterative reweighting algorithm for nonparametric maximum likelihood estimation which can be implemented by a weighted variant of the existing algorithms for inefficient estimation and can be computationally more efficient than an em type algorithm the weighting idea is further extended to semiparametric transformation models with mismeasured covariates

variable selection using mm algorithms variable selection is fundamental to high dimensional statistical modeling many variable selection techniques may be implemented by maximum penalized likelihood using various penalty functions optimizing the penalized likelihood function is often challenging because it may be nondifferentiable and or nonconcave this article proposes a new class of algorithms for finding a maximizer of the penalized likelihood for a broad class of penalty functions these algorithms operate by perturbing the penalty function slightly to render it differentiable then optimizing this differentiable function using a minorize maximize mm algorithm mm algorithms are useful extensions of the well known class of em algorithms a fact that allows us to analyze the local and global convergence of the proposed algorithm using some of the techniques employed for em algorithms in particular we prove that when our mm algorithms converge they must converge to a desirable point we also discuss conditions under which this convergence may be guaranteed we exploit the newton raphson like aspect of these algorithms to propose a sandwich estimator for the standard errors of the estimators our method performs well in numerical tests

estimating survival curves with left truncated and interval censored data under monotone hazards we show that the nonparametric maximum likelihood estimator npmle of a survival function may severely underestimate the survival probabilities at very early times for left truncated and interval censored data as an alternative we propose to compute the nonparametric mle under a nondecreasing hazard assumption the monotone mle by a gradient projection algorithm when the assumption holds the projection step is accomplished via an isotonic regression algorithm the pool adjacent violators algorithm this gradient projection algorithm is computationally efficient and converges globally monte carlo simulations show superior performance of the monotone mle over that of the npmle in terms of either bias or variance even for large samples the methodology is illustrated with the application to the wisconsin epidemiological study of diabetic retinopathy data to estimate the probability of incidence of retinopathy

dynamic linear models with switching abstract the problem of modeling change in a vector time series is studied using a dynamic linear model with measurement matrices that switch according to a time varying independent random process we derive filtered estimators for the usual state vectors and also for the state occupancy probabilities of the underlying nonstationary measurement process a maximum likelihood estimation procedure is given that uses a pseudo expectation maximization algorithm in the initial stages and nonlinear optimization we relate the models to those considered previously in the literature and give an application involving the tracking of multiple targets

acceleration of the em algorithm by using quasi newton methods the em algorithm is a popular method for maximum likelihood estimation its simplicity in many applications and desirable convergence properties make it very attractive its sometimes slow convergence however has prompted researchers to propose methods to accelerate it we review these methods classifying them into three groups pure hybrid and em type accelerators we propose a new pure and a new hybrid accelerator both based on quasi newton methods and numerically compare these and two other quasi newton accelerators for this we use examples in each of three areas poisson mixtures the estimation of covariance from incomplete data and multivariate normal mixtures in these comparisons the new hybrid accelerator was fastest on most of the examples and often dramatically so in some cases it accelerated the em algorithm by factors of over 100 the new pure accelerator is very simple to implement and competed well with the other accelerators it accelerated the em algorithm in some cases by factors of over 50 to obtain standard errors we propose to approximate the inverse of the observed information matrix by using auxiliary output from the new hybrid accelerator a numerical evaluation of these approximations indicates that they may be useful at least for exploratory purposes

comment feature screening and variable selection via iterative ridge regression ridge regression was originally introduced by hoerl and kennard 1970 to deal with collinearity issue in linear regression in the presence of highly correlated covariates it solves l2 penalized l

adaptive learning vector quantization for online parametric estimation this paper addresses the problem of parameter estimation in a quantized and online setting a sensing unit collects random vector valued samples from the environment these samples are quantized and transmitted to a central processor which generates an online estimate of the unknown parameter this paper provides a closed form expression of the excess mean square error mse caused by quantization in the high rate regime i e when the number of quantization levels is supposed to be large next we determine the quantizers which mitigate the excess mse the optimal quantization rule unfortunately depends on the unknown parameter to circumvent this issue we introduce a novel adaptive learning vector quantization scheme which allows to simultaneously estimate the parameter of interest and select an efficient quantizer

on the convergence properties of the em algorithm two convergence aspects of the em algorithm are studied i does the em algorithm find a local maximum or a stationary value of the incomplete data likelihood function ii does the sequence of parameter estimates generated by em converge several convergence results are obtained under conditions that are applicable to many practical situations two useful special cases are a if the unobserved complete data specification can be described by a curved exponential family with compact parameter space all the limit points of any em sequence are stationary points of the likelihood function b if the likelihood function is unimodal and a certain differentiability condition is satisfied then any em sequence converges to the unique maximum likelihood estimate a list of key properties of the algorithm is included

an em algorithm for fitting two level structural equation models maximum likelihood is an important approach to analysis of two level structural equation models different algorithms for this purpose have been available in the literature in this paper we present a new formulation of two level structural equation models and develop an em algorithm for fitting this formulation this new formulation covers a variety of two level structural equation models as a result the proposed em algorithm is widely applicable in practice a practical example illustrates the performance of the em algorithm and the maximum likelihood statistic

the use of finite difference element approaches for solving the time fractional subdiffusion equation in this paper two finite difference element approaches for the time fractional subdiffusion equation with dirichlet boundary conditions are developed in which the time direction is approximated by the fractional linear multistep method and the space direction is approximated by the finite element method the two methods are unconditionally stable and convergent of order o tau q h r 1 in the l 2 norm where q 2 beta or 2 when the analytical solution to the subdiffusion equation is sufficiently smooth beta 0 beta 1 is the order of the fractional derivative tau and h are the step sizes in time and space respectively and r is the degree of the polynomial space the corresponding schemes for the subdiffusion equation with neumann boundary conditions are presented as well where the stability and convergence are shown numerical examples are provided to verify the theoretical analysis comparisons between the algorithms derived in this paper and the existing algorithms are given

on line expectation maximization algorithm for latent data models in this contribution we propose a generic online also sometimes called adaptive or recursive version of the expectation maximisation em algorithm applicable to latent variable models of independent observations compared to the algorithm of titterington 1984 this approach is more directly connected to the usual em algorithm and does not rely on integration with respect to the complete data distribution the resulting algorithm is usually simpler and is shown to achieve convergence to the stationary points of the kullback leibler divergence between the marginal distribution of the observation and the model distribution at the optimal rate i e that of the maximum likelihood estimator in addition the proposed approach is also suitable for conditional or regression models as illustrated in the case of the mixture of linear regressions model

a gradient algorithm locally equivalent to the em algorithm in many problems of maximum likelihood estimation it is impossible to carry out either the e step or the m step of the em algorithm the present paper introduces a gradient algorithm that is closely related to the em algorithm this em gradient algorithm approximately solves the m step of the em algorithm by one iteration of newton s method since newton s method converges quickly the local properties of the em gradient algorithm are almost identical with those of the em algorithm any strict local maximum point of the observed likelihood locally attracts the em and em gradient algorithm at the same rate of convergence and near the maximum point the em gradient algorithm always produces an increase in the likelihood with proper modification the em gradient algorithm also exhibits global convergence properties that are similar to those of the em algorithm our proof of global convergence applies and improves existing theory for the em algorithm these theoretical points are reinforced by a discussion of three realistic examples illustrating how the em gradient algorithm can succeed where the em algorithm is intractable

majorization minimization algorithms in signal processing communications and machine learning this paper gives an overview of the majorization minimization mm algorithmic framework which can provide guidance in deriving problem driven algorithms with low computational cost a general introduction of mm is presented including a description of the basic principle and its convergence results the extensions acceleration schemes and connection to other algorithmic frameworks are also covered to bridge the gap between theory and practice upperbounds for a large number of basic functions derived based on the taylor expansion convexity and special inequalities are provided as ingredients for constructing surrogate functions with the pre requisites established the way of applying mm to solving specific problems is elaborated by a wide range of applications in signal processing communications and machine learning

the restricted em algorithm for maximum likelihood estimation under linear restrictions on the parameters abstract the em algorithm is one of the most powerful algorithms for obtaining maximum likelihood estimates for many incomplete data problems but when the parameters must satisfy a set of linear restrictions the em algorithm may be too complicated to apply directly in this article we propose maximum likelihood estimation procedures under a set of linear restrictions for situations in which the em algorithm could be used if there were no such restrictions on the parameters we develop a modification to the em algorithm which we call the restricted em algorithm incorporating the linear restrictions on the parameters this algorithm is easily updated by using the code for the complete data information matrix and the code for the usual em algorithm major applications of the restricted em algorithm are to construct likelihood ratio tests and profile likelihood confidence intervals we illustrate the procedure with two models a variance component model and a bivariate normal model

asymptotic convergence rate of the em algorithm for gaussian mixtures it is well known that the convergence rate of the expectation maximization em algorithm can be faster than those of convention first order iterative algorithms when the overlap in the given mixture is small but this argument has not been mathematically proved yet this article studies this problem asymptotically in the setting of gaussian mixtures under the theoretical framework of xu and jordan 1996 it has been proved that the asymptotic convergence rate of the em algorithm for gaussian mixtures locally around the true solution is o e0 5 e where e 0 is an arbitrarily small number o x means that it is a higher order infinitesimal as x 0 and e is a measure of the average overlap of gaussians in the mixture in other words the large sample local convergence rate for the em algorithm tends to be asymptotically superlinear when e tends to zero

high dimensional exploratory item factor analysis by a metropolis hastings robbins monro algorithm a metropolis hastings robbins monro mh rm algorithm for high dimensional maximum marginal likelihood exploratory item factor analysis is proposed the sequence of estimates from the mh rm algorithm converges with probability one to the maximum likelihood solution details on the computer implementation of this algorithm are provided the accuracy of the proposed algorithm is demonstrated with simulations as an illustration the proposed algorithm is applied to explore the factor structure underlying a new quality of life scale for children it is shown that when the dimensionality is high mh rm has advantages over existing methods such as numerical quadrature based em algorithm extensions of the algorithm to other modeling frameworks are discussed

unsupervised frequency tracking beyond the nyquist frequency using markov chains this paper deals with the estimation of a sequence of frequencies from a corresponding sequence of signals this problem arises in fields such as doppler imaging where its specificity is twofold first only short noisy data records are available typically four sample long and experimental constraints may cause spectral aliasing so that measurements provide unreliable ambiguous information second the frequency sequence is smooth here this information is accounted for by a markov model and application of the bayes rule yields the a posteriori density the maximum a posteriori is computed by a combination of viterbi and descent procedures one of the major features of the method is that it is entirely unsupervised adjusting the hyperparameters that balance data based and prior based information is done automatically by maximum likelihood ml using an expectation maximization em based gradient algorithm we compared the proposed estimate to a reference one and found that it performed better variance was greatly reduced and tracking was correct even beyond the nyquist frequency

fractional spectral collocation method we develop an exponentially accurate fractional spectral collocation method for solving steady state and time dependent fractional pdes fpdes we first introduce a new family of interpolants called fractional lagrange interpolants which satisfy the kronecker delta property at collocation points we perform such a construction following a spectral theory recently developed in m zayernouri and g e karniadakis j comput phys 47 2013 pp 2108 2131 for fractional sturm liouville eigenproblems subsequently we obtain the corresponding fractional differentiation matrices and we solve a number of linear fodes in addition to linear and nonlinear fpdes to investigate the numerical performance of the fractional collocation method we first examine space fractional advection diffusion problem and generalized space fractional multiterm fodes next we solve fpdes including the time and space fractional advection diffusion equation time and space fractional multiterm fpdes and finally the space

maximum pseudo likelihood estimation in network tomography network monitoring and diagnosis are key to improving network performance the difficulties of performance monitoring lie in today s fast growing internet accompanied by increasingly heterogeneous and unregulated structures moreover these tasks become even harder since one cannot rely on the collaboration of individual routers and servers to measure network traffic directly even though the aggregative nature of possible network measurements gives rise to inverse problems existing methods for solving inverse problems are usually computationally intractable or statistically inefficient a pseudo likelihood approach is proposed to solve a group of network tomography problems the basic idea of pseudo likelihood is to form simple subproblems and ignore the dependences among the subproblems to form a product likelihood of the subproblems as a result this approach keeps a good balance between the computational complexity and the statistical efficiency of the parameter estimation some statistical properties of the pseudo likelihood estimator such as consistency and asymptotic normality are established a pseudo expectation maximization em algorithm is developed to maximize the pseudo log likelihood function two examples with simulated or real data are used to illustrate the pseudo likelihood proposal 1 inference of the internal link delay distributions through multicast end to end measurements 2 origin destination matrix estimation through link traffic counts

building an identifiable latent class model with covariate effects on underlying and measured variables in recent years latent class models have proven useful for analyzing relationships between measured multiple indicators and covariates of interest such models summarize shared features of the multiple indicators as an underlying categorical variable and the indicators substantive associations with predictors are built directly and indirectly in unique model parameters in this paper we provide a detailed study on the theory and application of building models that allow mediated relationships between primary predictors and latent class membership but that also allow direct effects of secondary covariates on the indicators themselves theory for model identification is developed we detail an expectation maximization algorithm for parameter estimation standard error calculation and convergent properties comparison of the proposed model with models underlying existing latent class modeling software is provided a detailed analysis of how visual impairments affect older persons functioning requiring distance vision is used for illustration