a variational formulation of the fpk equations with application to the first passage problem in random vibration by using the technique of eigenfunction expansions a variational formulation of the fokker planck kolmogorov fpk equations is developed in which the boundary conditions are incorporated within the functional this allows an approximate solution to the equations to be sought by using the rayleight ritz procedure with trial functions which do not satisfy the often complex boundary conditions application of the method to the first passage problem in random vibration shows that the common assumption of independent outcrossings from the safe region is equivalent to a one term rayleigh ritz solution of the fpk equations in which the exact solutions to the stationary vibration problem are used as trial functions the method is applied to the first passage probability of a linear oscillator and duffing s oscillator and good agreement with published results is found

predictive updating methods with application to bayesian classification we propose algorithms based on random draws from predictive distributions of unknown quantities missing values for instance this procedure can either be iterative which is a special variation of the gibbs sampler or be sequential which is a variation of sequential imputation in the latter case one can update the posterior distribution with new observations easily the methods proposed have intuitive statistical implications and can be generalized to accommodate other bayesian like procedures we display some applications of the method in connection with the bayesian bootstrap classification hierarchical models and selection of variables in particular as an application of the method we present a unified treatment of switching regression models driven by a general binary process and we develop a bayesian testing procedure some simulations and a real example are used to illustrate the methods proposed

transient fokker planck kolmogorov equation solved with smoothed particle hydrodynamics method summary probabilistic theories aim at describing the properties of systems subjected to random excitations by means of statistical characteristics such as the probability density function pdf the time evolution of the pdf of the response of a randomly excited deterministic system is commonly described with the transient fokker planck kolmogorov fpk equation the fpk equation is a conservation equation of a hypothetical or abstract fluid which models the transport of probability this paper presents a generalized formalism for the resolution of the transient fpk equation by using the well known mesh free lagrangian method smoothed particle hydrodynamics numerical implementation shows notable advantages of this method in an unbounded state space 1 the conservation of total probability in the state space is explicitly written 2 no artifact is required to manage far field boundary conditions 3 the positivity of the pdf is ensured and 4 the extension to higher dimensions is straightforward furthermore thanks to the moving particles this method is adapted for a large kind of initial conditions even slightly dispersed distributions the fpk equation is solved without any a priori knowledge of the stationary distribution just a precise representation of the initial distribution is required copyright 2013 john wiley sons ltd

variational inclusions with a general h monotone operator in banach spaces in this paper we introduce a new class of operator general h monotone operators in banach space we define a proximal mapping associated with the general h monotone operator and show its lipschitz continuity we also consider a new class of variational inclusions involving these general h monotone operators and constructed a new iterative algorithm for solving the variational inclusion in banach spaces under some suitable conditions we prove the convergence of the iterative sequence generated by the algorithm

predictor corrector methods for general mixed quasi variational inequalities in this paper we use the auxiliary principle technique to suggest a class of predictor corrector methods for solving general mixed quasi variational inequalities the convergence of the proposed methods only requires the partially relaxed strongly monotonicity of the operator which is weaker than co coercivity as special cases we obtain various known and new results for solving various classes of variational inequalities and related problems

estimating latent distributions in recurrent choice data this paper introduces a flexible class of stochastic mixture models for the analysis and interpretation of individual differences in recurrent choice and other types of count data these choice models are derived by specifying elements of the choice process at the individual level probability distributions are introduced to describe variations in the choice process among individuals and to obtain a representation of the aggregate choice behavior due to the explicit consideration of random effect sources the choice models are parsimonious and readily interpretable an easy to implement em algorithm is presented for parameter estimation two applications illustrate the proposed approach

algorithm for solving a new class of generalized nonlinear implicit quasi variational inclusions in banach spaces abstract in this paper we introduce and study a new class of generalized nonlinear implicit quasi variational inclusions with general h monotone operator in banach spaces by using the proximal mapping and nadler s theorem we suggest and analyze a class of iterative algorithm for solving the generalized nonlinear implicit quasi variational inclusion first an existence theorem of solutions for the generalized nonlinear implicit quasi variational inclusion is proved under suitable assumptions then the convergence of the iterative sequences generated by the new algorithm is proved

time sequence diffeomorphic metric mapping and parallel transport track time dependent shape changes abstract serial mri human brain scans have facilitated the detection of brain development and of the earliest signs of neuropsychiatric and neurodegenerative diseases monitoring disease progression and resolving drug effects in clinical trials for preventing or slowing the rate of brain degeneration to track anatomical shape changes in serial images we introduce new point based time sequence large deformation diffeomorphic metric mapping ts lddmm to infer the time flow of within subject geometric shape changes that carry known observations through a period its euler lagrange equation is generalized for anatomies whose shapes are characterized by point sets such as landmarks curves and surfaces the time dependent momentum obtained from the ts lddmm encodes within subject shape changes for the purpose of across subject shape comparison we then propose a diffeomorphic analysis framework to translate within subject deformation in a global template without incorporating across subject anatomical variations via parallel transport technique the analysis involves the retraction of the within subject time dependent momentum along the ts lddmm trajectory from each time to the baseline the translation of the momentum in a global template and the reconstruction of the ts lddmm trajectory starting from the global template

dealing with label switching in mixture models summary in a bayesian analysis of finite mixture models parameter estimation and clustering are sometimes less straightforward than might be expected in particular the common practice of estimating parameters by their posterior mean and summarizing joint posterior distributions by marginal distributions often leads to nonsensical answers this is due to the so called label switching problem which is caused by symmetry in the likelihood of the model parameters a frequent response to this problem is to remove the symmetry by using artificial identifiability constraints we demonstrate that this fails in general to solve the problem and we describe an alternative class of approaches relabelling algorithms which arise from attempting to minimize the posterior expected loss under a class of loss functions we describe in detail one particularly simple and general relabelling algorithm and illustrate its success in dealing with the label switching problem on two examples

mixture densities maximum likelihood and the em algorithm the problem of estimating the parameters which determine a mixture density has been the subject of a large diverse body of literature spanning nearly ninety years during the last two decades the

on convergence and parameter selection of the em and da em algorithms for gaussian mixtures abstract the expectation maximization em for gaussian mixtures is popular as a clustering algorithm however the em algorithm is sensitive to initial values and so ueda and nakano 4 proposed the deterministic annealing em da em algorithm to improve it in this paper we investigate theoretical behaviors of the em and da em algorithms we first derive a general jacobian matrix of the da em algorithm with respect to posterior probabilities we then propose a theoretical lower bound for initialization of the annealing parameter in the da em algorithm on the other hand some researches mentioned that the em algorithm exhibits a self annealing behavior that is the equal posterior probability with small random perturbations can avoid the em algorithm to output the mass center for gaussian mixtures however there is no theoretical analysis on this self annealing property since the da em will become the em when the annealing parameter is 1 according to the jacobian matrix of the da em we can prove the self annealing property of the em algorithm for gaussian mixtures based on these results we give not only convergence behaviors of the equal posterior probabilities and initialization lower bound of the temperature parameter of the da em but also a theoretical explanation why the em algorithm for gaussian mixtures exhibits a self annealing behavior

the partition of unity finite element approach with hp refinement for the stationary fokker planck equation abstract in this paper the stationary fokker planck equation fpe is solved for nonlinear dynamical systems using a local numerical technique based on the meshless partition of unity finite element method pufem the method is applied to stationary fpe for 2 3 and 4 d systems and is argued to be an excellent candidate for higher dimensional problems and the transient problem local refinement is applied by introducing higher order polynomials in selected subdomains local p refinement to keep the problem size small while ensuring high approximation accuracy various local approximations are blended using novel pasting functions that provide any desired order of continuity results are compared with existing global and local techniques local p refinement is touted as an important step towards breaking the curse of dimensionality in numerical solution of fpe

hypergraph based anomaly detection of high dimensional co occurrences this paper addresses the problem of detecting anomalous multivariate co occurrences using a limited number of unlabeled training observations a novel method based on using a hypergraph representation of the data is proposed to deal with this very high dimensional problem hypergraphs constitute an important extension of graphs which allow edges to connect more than two vertices simultaneously a variational expectation maximization algorithm for detecting anomalies directly on the hypergraph domain without any feature selection or dimensionality reduction is presented the resulting estimate can be used to calculate a measure of anomalousness based on the false discovery rate the algorithm has o np computational complexity where n is the number of training observations and p is the number of potential participants in each co occurrence event this efficiency makes the method ideally suited for very high dimensional settings and requires no tuning bandwidth or regularization parameters the proposed approach is validated on both high dimensional synthetic data and the enron email database where p 75 000 and it is shown that it can outperform other state of the art methods

a new class of completely generalized quasi variational inclusions in banach spaces in this paper a new notion of j proximal mapping for a nonconvex lower semicontinuous subdifferentiable proper functional on banach space is introduced the existence and lipschitz continuity of j proximal mapping of a lower semicontinuous subdifferentiable proper functional are proved by applying the concept we introduce and study a new class of completely generalized quasi variational inclusions in reflexive banach spaces a novel and innovative iterative algorithm for finding the approximate solutions is suggested and analyzed the convergence criteria of the iterative sequences generated by the new iterative algorithm is also given these algorithm and existence result generalize many known results under hilbert space setting in recent literature to reflexive banach spaces

automatic segmentation of the left ventricle in cardiac mr and ct images this paper describes a segmentation technique to automatically extract the myocardium in 4d cardiac mr and ct datasets the segmentation algorithm is a two step process the global localization step roughly localizes the left ventricle using techniques such as maximum discrimination thresholding and connected component analysis the local deformations step combines em based region segmentation and dijkstra active contours using graph cuts spline fitting or point pattern matching the technique has been tested on a large number of patients and both quantitative and qualitative results are presented

semi supervised cross entropy clustering with information bottleneck constraint abstract in this paper we propose a semi supervised clustering method cec ib that models data with a set of gaussian distributions and that retrieves clusters based on a partial labeling provided by the user partition level side information by combining the ideas from cross entropy clustering cec with those from the information bottleneck method ib our method trades between three conflicting goals the accuracy with which the data set is modeled the simplicity of the model and the consistency of the clustering with side information experiments demonstrate that cec ib performs similar as gaussian mixture models in a classical semi supervised scenario but is faster more robust to noisy labels automatically determines the optimal number of clusters and performs well when not all classes are present in the side information moreover in contrast to many other semi supervised models it can be successfully applied in discovering natural subgroups if the partition level side information is derived from the top levels of a hierarchical clustering

maximum likelihood estimation of latent interaction effects with the lms method in the context of structural equation modeling a general interaction model with multiple latent interaction effects is introduced a stochastic analysis represents the nonnormal distribution of the joint indicator vector as a finite mixture of normal distributions the latent moderated structural equations lms approach is a new method developed for the analysis of the general interaction model that utilizes the mixture distribution and provides a ml estimation of model parameters by adapting the em algorithm the finite sample properties and the robustness of lms are discussed finally the applicability of the new method is illustrated by an empirical example

cluster number selection for a small set of samples using the bayesian ying yang model one major problem in cluster analysis is the determination of the number of clusters in this paper we describe both theoretical and experimental results in determining the cluster number for a small set of samples using the bayesian kullback ying yang byy model selection criterion under the second order approximation we derive a new equation for estimating the smoothing parameter in the cost function finally we propose a gradient descent smoothing parameter estimation approach that avoids complicated integration procedure and gives the same optimal result

application of the em algorithm for the multitarget multisensor tracking problem an important problem in surveillance and reconnaissance systems is the tracking of multiple moving targets in cluttered noise environments using outputs from a number of sensors possessing wide variations in individual characteristics and accuracies a number of approaches have been proposed for this multitarget multisensor tracking problem ranging from reasonably simple though ad hoc schemes to fairly complex but theoretically optimum approaches in this paper we describe an iterative procedure for time recursive multitarget multisensor tracking based on use of the expectation maximization em algorithm more specifically we pose the multitarget multisensor tracking problem as an incomplete data problem with the observable sensor outputs representing the incomplete data whereas the target associated sensor outputs constitute the complete data target updates at each time use an em based approach that calculates the maximum a posteriori map estimate of the target states under the assumption of appropriate motion models based on the outputs of disparate sensors the approach uses a markov random field mrf model of the associations between observations and targets and allows for estimation of joint association probabilities without explicit enumeration the advantage of this em based approach is that it provides a computationally efficient means for approaching the performance offered by theoretically optimum approaches that use explicit enumeration of the joint association probabilities we provide selected results illustrating the performance complexity characteristics of this em based approach compared with competing schemes

snakes active contour models a snake is an energy minimizing spline guided by external constraint forces and influenced by image forces that pull it toward features such as lines and edges snakes are active contour models they lock onto nearby edges localizing them accurately scale space continuation can be used to enlarge the capture region surrounding a feature snakes provide a unified account of a number of visual problems including detection of edges lines and subjective contours motion tracking and stereo matching we have used snakes successfully for interactive interpretation in which user imposed constraint forces guide the snake near features of interest

feature selection based on the approximation of class densities by finite mixtures of special type abstract a new method of feature selection based on the approximation of class conditional densities by a mixture of parameterized densities of a special type suitable especially for multimodal data is presented no search procedure is needed when using the proposed method its performance is tested both on real simulated data

a constrained formulation of maximum likelihood estimation for normal mixture distributions in this context the normal densities are sometimes referred to as component densities the log likelihood function l zy corresponding to a random sample xl xnj is defined by l ty ek 1 10g pm xk y the first problem associated with maximum likelihood estimation arises from the unboundedness of l qy on r day 1969 a global maximum likelihood estimate always fails to exist in addition the unboundedness of l y causes failures of optimization algorithms of both the em redner and walker 1984 and quasi newton fowlkes 1979 types in spite of the unboundedness of l 4y statistical theory kiefer 1978 guarantees that a particular local maximizer of l y is strongly consistent and asymptotically efficient several local maximizers can exist for a given sample and the other major maximum likelihood difficulty is in determining when the correct one has been found day 1969 noted that spurious maximizers corresponding to parameter points having some component standard deviations very small relative to others are generated by any small number of sample points grouped sufficiently close together the spurious maximizers like the unboundedness of l zy can create difflculties when using the em or quasi newton algorithms the constrained maximum likelihood formulation presented in the next section avoids the unconstrained problems of singularities and spurious maximizers of the type noted by day section 3 contains the statement and proof of a consistency result for the corresponding constrained estimator

probabilistic solution of some multi degree of freedom nonlinear systems under external independent poisson white noises this paper studies the stationary probability density function pdf of the response of multi degree of freedom nonlinear systems under external independent poisson white noises the pdf is governed by the high dimensional generalized fokker planck kolmogorov fpk equation the state space split 3s method is adopted to reduce the high dimensional generalized fpk equation to a low dimensional equation subsequently the exponential polynomial closure epc method is further used to solve the reduced fpk equation for the pdf solution two illustrative examples are presented to examine the accuracy of the 3s epc solution procedure one example involves a two degree of freedom coupled nonlinear system the other example is concerned with a ten degree of freedom system with cubic terms in displacement a monte carlo simulation is also performed for simulating the pdf solution of the response the comparison with the simulated result shows that the 3s epc solution procedure can provide satisfactory pdf solut

anderson acceleration for fixed point iterations this paper concerns an acceleration method for fixed point iterations that originated in work of d g anderson j assoc comput mach 12 1965 pp 547 560 which we accordingly call anderson acceleration here this method has enjoyed considerable success and wide usage in electronic structure computations where it is known as anderson mixing however it seems to have been untried or underexploited in many other important applications moreover while other acceleration methods have been extensively studied by the mathematics and numerical analysis communities this method has received relatively little attention from these communities over the years a recent paper by h fang and y saad numer linear algebra appl 16 2009 pp 197 221 has clarified a remarkable relationship of anderson acceleration to quasi newton secant updating methods and extended it to define a broader anderson family of acceleration methods in this paper our goals are to shed additional light on anderson acceleration and to draw further attention to its usefulness as a general tool we first show that on linear problems anderson acceleration without truncation is essentially equivalent in a certain sense to the generalized minimal residual gmres method we also show that the type 1 variant in the fang saad anderson family is similarly essentially equivalent to the arnoldi full orthogonalization method we then discuss practical considerations for implementing anderson acceleration and illustrate its performance through numerical experiments involving a variety of applications

a finite element method for the statistics of non linear random vibration the transitional probability density function for the random response of a certain class of non linear system satisfies the fokker planck kolmogorov equation this paper concerns the numerical solution of the stationary form of this equation yielding the stationary probability density function of response the weighted residual statement for the problem is integrated by parts to yield the weak form of the equations which are then solved by the finite element method the method is applied to a duffing oscillator and good agreement is found with the exact result and the method is compared favourably with a galerkin solution method given by bhandari and sherrer 1 also the method is applied to the ship rolling problem and good agreement is found with an approximate analytical result due to roberts 2

assessing a mixture model for clustering with the integrated completed likelihood we propose an assessing method of mixture model in a cluster analysis setting with integrated completed likelihood for this purpose the observed data are assigned to unknown clusters using a maximum a posteriori operator then the integrated completed likelihood icl is approximated using the bayesian information criterion bic numerical experiments on simulated and real data of the resulting icl criterion show that it performs well both for choosing a mixture model and a relevant number of clusters in particular icl appears to be more robust than bic to violation of some of the mixture model assumptions and it can select a number of dusters leading to a sensible partitioning of the data

finite mixture varying coefficient models for analyzing longitudinal heterogenous data this paper aims to develop a mixture model to study heterogeneous longitudinal data on the treatment effect of heroin use from a california civil addict program each component of the mixture is characterized by a varying coefficient mixed effect model we use the bayesian p splines approach to approximate the varying coefficient functions we develop markov chain monte carlo algorithms to estimate the smooth functions unknown parameters and latent variables in the model we use modified deviance information criterion to determine the number of components in the mixture a simulation study demonstrates that the modified deviance information criterion selects the correct number of components and the estimation of unknown quantities is accurate we apply the proposed model to the heroin treatment study furthermore we identify heterogeneous longitudinal patterns copyright 2011 john wiley sons ltd

gaussian mixture density modeling decomposition and applications we present a new approach to the modeling and decomposition of gaussian mixtures by using robust statistical methods the mixture distribution is viewed as a contaminated gaussian density using this model and the model fitting mf estimator we propose a recursive algorithm called the gaussian mixture density decomposition gmdd algorithm for successively identifying each gaussian component in the mixture the proposed decomposition scheme has advantages that are desirable but lacking in most existing techniques in the gmdd algorithm the number of components does not need to be specified a priori the proportion of noisy data in the mixture can be large the parameter estimation of each component is virtually initial independent and the variability in the shape and size of the component densities in the mixture is taken into account gaussian mixture density modeling and decomposition has been widely applied in a variety of disciplines that require signal or waveform characterization for classification and recognition we apply the proposed gmdd algorithm to the identification and extraction of clusters and the estimation of unknown probability densities probability density estimation by identifying a decomposition using the gmdd algorithm that is a superposition of normal distributions is successfully applied to automated cell classification computer experiments using both real data and simulated data demonstrate the validity and power of the gmdd algorithm for various models and different noise assumptions

an incremental nested partition method for data clustering clustering methods are a powerful tool for discovering patterns in a given data set through an organization of data into subsets of objects that share common features motivated by the independent use of some different partitions criteria and the theoretical and empirical analysis of some of its properties in this paper we introduce an incremental nested partition method which combines these partitions criteria for finding the inner structure of static and dynamic datasets for this we proved that there are relationships of nesting between partitions obtained respectively from these partition criteria and besides that the sensitivity when a new object arrives to the dataset is rigorously studied our algorithm exploits all of these mathematical properties for obtaining the hierarchy of clusterings moreover we realize a theoretical and experimental comparative study of our method with classical hierarchical clustering methods such as single link and complete link and other more recently introduced methods the experimental results over databases of uci repository and the afp and tdt2 news collections show the usefulness and capability of our method to reveal different levels of information hidden in datasets

embedding overlap priors in variational left ventricle tracking we propose to embed overlap priors in variational tracking of the left ventricle lv in cardiac magnetic resonance mr sequences the method consists of evolving two curves toward the lv endo and epicardium boundaries we derive the curve evolution equations by minimizing two functionals each containing an original overlap prior constraint the latter measures the conformity of the overlap between the nonparametric kernel based intensity distributions within the three target regions lv cavity myocardium and background to a prior learned from a given segmentation of the first frame the bhattacharyya coefficient is used as an overlap measure different from existing intensity driven constraints the proposed priors do not assume implicitly that the overlap between the intensity distributions within different regions has to be minimal this prevents both the papillary muscles from being included erroneously in the myocardium and the curves from spilling into the background although neither geometric training nor preprocessing were used quantitative evaluation of the similarities between automatic and independent manual segmentations showed that the proposed method yields a competitive score in comparison with existing methods this allows more flexibility in clinical use because our solution is based only on the current intensity data and consequently the results are not bounded to the characteristics variability and mathematical description of a finite training set we also demonstrate experimentally that the overlap measures are approximately constant over a cardiac sequence which allows to learn the overlap priors from a single frame

a latent class mixture model for incomplete longitudinal gaussian data in the analyses of incomplete longitudinal clinical trial data there has been a shift away from simple methods that are valid only if the data are missing completely at random to more principled ignorable analyses which are valid under the less restrictive missing at random assumption the availability of the necessary standard statistical software nowadays allows for such analyses in practice while the possibility of data missing not at random mnar cannot be ruled out it is argued that analyses valid under mnar are not well suited for the primary analysis in clinical trials rather than either forgetting about or blindly shifting to an mnar framework the optimal place for mnar analyses is within a sensitivity analysis context one such route for sensitivity analysis is to consider next to selection models pattern mixture models or shared parameter models the latter can also be extended to a latent class mixture model the approach taken in this article the performance of the so obtained flexible model is assessed through simulations and the model is applied to data from a depression trial

combined analytical and numerical approaches in dynamic stability analyses of engineering systems abstract dynamic stability is a widely studied area that has attracted many researchers from various disciplines although dynamic stability is usually associated with mechanics theoretical physics or other natural and technical disciplines it is also relevant to social economic and philosophical areas of our lives therefore it is useful to occasionally highlight the general aspects of this amazing area to present some relevant examples and to evaluate its position among the various branches of rational mechanics from this perspective the aim of this study is to present a brief review concerning the dynamic stability problem its basic definitions and principles important phenomena research motivations and applications in engineering the relationships with relevant systems that are prone to stability loss encountered in other areas such as physics other natural sciences and engineering are also noted the theoretical background which is applicable to many disciplines is presented in this paper the most frequently used dynamic stability analysis methods are presented in relation to individual dynamic systems that are widely discussed in various engineering branches in particular the lyapunov function and exponent procedures routh hurwitz lienard and other theorems are outlined together with demonstrations the possibilities for analytical and numerical procedures are mentioned together with possible feedback from experimental research and testing the strengths and shortcomings of these approaches are evaluated together with examples of their effective complementing of each other the systems that are widely encountered in engineering are presented in the form of mathematical models the analyses of their dynamic stability and post critical behaviour are also presented the stability limits bifurcation points quasi periodic response processes and chaotic regimes are discussed the limit cycle existence and stability are examined together with their separating roles as attractors and repulsers two levels of stability loss recovery of the system is possible or final collapse is inevitable as can be observed in softening systems are noted time limited excitation and relevant transition effects e g seismic excitation are also discussed together with the evaluation of possible system reliability improvement the dynamic stability investigation of two degrees of freedom aero elastic systems in a linear formulation using several approaches is briefly highlighted further systems modelling problems that arise in transport engineering are also outlined a few hints for applications are given some open problems and possible future research strategies are outlined

semiparametric estimation of normal mixture densities a semiparametric method for estimating densities of normal mean mixtures is presented this consistent data driven method of estimation is based on probability spacings the estimation technique involves iteratively fixing the standard deviation of the normal kernel that serves as a smoothing parameter and then maximizing a function of the probability spacings over all mixing distributions based on the distribution of uniform spacings a distribution free goodness of fit criterion is developed to guide the selection of the smoothing parameter the result is a set of consistent estimators indexed by a range of smoothing parameters empirical process results are used to prove consistency

a study of regularized gaussian classifier in high dimension small sample set case based on mdl principle with application to spectrum recognition in classifying high dimensional patterns such as stellar spectra by a gaussian classifier the covariance matrix estimated with a small number sample set becomes unstable leading to degraded classification accuracy in this paper we investigate the covariance matrix estimation problem for small number samples with high dimension setting based on minimum description length mdl principle a new covariance matrix estimator is developed and a formula for fast estimation of regularization parameters is derived experiments on spectrum pattern recognition are conducted to investigate the classification accuracy with the developed covariance matrix estimator higher classification accuracy results are obtained and demonstrated in our new approach

analysis of the power flow in nonlinear oscillators driven by random excitation using the first wiener kernel abstract random excitation of mechanical systems occurs in a wide variety of structures and in some applications calculation of the power dissipated by such a system will be of interest in this paper using the wiener series a general methodology is developed for calculating the power dissipated by a general nonlinear multi degree of freedom oscillatory system excited by random gaussian base motion of any spectrum the wiener series method is most commonly applied to systems with white noise inputs but can be extended to encompass a general non white input from the extended series a simple expression for the power dissipated can be derived in terms of the first term or kernel of the series and the spectrum of the input calculation of the first kernel can be performed either via numerical simulations or from experimental data and a useful property of the kernel namely that the integral over its frequency domain representation is proportional to the oscillating mass is derived the resulting equations offer a simple conceptual analysis of the power flow in nonlinear randomly excited systems and hence assist the design of any system where power dissipation is a consideration the results are validated both numerically and experimentally using a base excited cantilever beam with a nonlinear restoring force produced by magnets

hidden markov processes an overview of statistical and information theoretic aspects of hidden markov processes hmps is presented an hmp is a discrete time finite state homogeneous markov chain observed through a discrete time memoryless invariant channel in recent years the work of baum and petrie 1966 on finite state finite alphabet hmps was expanded to hmps with finite as well as continuous state spaces and a general alphabet in particular statistical properties and ergodic theorems for relative entropy densities of hmps were developed consistency and asymptotic normality of the maximum likelihood ml parameter estimator were proved under some mild conditions similar results were established for switching autoregressive processes these processes generalize hmps new algorithms were developed for estimating the state parameter and order of an hmp for universal coding and classification of hmps and for universal decoding of hidden markov channels these and other related topics are reviewed

a note on the accuracy of stochastic averaging applied to non linear oscillator response prediction abstract stationary response statistics derived from a non linear single degree of freedom oscillator driven by broadband noise are obtained by using the stochastic averaging method a fully numerical implementation of the method is used to approximate large amplitude mean threshold crossing intervals and comparisons with time domain simulations are presented by uniformly increasing the level of non linear damping both the accuracy and the limitations of the method are demonstrated

h φ η monotone operators in banach spaces with an application to variational inclusions in this paper we introduce and study a new class of h f email protected operators in banach spaces we define a proximal mapping associated with the h f email protected operator and show its lipschitz continuity as an application we consider the solvability for a class of variational inclusions involving the h f email protected operators in banach spaces by using the technique of proximal mapping an iterative algorithm for solving such class of variational inclusions is constructed in banach spaces under some suitable conditions we prove the convergence of iterative sequence generated by the algorithm

on the method of logarithmic cumulants for parametric probability density function estimation parameter estimation of probability density functions is one of the major steps in the area of statistical image and signal processing in this paper we explore several properties and limitations of the recently proposed method of logarithmic cumulants molc parameter estimation approach which is an alternative to the classical maximum likelihood ml and method of moments mom approaches we derive the general sufficient condition for a strong consistency of the molc estimates which represents an important asymptotic property of any statistical estimator this result enables the demonstration of the strong consistency of molc estimates for a selection of widely used distribution families originating from but not restricted to synthetic aperture radar image processing we then derive the analytical conditions of applicability of molc to samples for the distribution families in our selection finally we conduct various synthetic and real data experiments to assess the comparative properties applicability and small sample performance of molc notably for the generalized gamma and k families of distributions supervised image classification experiments are considered for medical ultrasound and remote sensing sar imagery the obtained results suggest that molc is a feasible and computationally fast yet not universally applicable alternative to mom molc becomes especially useful when the direct ml approach turns out to be unfeasible

an alternative to em for gaussian mixture models batch and stochastic riemannian optimization we consider maximum likelihood estimation for gaussian mixture models gmm s this task is almost invariably solved in theory and practice via the expectation maximization em algorithm em owes its success to various factors of which is its ability to fulfill positive definiteness constraints in closed form is of key importance we propose an alternative to em grounded in the riemannian geometry of positive definite matrices using which we cast gmm parameter estimation as a riemannian optimization problem surprisingly such an out of the box riemannian formulation completely fails and proves much inferior to em this motivates us to take a closer look at the problem geometry and derive a better formulation that is much more amenable to riemannian optimization we then develop riemannian batch and stochastic gradient algorithms that outperform em often substantially we provide a non asymptotic convergence analysis for our stochastic method which is also the first to our knowledge such global analysis for riemannian stochastic gradient numerous empirical results are included to demonstrate the effectiveness of our methods

on simultaneous signal estimation and parameter identification using a generalized likelihood approach a common approach to blind deconvolution of bernoulli gaussian processes consists of performing both signal restoration and hyperparameter identification through maximization of a single generalized likelihood criterion it is shown on a simple example that the resulting hyperparameter estimates may not converge toward any meaningful value therefore other more reliable approaches should be adopted whenever possible

semiparametric maximum likelihood for measurement error model regression this paper presents an em algorithm for semiparametric likelihood analysis of linear generalized linear and nonlinear regression models with measurement errors in explanatory variables a structural model is used in which probability distributions are specified for a the response and b the measurement error a distribution is also assumed for the true explanatory variable but is left unspecified and is estimated by nonparametric maximum likelihood for various types of extra information about the measurement error distribution the proposed algorithm makes use of available routines that would be appropriate for likelihood analysis of a and b if the true x were available simulations suggest that the semiparametric maximum likelihood estimator retains a high degree of efficiency relative to the structural maximum likelihood estimator based on correct distributional assumptions and can outperform maximum likelihood based on an incorrect distributional assumption the approach is illustrated on three examples with a variety of structures and types of extra information about the measurement error distribution

a new class of variational inclusions with b monotone operators in banach spaces in this paper we introduce and study a new class of variational inclusions in banach spaces for solving such class of variational inclusions we introduce a new notion of b monotone operator and prove the lipschitz continuity of the proximal mapping associated with the b monotone operator by using the proximal mapping an iterative algorithm for solving such class of variational inclusions is constructed in banach spaces under some suitable conditions we prove the convergence of iterative sequence generated by the algorithm

a hybrid sem algorithm for high dimensional unsupervised learning using a finite generalized dirichlet mixture this paper applies a robust statistical scheme to the problem of unsupervised learning of high dimensional data we develop analyze and apply a new finite mixture model based on a generalization of the dirichlet distribution the generalized dirichlet distribution has a more general covariance structure than the dirichlet distribution and offers high flexibility and ease of use for the approximation of both symmetric and asymmetric distributions we show that the mathematical properties of this distribution allow high dimensional modeling without requiring dimensionality reduction and thus without a loss of information this makes the generalized dirichlet distribution more practical and useful we propose a hybrid stochastic expectation maximization algorithm hsem to estimate the parameters of the generalized dirichlet mixture the algorithm is called stochastic because it contains a step in which the data elements are assigned randomly to components in order to avoid convergence to a saddle point the adjective hybrid is justified by the introduction of a newton raphson step moreover the hsem algorithm autonomously selects the number of components by the introduction of an agglomerative term the performance of our method is tested by the classification of several pattern recognition data sets the generalized dirichlet mixture is also applied to the problems of image restoration image object recognition and texture image database summarization for efficient retrieval for the texture image summarization problem results are reported for the vistex texture image database from the mit media lab

algorithm for solving a new class of general mixed variational inequalities in banach spaces in this paper a new concept of h proximal mapping for a proper subdifferentiable functional which may not be convex on a banach space is introduced an existence and lipschitz continuity of the h proximal mapping are proved by using properties of the h proximal mapping a new class of general mixed variational inequalities is introduced and studied in banach spaces an existence theorem of solutions is established and a new iterative algorithm for solving the general mixed variational inequality is suggested a convergence criteria of the iterative sequence generated by the new algorithm is also given

parameter estimation and tissue segmentation from multispectral mr images a statistical method is developed to classify tissue types and to segment the corresponding tissue regions from relaxation time t sub 1 t sub 2 and proton density p sub d weighted magnetic resonance images the method assumes that the distribution of image intensities associated with each tissue type can be expressed as a multivariate likelihood function of three weighted signal intensity values t sub 1 t sub 2 p sub d at each location within that tissue regions the method further assumes that the underlying tissue regions are piecewise contiguous and can be characterized by a markov random field prior in classifying the tissue types the method models the likelihood of realizing the images as a finite multivariate mixture function the class parameters associated with the tissue types i e the weighted intensity means variances and correlation coefficients of the multivariate function as well as the number of voxels within regions of the tissue types of are estimated by maximum likelihood the estimation fits the class parameters to the image data via the expectation maximization algorithm the number of classes associated with the tissue types is determined by the information criterion of minimum description length the method segments the tissue regions given the estimated class parameters by maximum a posteriori probability the prior is constructed by the tissue region membership of the first and second order neighborhood the method is tested by a few sets of t sub 1 t sub 2 and p sub d weighted images of the brain acquired with a 1 5 tesla whole body scanner the number of classes and the associated class parameters are automatically estimated the regions of different brain tissues are satisfactorily segmented

sar amplitude probability density function estimation based on a generalized gaussian model in the context of remotely sensed data analysis an important problem is the development of accurate models for the statistics of the pixel intensities focusing on synthetic aperture radar sar data this modeling process turns out to be a crucial task for instance for classification or for denoising purposes in this paper an innovative parametric estimation methodology for sar amplitude data is proposed that adopts a generalized gaussian gg model for the complex sar backscattered signal a closed form expression for the corresponding amplitude probability density function pdf is derived and a specific parameter estimation algorithm is developed in order to deal with the proposed model specifically the recently proposed method of log cumulants molc is applied which stems from the adoption of the mellin transform instead of the usual fourier transform in the computation of characteristic functions and from the corresponding generalization of the concepts of moment and cumulant for the developed gg based amplitude model the resulting molc estimates turn out to be numerically feasible and are also analytically proved to be consistent the proposed parametric approach was validated by using several real ers 1 xsar e sar and nasa jpl airborne sar images and the experimental results prove that the method models the amplitude pdf better than several previously proposed parametric models for backscattering phenomena

maximum likelihood array processing in non gaussian noise with gaussian mixtures many approaches have been studied for the array processing problem when the additive noise is modeled with a gaussian distribution but these schemes typically perform poorly when the noise is non gaussian and or impulsive this paper is concerned with maximum likelihood array processing in non gaussian noise we present the cramer rao bound on the variance of angle of arrival estimates for arbitrary additive independent identically distributed iid symmetric non gaussian noise then we focus on non gaussian noise modeling with a finite gaussian mixture distribution which is capable of representing a broad class of non gaussian distributions that include heavy tailed impulsive cases arising in wireless communications and other applications based on the gaussian mixture model we develop an expectation maximization em algorithm for estimating the source locations the signal waveforms and the noise distribution parameters the important problems of detecting the number of sources and obtaining initial parameter estimates for the iterative em algorithm are discussed in detail the initialization procedure by itself is an effective algorithm for array processing in impulsive noise novel features of the em algorithm and the associated maximum likelihood formulation include a nonlinear beamformer that separates multiple source signals in non gaussian noise and a robust covariance matrix estimate that suppresses impulsive noise while also performing a model based interpolation to restore the low rank signal subspace the em approach yields improvement over initial robust estimates and is valid for a wide snr range the results are also robust to pdf model mismatch and work well with infinite variance cases such as the symmetric stable distributions simulations confirm the optimality of the em estimation procedure in a variety of cases including a multiuser communications scenario we also compare with existing array processing algorithms for non gaussian noise

integral global optimization method in statistical applications abstract many statistical computations need a minimization method which can find global minima of a nonconvex nonsmooth and even discontinuous objective function conventional optimization methods hardly serve that the integral optimization method has been developed to satisfy such requirements in this paper we apply this method to several problems in statistics as nonlinear regression maximum likelihood estimation for three parameter weibull distribution mixture densities and sampling and show that the results obtained by a global minimization method are better than those by a gradient based one

numerical solution of the stationary fpk equation using shannon wavelets abstract the fokker planck kolmogorov fpk equation governs the probability density function p d f of the dynamic response of a particular class of linear or non linear system to random excitation this paper proposes a numerical method for calculating the stationary solution of the fpk equation which is based upon the weighted residual approach using shannon wavelets as shape functions the method is developed here for an n dimensional system and its relationship with the distributed approximating functional daf approach is investigated for the purposes of validation numerical results obtained using the proposed method are compared with available exact solutions and numerical solutions for some non linear oscillators for the systems considered excellent results over the main body and tails of the marginal distributions are obtained the accuracy and efficiency of the method are investigated in comparison to the finite element method fem

a bayesian approach to finite mixture models in bioassay via data augmentation and gibbs sampling and its application to insecticide resistance after continued treatment with an insecticide within the population of the susceptible insects resistant strains will occur it is important to know whether there are any resistant strains what the proportions are and what the median lethal doses are for the insecticide lwin and martin 1989 biometrics 45 721 732 propose a probit mixture model and use the em algorithm to obtain the maximum likelihood estimates for the parameters this approach has difficulties in estimating the confidence intervals and in testing the number of components we propose a bayesian approach to obtaining the credible intervals for the location and scale of the tolerances in each component and for the mixture proportions by using data augmentation and gibbs sampler we use bayes factor for model selection and determining the number of components we illustrate the method with data published in lwin and martin 1989

edgeworth expanded gaussian mixture density modeling instead of increasing the order of the edgeworth expansion of a single gaussian kernel we suggest using mixtures of edgeworth expanded gaussian kernels of moderate order we introduce a simple closed form solution for estimating the kernel parameters based on weighted moment matching furthermore we formulate the extension to the multivariate case which is not always feasible with algebraic density approximation procedures

adaptive mixture density estimation abstract a recursive nonparametric method is developed for performing density estimation derived from mixture models kernel estimation and stochastic approximation the asymptotic performance of the method dubbed adaptive mixtures priebe and marchette pattern recognition24 1197 1209 1991 for its data driven development of a mixture model approximation to the true density is investigated using the method of sieves simulations are included indicating convergence properties for some simple examples

numerical simulation for a duffing oscillator driven by colored noise using nonstandard difference scheme abstract the present paper is devoted to the numerical solution of the fokker planck fp equation associated to the duffing oscillator driven by colored noise we propose an improved discretization of the standard fp operator splitting method which renders the scheme unconditionally stable this scheme is successfully applied to explore the influence of the colored noise on the probability density

approximate probability distributions for stochastic systems maximum entropy method abstract the effective analytical methods for stochastic nonlinear dynamical systems are applicable only in some simple cases if one deals with more complex systems and with the so called real life applications the approximate methods and numerical integration are necessary in this paper we present the possible approaches to approximate characterization of the probability distributions of stochastic nonlinear systems starting from the description of the basic properties of such systems the most notable recent efforts to evaluation of their probability distributions are presented with emphasis on the maximum entropy method this method originated in its simple classical form in statistical physics when suitably generalized allows complicated stochastic systems to be treated successfully using information contained in the equations for statistical moments of the solution or response in this paper the general scheme of the method is presented both for stationary and nonstationary distributions and then its numerical implementation is expounded nonlinear stochastic oscillatory systems are treated in detail and the obtained probability distributions are shown graphically in comparison with the exact solutions and with the simulation results

dynamics of offshore structures part ii stochastic averaging analysis this is the second of two companion papers concerned with an investigation into the non linear dynamics of drag dominated offshore structures excited by waves the equations of motion are approximated by a linear system with a time varying damping term the excitation however is a non linear function of the wave particle kinematics to study the effect of the time varying damping the resonant response is here examined by means of a narrow band idealization a stochastic averaging of the governing equations is applied and the fokker planck equation for the response amplitude probability densities is thereby obtained from the stationary solution of this equation second order statistics are derived together with an expression for the hydrodynamic damping which is equivalent to the time varying term numerical results are given to illustrate the magnitude of this term for a typical framed structure and to provide comparisons with results obtained by the conventional technique of linearization

automatic learning of cost functions for graph edit distance abstract graph matching and graph edit distance have become important tools in structural pattern recognition the graph edit distance concept allows us to measure the structural similarity of attributed graphs in an error tolerant way the key idea is to model graph variations by structural distortion operations as one of its main constraints however the edit distance requires the adequate definition of edit cost functions which eventually determine which graphs are considered similar in the past these cost functions were usually defined in a manual fashion which is highly prone to errors the present paper proposes a method to automatically learn cost functions from a labeled sample set of graphs to this end we formulate the graph edit process in a stochastic context and perform a maximum likelihood parameter estimation of the distribution of edit operations the underlying distortion model is learned using an expectation maximization algorithm from this model we finally derive the desired cost functions in a series of experiments we demonstrate the learning effect of the proposed method and provide a performance comparison to other models

graph convergence for the h accretive operator in banach spaces with an application abstract in this paper a concept of graph convergence concerned with the h accretive operator is introduced in banach spaces and some equivalence theorems between of graph convergence and resolvent operator convergence for the h accretive operator sequence are proved as an application a perturbed algorithm for solving a class of variational inclusions involving the h accretive operator is constructed under some suitable conditions the existence of the solution for the variational inclusions and the convergence of iterative sequence generated by the perturbed algorithm are also given

maximum likelihood topographic map formation we introduce a new unsupervised learning algorithm for kernel based topographic map formation of heteroscedastic gaussian mixtures that allows for a unified account of distortion error vector quantization log likelihood and kullback leibler divergence

a numerical solver for high dimensional transient fokker planck equation in modeling polymeric fluids in this paper a tensor decomposition approach combined with chebyshev spectral differentiation is presented to solve the high dimensional transient fokker planck equations fpe arising in the simulation of polymeric fluids via multi bead spring mbs model generalizing the authors previous work on the stationary fpe the transient solution is obtained in a single candecomp parafac decomposition cpd form for all times via the alternating least squares algorithm this is accomplished by treating the temporal dimension in the same manner as all other spatial dimensions thereby decoupling it from them as a result the transient solution is obtained without resorting to expensive time stepping schemes a new relaxed approach for imposing the vanishing boundary conditions is proposed improving the quality of the approximation the asymptotic behavior of the temporal basis functions is studied the proposed solver scales very well with the dimensionality of the mbs model numerical results for systems up to 14 dimensional state space are successfully obtained on a regular personal computer and compared with the corresponding matrix riccati differential equation for linear models or monte carlo simulations for nonlinear models

dynamic programming for detecting tracking and matching deformable contours the problem of segmenting an image into separate regions and tracking them over time is one of the most significant problems in vision terzopoulos et al 1987 proposed an approach to detect the contour regions of complex shapes assuming a user selected initial contour not very far from the desired solution we propose to further explore the information provided by the user s selected points and apply an optimal method to detect contours which allows a segmentation of the image the method is based on dynamic programming dp and applies to a wide variety of shapes it is exact and not iterative we also consider a multiscale approach capable of speeding up the algorithm by a factor of 20 although at the expense of losing the guaranteed optimality characteristic the problem of tracking and matching these contours is addressed for tracking the final contour obtained at one frame is sampled and used as initial points for the next frame then the same dp process is applied for matching a novel strategy is proposed where the solution is a smooth displacement field in which unmatched regions are allowed while cross vectors are not the algorithm is again based on dp and the optimal solution is guaranteed we have demonstrated the algorithms on natural objects in a large spectrum of applications including interactive segmentation and automatic tracking of the regions of interest in medical images

numerical solution of high dimensional stationary fokker planck equations via tensor decomposition and chebyshev spectral differentiation abstract this paper focuses on the curse of dimensionality in the numerical solution of the stationary fokker planck equation for systems with state independent excitation a tensor decomposition approach is combined with chebyshev spectral differentiation to drastically reduce the number of degrees of freedom required to maintain accuracy as dimensionality increases following the enforcement of the normality condition via a penalty method the discretized system is solved using alternating least squares algorithm numerical results for a variety of systems including separable non separable systems linear nonlinear systems and systems with without closed form stationary solutions up to ten dimensional state space are presented to illustrate the effectiveness of the proposed method

asymptotic convergence rate of the em algorithm for gaussian mixtures it is well known that the convergence rate of the expectation maximization em algorithm can be faster than those of convention first order iterative algorithms when the overlap in the given mixture is small but this argument has not been mathematically proved yet this article studies this problem asymptotically in the setting of gaussian mixtures under the theoretical framework of xu and jordan 1996 it has been proved that the asymptotic convergence rate of the em algorithm for gaussian mixtures locally around the true solution is o e0 5 e where e 0 is an arbitrarily small number o x means that it is a higher order infinitesimal as x 0 and e is a measure of the average overlap of gaussians in the mixture in other words the large sample local convergence rate for the em algorithm tends to be asymptotically superlinear when e tends to zero

statistical approach to segmentation of single channel cerebral mr images a statistical model is presented that represents the distributions of major tissue classes in single channel magnetic resonance mr cerebral images using the model cerebral images are segmented into gray matter white matter and cerebrospinal fluid csf the model accounts for random noise magnetic field inhomogeneities and biological variations of the tissues intensity measurements are modeled by a finite gaussian mixture smoothness and piecewise contiguous nature of the tissue regions are modeled by a three dimensional 3 d markov random field mrf a segmentation algorithm based on the statistical model approximately finds the maximum a posteriori map estimation of the segmentation and estimates the model parameters from the image data the proposed scheme for segmentation is based on the iterative conditional modes icm algorithm in which measurement model parameters are estimated using local information at each site and the prior model parameters are estimated using the segmentation after each cycle of iterations application of the algorithm to a sample of clinical mr brain scans comparisons of the algorithm with other statistical methods and a validation study with a phantom are presented the algorithm constitutes a significant step toward a complete data driven unsupervised approach to segmentation of mr images in the presence of the random noise and intensity inhomogeneities

h monotone operator and resolvent operator technique for variational inclusions in this paper we introduce a new class of monotone operators h monotone operators the resolvent operator associated with an h monotone operator is defined and its lipschitz continuity is presented we also introduce and study a new class of variational inclusions involving h monotone operators and construct a new algorithm for solving this class of variational inclusions by using the resolvent operator technique

data mapping by probabilistic modular networks and information theoretic criteria the quantitative mapping of a database that represents a finite set of classified and or unclassified data points may be decomposed into three distinctive learning tasks 1 detection of the structure of each class model with locally mixture clusters 2 estimation of the data distributions for each induced cluster inside each class and 3 classification of the data into classes that realizes the data memberships the mapping function accomplished by the probabilistic modular networks may then be constructed as the optimal estimator with respect to information theory and each of the three tasks can be interpreted as an independent objective in real world applications we adapt a model fitting scheme that determines both the number and kernel of local clusters using information theoretic criteria the class distribution functions are then obtained by learning generalized gaussian mixtures where a soft classification of the data is performed by an efficient incremental algorithm further classification of the data is treated as a hard bayesian detection problem in particular the decision boundaries between the classes are fine tuned by a reinforce or antireinforce supervised learning scheme examples of the application of this framework to medical image quantification automated face recognition and featured database analysis are presented as well

application of the principle of detailed balance to the random vibration of non linear oscillators abstract the principle of detailed balance which was developed in the field of statistical physics relates to the microscopic reversibility of a stochastic process it has recently been suggested that this principle may be used to expedite the solution of the fokker planck equation which arises when the response of a non linear system to white noise excitation is considered to investigate this the most general externally excited non linear oscillator which satisfies detailed balance is derived and it is found that the result coincides with a class of system for which there is an existing solution to the stationary form of the fokker planck equation in deriving this result the stationary response statistics of a slightly more general class of oscillator which does not satisfy detailed balance are found and a number of results regarding the probability current of the response are obtained

convergence analysis of new over relaxed proximal point algorithm frameworks with errors and applications to general a monotone nonlinear inclusion forms abstract the purpose of this paper is to introduce and study a new class of over relaxed proximal point algorithm frameworks with errors based on general a monotonicity further by using alber s inequalities the definition of normalized duality mapping on the dual spaces of banach spaces and the new proximal mapping technique associated with the general a monotone operators we discuss the approximation solvability of general a monotone nonlinear inclusion forms in banach spaces and prove the convergence analysis of iterative sequences generated by the algorithm frameworks via applying the lipschitz continuity of m 1 that is the inverse of multi valued operator m and the lipschitz continuity of proximal mapping associated with the general a monotone operators respectively finally some applications are given to show that the results presented in this paper improve generalize and unify the corresponding results of recent works

beyond the markovian assumption response excitation probabilistic solution to random nonlinear differential equations in the long time uncertainty quantification for dynamical systems under non white excitation is a difficult problem encountered across many scientific and engineering disciplines difficulties originate from the la