accelerated iterative reconstruction for positron emission tomography based on the em algorithm for maximum likelihood estimation the em method that was originally developed for maximum likelihood estimation in the context of mathematical statistics may be applied to a stochastic model of positron emission tomography pet the result is an iterative algorithm for image reconstruction that is finding increasing use in pet due to its attractive theoretical and practical properties its major disadvantage is the large amount of computation that is often required due to the algorithm s slow rate of convergence this paper presents an accelerated form of the em algorithm for pet in which the changes to the image as calculated by the standard algorithm are multiplied at each iteration by an overrelaxation parameter the accelerated algorithm retains two of the important practical properties of the standard algorithm namely the selfnormalization and nonnegativity of the reconstructed images experimental results are presented using measured data obtained from a hexagonal detector system for pet the likelihood function and the norm of the data residual were monitored during the iterative process according to both of these measures the images reconstructed at iterations 7 and 11 of the accelerated algorithm are similar to those at iterations 15 and 30 of the standard algorithm for two different sets of data important theoretical properties remain to be investigated namely the convergence of the accelerated algorithm and its performance as a maximum likelihood estimator

inverse problem for multispecies ferromagneticlike mean field models in phase space with many states in this paper we solve the inverse problem for the curie weiss model and its multispecies version when multiple thermodynamic states are present as in the low temperature phase where the phase space is clustered the inverse problem consists of reconstructing the model parameters starting from configuration data generated according to the distribution of the model we demonstrate that without taking into account the presence of many states the application of the inversion procedure produces very poor inference results to overcome this problem we use the clustering algorithm when the system has two symmetric states of positive and negative magnetizations the parameter reconstruction can also be obtained with smaller computational effort simply by flipping the sign of the magnetizations from positive to negative or vice versa the parameter reconstruction fails when the system undergoes a phase transition in that case we give the correct inversion formulas for the curie weiss model and we show that they can be used to measure how close the system gets to being critical

growing scale free networks with tunable clustering we extend the standard scale free network model to include a triad formation step we analyze the geometric properties of networks generated by this algorithm both analytically and by numerical c

estimating rate constants in hidden markov models by the em algorithm the em algorithm e g the baum welch 1970 re estimation is an important tool for parameter estimation in discrete time hidden markov models we present a direct re estimation of rate constants for applications in which the underlying markov process is continuous in time previous estimation of discrete time transition probabilities is not necessary

ising distribution as a latent variable model during the past decades the ising distribution has attracted interest in many applied disciplines as the maximum entropy distribution associated to any set of correlated binary spin variables with observed means and covariances however numerically speaking the ising distribution is un practical so alternative models are often preferred to handle correlated binary data one popular alternative especially in life sciences is the cox distribution or the closely related dichotomized gaussian distribution and log normal cox point process where the spins are generated independently conditioned on the drawing of a latent variable with a multivariate normal distribution this article explores the conditions for a principled replacement of the ising distribution by a cox distribution it shows that the ising distribution itself can be treated as a latent variable model and it explores when this latent variable has a quasi normal distribution a variational approach to this question reveals a formal link with classic mean field methods especially opper and winther s adaptive tap approximation this link is confirmed by weak coupling plefka expansions of the different approximations and then by numerical tests overall this study suggests that an ising distribution can be replaced by a cox distribution in practical applications precisely when its parameters lie in the mean field domain

a time space finite element discretization technique for the calculation of the electromagnetic field in ferromagnetic materials the stability of time stepping methods for parabolic differential equations is mostly a critical issue furthermore solving such equations with a classical time stepping approach can be very expensive because many small time steps have to be taken if steep gradients occur in the solution even if these occur only in a narrow part of the space domain in this paper we present a discretization technique in which finite element approximations are used in time and space simultaneously for a relatively large time period called a time slab this technique may be repeatedly applied to obtain further parts of the solution in subsequent time intervals it will be shown that with the proposed method the solution can be computed cheaply even if it has steep gradients and that stability is automatically guaranteed for the solution of the non linear algebraic equations on each time slab fast iterative methods can be used

a disaggregate negative binomial regression procedure for count data analysis various research areas face the methodological problems presented by nonnegative integer count data drawn from heterogeneous populations we present a disaggregate negative binomial regression procedure for analysis of count data observed for a heterogeneous sample of cross sections possibly over some fixed time periods this procedure simultaneously pools or groups cross sections while estimating a separate negative binomial regression model for each group an e m algorithm is described within a maximum likelihood framework to estimate the group proportions the group specific regression coefficients and the degree of overdispersion in event rates within each derived group the proposed procedure is illustrated with count data entailing nonnegative integer counts of purchases events for a frequently bought consumer good

regression analysis of interval censored survival data with covariates using log linear models we considered the regression analysis of the event time data with left right or interval censored observations we extended life table techniques for censored survival data using log linear models to incorporate interval censored failures the em algorithm was used to calculate maximum likelihood estimates for the parameters we assumed that the hazard function was a stepwise function over disjoint intervals of time thus the nonparametric model the parametric exponential model and the semiparametric cox proportional hazard model were easily implemented as special cases we adapted the restricted em algorithm to test hypotheses and to construct confidence intervals for the parameters these methods were applied in an analysis of the recurrence time for treated melanoma patients

maximum likelihood variance components estimation for binary data abstract we consider a class of probit normal models for binary data and describe ml and reml estimation of variance components for that class as well as best prediction for the realized values of the random effects ml estimates are calculated using an em algorithm for complicated models em includes a gibbs step the computations are illustrated through two examples

a quick method for estimation of parameters of individual pulses in a multipath signal abstract consider a multipath signal whose individual signals are deterministic known increasing or decreasing pulses the problem is to estimate the amplitudes and delay times of individual signals several methods have been devoted to the solution of these unknown parameters the maximum likelihood ml estimation and the estimate maximize em algorithm are commonly used but they are computationally intensive and still insufficient to obtain accurate estimations the method can provide a quick and accurate estimate of the amplitudes and arrival delay times even in the closely spaced multipaths and heavy noise

em and beyond the basic theme of the em algorithm to repeatedly use complete data methods to solve incomplete data problems is also a theme of several more recent statistical techniques these techniques multiple imputation data augmentation stochastic relaxation and sampling importance resampling combine simulation techniques with complete data methods to attack problems that are difficult or impossible for em

maximum likelihood estimation and likelihood ratio test for square tables with missing data missing data complicate the analysis of paired categorical data this paper considers with the use of the em algorithm the maximum likelihood estimation and likelihood ratio test for incomplete square tables with missing data an example involving the susceptibility of micro organisms to antimicrobial drugs illustrates the procedure

an incomplete data approach to the analysis of covariance structures in this paper linear structural equation models with latent variables are considered it is shown how many common models arise from incomplete observation of a relatively simple system subclasses of models with conditional independence interpretations are also discussed using an incomplete data point of view the relationships between the incomplete and complete data likelihoods assuming normality are highlighted for computing maximum likelihood estimates the em algorithm and alternatives are surveyed for the alternative algorithms simplified expressions for computing function values and derivatives are given likelihood ratio tests based on complete and incomplete data are related and an example on using their relationship to improve the fit of a model is given

small world graphs by iterated local edge formation we study graphs obtained by successive creation and destruction of edges into small neighborhoods of the vertices starting with a circle graph of large diameter we obtain small world graphs with logarithmic diameter high clustering coefficients and a fat tail distribution for the degree only local edge formation processes are involved and no preferential attachment was used furthermore we found an interesting phase transition with respect to the initial conditions

network inference using asynchronously updated kinetic ising model network structures are reconstructed from dynamical data by respectively naive mean field nmf and thouless anderson palmer tap approximations tap approximation adds simple corrections to the nmf approximation taking into account the effect of the focused spin on itself via its influence on other neighboring spins for tap approximation we use two methods to reconstruct the network a iterative method b casting the inference formula to a set of cubic equations and solving it directly we investigate inference of the asymmetric sherrington kirkpatrick as k model using asynchronous update the solutions of the set of cubic equations depend on temperature t in the as k model and a critical temperature t c 2 1 is found the two methods for tap approximation produce the same results when the iterative method is convergent compared to nmf tap is somewhat better at low temperatures but approaches the same performance as temperature increases both nmf and tap approximation reconstruct better for longer data length l but for the degree of improvement tap performs better than nmf

signal modeling with filtered discrete fractional noise processes filtered versions of fractionally differenced gaussian noise fdgn processes are studied fractionally differenced gaussian noise is a discrete time equivalent of fractional brownian motion filtered versions of such processes are ideally suited for modeling signals with different short term and long term correlation structure two iterative algorithms for estimating the parameters of filtered fdgn processes are described the first technique is based on the expectation maximization algorithm it converges to a stationary point of the log likelihood function corresponding to the parameters of the model the second technique is a computationally efficient approximate approach it is found to converge experimentally but no proof of its convergence is given the usefulness of filtered fdgn models and the performance of the proposed iterative algorithms are illustrated by fitting filtered fdgn models to speech waveforms and other data corresponding to natural phenomena

a generalized conjugate gradient least square method a generalizeds term truncated conjugate gradient method of least square type proposed in 1a b is extended to a form more suitable for proving when the truncated version is identical to the full term version advantages with keeping a control term in the truncated version is pointed out a computationally efficient new algorithm based on a special inner product with a small demand of storage is also presented we also give simplified and slightly extended proofs of termination of the iterative sequence and of existence of ans term recursion identical to the full term version important earlier results on this latter topic are found in 15 16 8 and 11

correspondence between temporal correlations in time series inverse problems and the spherical model in this paper we employ methods from statistical mechanics to model temporal correlations in time series we put forward a methodology based on the maximum entropy principle to generate ensembles of time series constrained to preserve part of the temporal structure of an empirical time series of interest we show that a constraint on the lag one autocorrelation can be fully handled analytically and corresponds to the well known spherical model of a ferromagnet we then extend such a model to include constraints on more complex temporal correlations by means of perturbation theory showing that this leads to substantial improvements in capturing the lag one autocorrelation in the variance we apply our approach on synthetic data and illustrate how it can be used to formulate expectations on the future values of a data generating process

memory free dynamics for the thouless anderson palmer equations of ising models with arbitrary rotation invariant ensembles of random coupling matrices we propose an iterative algorithm for solving the thouless anderson palmer equations of ising models with arbitrary rotation invariant random coupling matrices in the thermodynamic limit we prove by means of the dynamical functional method that the proposed algorithm converges when the so called de almeida thouless criterion is fulfilled moreover we give exact analytical expressions for the rate of the convergence

a canonical space time state space model state and parameter estimation the maximum likelihood estimation of a dynamic spatiotemporal model is introduced centred around the inclusion of a prior arbitrary spatiotemporal neighborhood description the neighborhood description defines a specific parameterization of the state transition matrix chosen on the basis of prior knowledge about the system the model used is inspired by the spatiotemporal arma starma model but the representation used is based on the standard state space model the inclusion of the neighborhood into an expectation maximization based joint state and parameter estimation algorithm allows for accurate characterization of the spatiotemporal model the process of including the neighborhood and the effect it has on the maximum likelihood parameter estimate is described and demonstrated in this paper

boundary value techniques for initial value problems in ordinary differential equations the numerical solution of initial value problems in ordinary differential equations by means of boundary value techniques is considered we discuss a finite difference method which was already investigated by fox in 1954 and fox and mitchell in 1957 hereby we concentrate on explaining the fundamentals of the method because for initial value problems the boundary value method seems to be fairly unknown we further propose and discuss new jalerkin methods for initial value problems along the lines of the boundary value approach

growing network with local rules preferential attachment clustering hierarchy and degree correlations the linear preferential attachment hypothesis has been shown to be quite successful in explaining the existence of networks with power law degree distributions it is then quite important to determine if this mechanism is the consequence of a general principle based on local rules in this work it is claimed that an effective linear preferential attachment is the natural outcome of growing network models based on local rules it is also shown that the local models offer an explanation for other properties like the clustering hierarchy and degree correlations recently observed in complex networks these conclusions are based on both analytical and numerical results for different local rules including some models already proposed in the literature

numerical null controllability of the 1d linear schrodinger equation abstract this paper deals with the numerical approximation to boundary controls that drive the solution to the 1 d linear schrodinger equation to a prescribed state at a final time using ideas from fursikov and imanuvilov we consider the control that minimizes over the class of admissible controls a functional that involves weighted integrals with weights that blow up at t we will see that this extremal problem is equivalent to a differential problem that is fourth order in space and second order in time adapting some numerical techniques applied by the first author and munch to the heat equation we approximate the variational formulation by introducing appropriate space time finite elements that are c 1 in space and c 0 in time we present two approaches the second one relies on a change of variable which leads to a lower condition number for the stiffness matrix the results of some experiments show the efficiency of these methods

how to expand around mean field theory using high temperature expansions high temperature expansions performed at a fixed order parameter provide a simple and systematic way to derive and correct mean field theories for statistical mechanical models for models like spin glasses which have general couplings between spins the authors show that these expansions generate the thouless anderson palmer equations at low order they explicitly calculate the corrections to tap theory for these models for ferromagnetic models they show that their expansions can easily be converted into 1 d expansions around mean field theory where d is the number of spatial dimensions only a small finite number of graphs need to be calculated to generate each order in 1 d for thermodynamic quantities like free energy or magnetization unlike previous 1 d expansions the expansions are valid in the low temperature phases of the models considered they consider alternative ways to expand around mean field theory besides 1 d expansions in contrast to the 1 d expansion for the critical temperature which is presumably asymptotic these schemes can be used to devise convergent expansions for the critical temperature they also appear to give convergent series for thermodynamic quantities and critical exponents they test the schemes using the spherical model where their properties can be studied using exact expressions

an analysis of the discontinuous galerkin method for a scalar hyperbolic equation we prove lp stability and error estimates for the discontinuous galerkin method when applied to a scalar linear hyperbolic equation on a convex polygonal plane domain using finite element analysis techniques we obtain l2 estimates that are valid on an arbitrary locally regular triangulation of the domain and for an arbitrary degree of polynomials lp estimates for p 2 are restricted to either a uniform or piecewise uniform triangulation and to polynomials of not higher than first degree the latter estimates are proved by combining finite difference and finite element analysis techniques

evolving networks with distance preferences we study evolving networks where new nodes when attached to the network form links with other nodes of preferred distances a particular case is where always the shortest distances are selected make friends with the friends of your present friends we present simulation results for network parameters like the first eigenvalue of the graph laplacian synchronizability clustering coefficients average distances and degree distributions for different distance preferences and compare them with the parameter values for random and scale free networks we find that for the shortest distance rule we obtain a power law degree distribution as in scale free networks while the other parameters are significantly different especially the clustering coefficient

ising model for neural data model quality and approximate methods for extracting functional connectivity we study pairwise ising models for describing the statistics of multineuron spike trains using data from a simulated cortical network we explore efficient ways of finding the optimal couplings in

inverse problem robustness for multi species mean field spin models the inverse problem method is tested for a class of mean field statistical mechanics models representing a mixture of particles of different species the robustness of the inversion is investigated for different values of the physical parameters systemsizesandindependentsamples weshowhowtoreconstruct the parameter values with a precision of a few per cent

publication bias and meta analysis for 2x2 tables an average markov chain monte carlo em algorithm summary a major difficulty in meta analysis is publication bias studies with positive outcomes are more likely to be published than studies reporting negative or inconclusive results correcting for this bias is not possible without making untestable assumptions in this paper a sensitivity analysis is discussed for the meta analysis of 2 2 tables using exact conditional distributions a markov chain monte carlo em algorithm is used to calculate maximum likelihood estimates a rule for increasing the accuracy of estimation and automating the choice of the number of iterations is suggested

orthogonalizing em a design based least squares algorithm we introduce an efficient iterative algorithm intended for various least squares problems based on a design of experiments perspective the algorithm called orthogonalizing em oem works for ordinary least squares ols and can be easily extended to penalized least squares the main idea of the procedure is to orthogonalize a design matrix by adding new rows and then solve the original problem by embedding the augmented design in a missing data framework we establish several attractive theoretical properties concerning oem for the ols with a singular regression matrix an oem sequence converges to the moore penrose generalized inverse based least squares estimator for ordinary and penalized least squares with various penalties it converges to a point having grouping coherence for fully aliased regression matrices convergence and the convergence rate of the algorithm are examined finally we demonstrate that oem is highly efficient for large scale least squares and penalized least squares proble

variable selection in penalized model based clustering via regularization on grouped parameters penalized model based clustering has been proposed for high dimensional but small sample sized data such as arising from genomic studies in particular it can be used for variable selection a new regularization scheme is proposed to group together multiple parameters of the same variable across clusters which is shown both analytically and numerically to be more effective than the conventional l 1 penalty for variable selection in addition we develop a strategy to combine this grouping scheme with grouping structured variables simulation studies and applications to microarray gene expression data for cancer subtype discovery demonstrate the advantage of the new proposal over several existing approaches

nonparametric estimation of the distribution of time to onset for specific diseases in survival sacrifice experiments this paper concerns the analysis of an animal survival sacrifice experiment designed to investigate the incidence of a particular disease of interest the disease is assumed to be irreversible and detectable only at death for example by a necropsy each observation can be of one of three types i death caused by the disease ii death from a competing cause such as sacrifice with the disease present or iii death with the disease absent a two dimensional em algorithm is proposed for the nonparametric maximum likelihood estimation of the distributions of the time to onset and of the time to death from the disease these can be compared with nonparametric estimators recently proposed by kodell shaw and johnson 1982 biometrics 38 43 58 and by dinse and lagakos 1982 biometrics 38 921 932 a slight modification of the algorithm permits the construction of likelihood based interval estimates of quantiles of the distributions some extensions and generalizations are indicated

em algorithms for ml factor analysis the details of em algorithms for maximum likelihood factor analysis are presented for both the exploratory and confirmatory models the algorithm is essentially the same for both cases and involves only simple least squares regression operations the largest matrix inversion required is for aq q symmetric matrix whereq is the matrix of factors the example that is used demonstrates that the likelihood for the factor analysis model may have multiple modes that are not simply rotations of each other such behavior should concern users of maximum likelihood factor analysis and certainly should cast doubt on the general utility of second derivatives of the log likelihood as measures of precision of estimation

from regular to growing small world networks abstract we propose a growing model which interpolates between one dimensional regular lattice and small world networks the model undergoes an interesting phase transition from large to small worlds we investigate the structural properties by both theoretical predictions and numerical simulations our growing model is a complementarity for the important static watts strogatz network model

on the convergence properties of the em algorithm two convergence aspects of the em algorithm are studied i does the em algorithm find a local maximum or a stationary value of the incomplete data likelihood function ii does the sequence of parameter estimates generated by em converge several convergence results are obtained under conditions that are applicable to many practical situations two useful special cases are a if the unobserved complete data specification can be described by a curved exponential family with compact parameter space all the limit points of any em sequence are stationary points of the likelihood function b if the likelihood function is unimodal and a certain differentiability condition is satisfied then any em sequence converges to the unique maximum likelihood estimate a list of key properties of the algorithm is included

using persistent homology to reveal hidden covariates in systems governed by the kinetic ising model we propose a method based on persistent homology to uncover topological properties of a priori unknown covariates in a system governed by the kinetic ising model with time varying external fields as its starting point the method takes observations of the system under study a list of suspected or known covariates and observations of those covariates we infer away the contributions of the suspected or known covariates after which persistent homology reveals topological information about unknown remaining covariates our motivating example system is the activity of neurons tuned to the covariates physical position and head direction but the method is far more general

small correlation expansions for the inverse ising problem we present a systematic small correlation expansion to solve the inverse ising problem and find a set of couplings and fields corresponding to a given set of correlations and magnetizations couplings are calculated up to the third order in the correlations for generic magnetizations and to the seventh order in the case of zero magnetizations in addition we show how to sum some useful classes of diagrams exactly the resulting expansion outperforms existing algorithms on the sherrington kirkpatrick spin glass model

sparse kalman filtering approaches to realized covariance estimation from high frequency financial data estimation of the covariance matrix of asset returns from high frequency data is complicated by asynchronous returns market microstructure noise and jumps one technique for addressing both asynchronous returns and market microstructure is the kalman expectation maximization kem algorithm however the kem approach assumes log normal prices and does not address jumps in the return process which can corrupt estimation of the covariance matrix in this paper we extend the kem algorithm to price models that include jumps we propose a sparse kalman filtering approach to this problem in particular we develop a kalman expectation conditional maximization algorithm to determine the unknown covariance as well as detecting the jumps in order to promote a sparse estimate of the jumps we consider both laplace and the spike and slab jump priors numerical results using simulated data show that each of these approaches provide for improved covariance estimation relative to the kem method in a variety of settings where jumps occur

an em algorithm for fitting two level structural equation models maximum likelihood is an important approach to analysis of two level structural equation models different algorithms for this purpose have been available in the literature in this paper we present a new formulation of two level structural equation models and develop an em algorithm for fitting this formulation this new formulation covers a variety of two level structural equation models as a result the proposed em algorithm is widely applicable in practice a practical example illustrates the performance of the em algorithm and the maximum likelihood statistic

convergence condition of the tap equation for the infinite ranged ising spin glass model it is shown that the power expansion of the gibbs potential of the sk model up to second order in the exchange couplings leads to the tap equation this result remains valid for the general including a ferromagnetic exchange sk model theorems of power expansions and resolvent techniques are employed to solve the convergence problem the convergence condition is presented for the whole temperature range and for general distributions of the local magnetisations

maximum likelihood estimation under a successive sampling discovery model a common problem encountered in the analysis of discovery data is the size bias phenomenon in which the larger units tend to be discovered first one approach to account for this bias is to model the discovery process as sampling successively from a finite population without replacement and with probability proportional to size we consider in this article a generalized version of this model for analyzing multivariate data with any given measure of size we assume a superpopulation framework and develop procedures for maximum likelihood estimation of the parameters of the distribution the use of the em algorithm for computing the maximum likelihood estimates associated computational issues and relationships to regression estimators in survey sampling are discussed oil discovery data from the rimbey meadowbrook reef play are used to illustrate the techniques

applying the multivariate time rescaling theorem to neural population models statistical models of neural activity are integral to modern neuroscience recently interest has grown in modeling the spiking activity of populations of simultaneously recorded neurons to study the effects of correlations and functional connectivity on neural information processing however any statistical model must be validated by an appropriate goodness of fit test kolmogorov smirnov tests based on the time rescaling theorem have proven to be useful for evaluating point process based statistical models of single neuron spike trains here we discuss the extension of the time rescaling theorem to the multivariate neural population case we show that even in the presence of strong correlations between spike trains models that neglect couplings between neurons can be erroneously passed by the univariate time rescaling test we present the multivariate version of the time rescaling theorem and provide a practical step by step procedure for applying it to testing the sufficiency of neural population models using several simple analytically tractable models and more complex simulated and real data sets we demonstrate that important features of the population activity can be detected only using the multivariate extension of the test

consistency and asymptotic normality of the maximum likelihood estimator in generalized linear models generalized linear models are used for regression analysis in a number of cases including categorical responses where the classical assumptions are violated the statistical analysis of such models is based on the asymptotic properties of the maximum likelihood estimator we present mild general conditions which respectively assure weak or strong consistency or asymptotic normality most of the previous work has been concerned with natural link functions in this case our normality condition though obtained by a different approach is closely related to a condition of haberman 1977a examples show how the general conditions reduce to weak requirements for special exponential families further for regressors with a compact range sufficient conditions are given which do not involve the unknown parameter and are therefore easy to check in practice responses with a bounded range e g categorical responses and stochastic regressors also are treated

chaotic balanced state in a model of cortical circuits the nature and origin of the temporal irregularity in the electrical activity of cortical neurons in vivo are not well understood we consider the hypothesis that this irregularity is due to a balance of excitatory and inhibitory currents into the cortical cells we study a network model with excitatory and inhibitory populations of simple binary units the internal feedback is mediated by relatively large synaptic strengths so that the magnitude of the total excitatory and inhibitory feedback is much larger than the neuronal threshold the connectivity is random and sparse the mean number of connections per unit is large though small compared to the total number of cells in the network the network also receives a large temporally regular input from external sources we present an analytical solution of the mean field theory of this model which is exact in the limit of large network size this theory reveals a new cooperative stationary state of large networks which we term a balanced state in thi

ensemble inhibition and excitation in the human cortex an ising model analysis with uncertainties the pairwise maximum entropy model also known as the ising model has been widely used to analyze the collective activity of neurons however controversy persists in the literature about seemingly inconsistent findings whose significance is unclear due to lack of reliable error estimates we therefore develop a method for accurately estimating parameter uncertainty based on random walks in parameter space using adaptive markov chain monte carlo after the convergence of the main optimization algorithm we apply our method to the activity patterns of excitatory and inhibitory neurons recorded with multielectrode arrays in the human temporal cortex during the wake sleep cycle our analysis shows that the ising model captures neuronal collective behavior much better than the independent model during wakefulness light sleep and deep sleep when both excitatory e and inhibitory i neurons are modeled ignoring the inhibitory effects of i neurons dramatically overestimates synchrony among e neurons furthermore information theoretic measures reveal that the ising model explains about 80 95 of the correlations depending on sleep state and neuron type thermodynamic measures show signatures of criticality although we take this with a grain of salt as it may be merely a reflection of long range neural correlations

granger causality and the inverse ising problem abstract the inference of the couplings of an ising model with given means and correlations is called the inverse ising problem this approach has received a lot of attention as a tool to analyze neural data we show that autoregressive methods may be used to learn the couplings of an ising model also in the case of asymmetric connections and for multispin interactions we find that for each link the linear granger causality is two times the corresponding transfer entropy i e the information flow on that link in the weak coupling limit for sparse connections and a low number of samples the l 1 regularized least squares method is used to detect the interacting pairs of spins nonlinear granger causality is related to multispin interactions

expansion of the effective action around non gaussian theories this paper derives the feynman rules for the diagrammatic perturbation expansion of the effective action around an arbitrary solvable problem the perturbation expansion around a gaussian theory is well known and composed of one line irreducible diagrams only for the expansions around an arbitrary non gaussian problem we show that a more general class of irreducible diagrams remains in addition to a second set of diagrams that has no analogue in the gaussian case the effective action is central to field theory in particular to the study of phase transitions symmetry breaking effective equations of motion and renormalization we exemplify the method on the ising model where the effective action amounts to the gibbs free energy recovering the thouless anderson palmer mean field theory in a fully diagrammatic derivation higher order corrections follow with only minimal effort compared to existing techniques our results show further that the plefka expansion and the high temperature expansion are special cases of the general formalism presented here

image classification by a two dimensional hidden markov model for block based classification an image is divided into blocks and a feature vector is formed for each block by grouping statistics extracted from the block conventional block based classification algorithms decide the class of a block by examining only the feature vector of this block and ignoring context information in order to improve classification by context an algorithm is proposed that models images by two dimensional 2 d hidden markov models hmms the hmm considers feature vectors statistically dependent through an underlying state process assumed to be a markov mesh which has transition probabilities conditioned on the states of neighboring blocks from both horizontal and vertical directions thus the dependency in two dimensions is reflected simultaneously the hmm parameters are estimated by the em algorithm to classify an image the classes with maximum a posteriori probability are searched jointly for all the blocks applications of the hmm algorithm to document and aerial image segmentation show that the algorithm outperforms cart sup tm lvq and bayes vq

em based joint data detection and channel estimation of ds cdma signals we present two efficient iterative receiver structures of tractable complexity for joint multiuser detection and multichannel estimation jde of direct sequence code division multiple access signals the schemes result from an application of the expectation maximization em and the space alternating generalized expectation maximization sage algorithms respectively the em jde receiver updates the data bit sequences in parallel while the sage jde receiver reestimates them successively the channel parameters are updated in parallel in both schemes the em algorithm provides a set of free parameters called weight coefficients which can be selected to optimize its performance two optimality criteria are defined and analytical expressions for the corresponding optimized weight coefficients are given monte carlo simulations of a synchronous scenario show that the proposed jde receivers have excellent multiuser efficiency and are robust against errors in the estimation of the channel parameters moreover very short training sequences are required for the jde schemes to converge simulation results further demonstrate that the sage jde receiver exhibits a better performance when the users bit sequences are updated in the order of increasing signal strength i e the bit sequence of the user with the weakest signal strength is updated first at each stage

efficient learning in boltzmann machines using linear response theory the learning process in boltzmann machines is computationally very expensive the computational complexity of the exact algorithm is exponential in the number of neurons we present a new approximate learning algorithm for boltzmann machines based on mean field theory and the linear response theorem the computational complexity of the algorithm is cubic in the number of neurons in the absence of hidden units we show how the weights can be directly computed from the fixed point equation of the learning rules thus in this case we do not need to use a gradient descent procedure for the learning process we show that the solutions of this method are close to the optimal solutions and give a significant improvement when correlations play a significant role finally we apply the method to a pattern completion task and show good performance for networks up to 100 neurons

robust continuous time smoothers without two sided stochastic integrals we consider the problem of fixed interval smoothing of a continuous time partially observed nonlinear stochastic dynamical system existing results for such smoothers require the use of two sided stochastic calculus the main contribution of the paper is to present a robust formulation of the smoothing equations under this robust formulation the smoothing equations are nonstochastic parabolic partial differential equations with random coefficients and hence the technical machinery associated with two sided stochastic calculus is not required furthermore the robust smoothed state estimates are locally lipschitz in the observations which is useful for numerical simulation as examples finite dimensional robust versions of the benes and hidden markov model smoothers and smoothers for piecewise linear dynamics are derived these finite dimensional smoothers do not involve stochastic integrals

inference of kinetic ising model on sparse graphs based on dynamical cavity method we propose an approach to the inference of kinetic ising model which asks to reconstruct couplings and external fields from given time dependent output of original system our approach gives an exact result on tree graphs and a good approximation on sparse graphs it can be seen as an extension of belief propagation inference of static ising model to kinetic ising model while existing mean field methods to the kinetic ising inference e g naive mean field tap equation and simply mean field use approximations which calculate magnetizations and correlations at time t from statistics of data at time t 1 dynamical cavity method can use statistics of data at times earlier than t 1 to capture more correlations at different time steps extensive numerical experiments show that our inference method is superior to existing mean field approaches on diluted networks

mean field theory of boltzmann machine learning i present a mean field theory for boltzmann machine learning derived by employing thouless anderson palmer free energy formalism to a full extent using the plefka expansion an extended theory that takes higher order correction to mean field free energy formalism into consideration is presented from which the mean field approximation of general orders along with the linear response correction are derived by truncating the plefka expansion up to desired orders a theoretical foundation for an effective trick of using diagonal weights introduced by kappen and rodr i guez is also given because of the finite system size and a lack of scaling assumptions on interaction coefficients the truncated free energy formalism cannot provide an exact description in the case of boltzmann machines accuracies of mean field approximations of several orders are compared by computer simulations

asymptotic convergence rate of the em algorithm for gaussian mixtures it is well known that the convergence rate of the expectation maximization em algorithm can be faster than those of convention first order iterative algorithms when the overlap in the given mixture is small but this argument has not been mathematically proved yet this article studies this problem asymptotically in the setting of gaussian mixtures under the theoretical framework of xu and jordan 1996 it has been proved that the asymptotic convergence rate of the em algorithm for gaussian mixtures locally around the true solution is o e0 5 e where e 0 is an arbitrarily small number o x means that it is a higher order infinitesimal as x 0 and e is a measure of the average overlap of gaussians in the mixture in other words the large sample local convergence rate for the em algorithm tends to be asymptotically superlinear when e tends to zero

an expert system for predicting nonlinear aeroelastic behavior of an airfoil in this paper we present an expert system to perform steady state response predictions we consider an aeroelastic model simulating a two degree of freedom airfoil oscillating in pitch and plunge with a freeplay nonlinearity in the pitch degree of freedom in the proposed data driven methodology a freeplay is first confirmed and then the locations of the switching points are determined a state space formulation is constructed to model the piece wise linear system the parameters of the system are estimated using the kalman filter and the expectation maximization algorithm the attractive feature of the present approach is its ability to accurately predict the steady state behavior of the nonlinear aeroelastic system with freeplay using only a limited amount of transient input data to demonstrate the effectiveness of the proposed methodology we present applications to freeplay aeroelastic data arising from wind tunnel experiments and numerical simulations

combining monte carlo and mean field like methods for inference in hidden markov random fields issues involving missing data are typical settings where exact inference is not tractable as soon as nontrivial interactions occur between the missing variables approximations are required and most of them are based either on simulation methods or on deterministic variational methods while variational methods provide fast and reasonable approximate estimates in many scenarios simulation methods offer more consideration of important theoretical issues such as accuracy of the approximation and convergence of the algorithms but at a much higher computational cost in this work we propose a new class of algorithms that combine the main features and advantages of both simulation and deterministic methods and consider applications to inference in hidden markov random fields hmrfs these algorithms can be viewed as stochastic perturbations of variational expectation maximization vem algorithms which are not tractable for hmrf we focus more specifically on one of these perturbations and we prove their almost sure convergence to the same limit set as the limit set of vem in addition experiments on synthetic and real world images show that the algorithm performance is very close and sometimes better than that of other existing simulation based and variational em like algorithms

on the updating and assembly of the hessian matrix in finite element methods abstract for many nonlinear differential equation solvers the assembly of the hessian matrix is an expensive task hence numerical methods like the damped inexact newton method see 3 11 are often too expensive to use as the hessian matrix has to be updated many times the goal of this paper is to show that in some special but frequently occurring cases where the hessian has a speical structure updating it is not more expensive than updating the gradient vector if a particular factorization of the stiffness matrix is used in these cases most of the computing time spent in the nonlinear solver will be used to solve linear systems of equations and the assembly of the hessian usually becomes a minor task the present paper extends earlier results of this nature in 4 6

high dimensional inference with the generalized hopfield model principal component analysis and corrections we consider the problem of inferring the interactions between a set of n binary variables from the knowledge of their frequencies and pairwise correlations the inference framework is based on the hopfield model a special case of the ising model where the interaction matrix is defined through a set of patterns in the variable space and is of rank much smaller than n we show that maximum likelihood inference is deeply related to principal component analysis when the amplitude of the pattern components ensuremath xi is negligible compared to sqrt n using techniques from statistical mechanics we calculate the corrections to the patterns to the first order in ensuremath xi sqrt n we stress the need to generalize the hopfield model and include both attractive and repulsive patterns in order to correctly infer networks with sparse and strong interactions we present a simple geometrical criterion to decide how many attractive and repulsive patterns should be considered as a function of the sampling noise we moreover discuss how many sampled configurations are required for a good inference as a function of the system size n and of the amplitude ensuremath xi the inference approach is illustrated on synthetic and biological data

market structure explained by pairwise interactions financial markets are a typical example of complex systems where interactions between constituents lead to many remarkable features here we give empirical evidence by making as few assumptions as possible that the market microstructure capturing almost all of the available information in the data of stock markets does not involve higher order than pairwise interactions we give an economic interpretation of this pairwise model we show that it accurately recovers the empirical correlation coefficients thus the collective behaviors are quantitatively described by models that capture the observed pairwise correlations but no higher order interactions furthermore we show that an order disorder transition occurs as predicted by the pairwise model last we make the link with the graph theoretic description of stock markets recovering the non random and scale free topology shrinking length during crashes and meaningful clustering features as expected

diversity combining for ds ss systems with time varying correlated fading branches diversity combining of multiple time varying and correlated fading branches is investigated for direct sequence spread spectrum systems the correlated branches are modeled and estimated jointly as a vector autoregressive var process the joint estimation is shown to provide performance gain over separate estimation of the fading branches the parameter matrices of the var model are estimated via the method of expectation maximization em with two algorithms the first algorithm using results from kalman smoothing provides a closed form solution to the maximization problem in the iterative em procedure however the iterative em kalman algorithm operates repeatedly on a batch of training data and results in large storage requirements and long processing delays to overcome these disadvantages a new algorithm with only forward time recursions is proposed that approximates the iterative em solution and efficiently adapts to slowly changing doppler spreads as a result the new algorithm significantly reduces memory and training sequence requirements through computer simulations a near ideal bit error rate performance is found for both algorithms and the efficacy of the new adaptive algorithm for channels with changing doppler spreads is demonstrated