mean field message passing equations in the hopfield model and its generalizations motivated by recent progress in using restricted boltzmann machines as preprocessing algorithms for deep neural network we revisit the mean field equations belief propagation and thouless anderson palmer tap equations in the best understood of such machines namely the hopfield model of neural networks and we explicit how they can be used as iterative message passing algorithms providing a fast method to compute the local polarizations of neurons in the retrieval phase where neurons polarize in the direction of one memorized pattern we point out a major difference between the belief propagation and tap equations the set of belief propagation equations depends on the pattern which is retrieved while one can use a unique set of tap equations this makes the latter method much better suited for applications in the learning process of restricted boltzmann machines in the case where the patterns memorized in the hopfield model are not independent but are correlated through a combinatorial structure we show that the tap equations have to be modified this modification can be seen either as an alteration of the reaction term in tap equations or more interestingly as the consequence of message passing on a graphical model with several hidden layers where the number of hidden layers depends on the depth of the correlations in the memorized patterns this layered structure is actually necessary when one deals with more general restricted boltzmann machines

expectation consistent approximate inference we propose a novel framework for approximations to intractable probabilistic models which is based on a free energy formulation the approximation can be understood as replacing an average over the original intractable distribution with a tractable one it requires two tractable probability distributions which are made consistent on a set of moments and encode different features of the original intractable distribution in this way we are able to use gaussian approximations for models with discrete or bounded variables which allow us to include non trivial correlations these are neglected in many other methods we test the framework on toy benchmark problems for binary variables on fully connected graphs and 2d grids and compare with other methods such as loopy belief propagation good performance is already achieved by using single nodes as tractable substructures significant improvements are obtained when a spanning tree is used instead

neural networks for optimization problems with inequality constraints the knapsack problem a strategy for finding approximate solutions to discrete optimization problems with inequality constraints using mean field neural networks is presented the constraints x d 0 are encoded by x x terms in the energy function a careful treatment of the mean field approximation for the self coupling parts of the energy is crucial and results in an essentially parameter free algorithm this methodology is extensively tested on the knapsack problem of size up to 103 items the algorithm scales like nm for problems with n items and m constraints comparisons are made with an exact branch and bound algorithm when this is computationally possible n d 30 the quality of the neural network solutions consistently lies above 95 of the optimal ones at a significantly lower cpu expense for the larger problem sizes the algorithm is compared with simulated annealing and a modified linear programming approach for nonhomogeneous problems these produce good solutions whereas for the more difficult homogeneous problems the neural approach is a winner with respect to solution quality and or cpu time consumption the approach is of course also applicable to other problems of similar structure like set covering

dsc time domain solution of maxwell s equations a new computational algorithm the discrete singular convolution dsc is introduced for solving scattering and guided wave problems described by time domain maxwell s equations the dsc algorithm is utilized for the spatial discretization and the fourth order runge kutta scheme is used for the time advancing staggered meshes are used for electromagnetic fields four standard test problems a hollow air filled waveguide a dielectric slab loaded rectangular waveguide a shield microstrip line and a dielectric square are employed to illustrate the usefulness to test the accuracy and to explore the limitation of the dsc algorithm results are compared with those of finite difference scaling function multi resolution time domain and finite element based high frequency structure simulator numerical experiments indicate that the present algorithm is a promising approach for achieving high accuracy in electromagnetic wave computations

staggered time integrators for wave equations we consider variations of the adams bashforth backward differentiation and runge kutta families of time integrators to solve systems of linear wave equations on uniform time staggered grids these methods are found to have smaller local truncation errors and to allow larger stable time steps than traditional nonstaggered versions of equivalent orders we investigate the accuracy and stability of these methods analytically experimentally and through the use of a novel root portrait technique

numerical treatment of two dimensional interfaces for acoustic and elastic waves we present a numerical method to take into account 2d arbitrary shaped interfaces in classical finite difference schemes on a uniform cartesian grid this work extends the explicit simplified interface method esim previously proposed in 1d j comput phys 168 2001 227 248 the physical problem under study concerns the linear hyperbolic systems of acoustics and elastodynamics with stationary interfaces our method maintains near the interfaces properties of the schemes in homogeneous medium such as the order of accuracy and the stability limit moreover it enforces the numerical solution to satisfy the exact interface conditions lastly it provides subcell geometrical features of the interface inside the meshing the esim can be coupled automatically with a wide class of numerical schemes lax wendroff flux limiter schemes etc for a negligible additional computational cost throughout the paper we focus on the challenging case of an interface between a fluid and an elastic solid in numerical experiments we provide comparisons between numerical solutions and original analytic solutions showing the efficiency of the method

conservative nonlinear difference scheme for the cahn hilliard equation ii abstract numerical solutions for the cahn hilliard equation is considered using the crank nicolson type finite difference method existence of the solution for the difference scheme has been shown by brouwer fixed point theorem stability convergence and error analysis of the scheme are shown we also show that the scheme preserves the discrete mass even though the linearized scheme in 1 is conditionally stable and does not preserve the mass

ising distribution as a latent variable model during the past decades the ising distribution has attracted interest in many applied disciplines as the maximum entropy distribution associated to any set of correlated binary spin variables with observed means and covariances however numerically speaking the ising distribution is un practical so alternative models are often preferred to handle correlated binary data one popular alternative especially in life sciences is the cox distribution or the closely related dichotomized gaussian distribution and log normal cox point process where the spins are generated independently conditioned on the drawing of a latent variable with a multivariate normal distribution this article explores the conditions for a principled replacement of the ising distribution by a cox distribution it shows that the ising distribution itself can be treated as a latent variable model and it explores when this latent variable has a quasi normal distribution a variational approach to this question reveals a formal link with classic mean field methods especially opper and winther s adaptive tap approximation this link is confirmed by weak coupling plefka expansions of the different approximations and then by numerical tests overall this study suggests that an ising distribution can be replaced by a cox distribution in practical applications precisely when its parameters lie in the mean field domain

high order fdtd methods via derivative matching for maxwell s equations with material interfaces this paper introduces a series of novel hierarchical implicit derivative matching methods to restore the accuracy of high order finite difference time domain fdtd schemes of computational electromagnetics cem with material interfaces in one 1d and two spatial dimensions 2d by making use of fictitious points systematic approaches are proposed to locally enforce the physical jump conditions at material interfaces in a preprocessing stage to arbitrarily high orders of accuracy in principle while often limited by numerical instability orders up to 16 and 12 are achieved respectively in 1d and 2d detailed stability analyses are presented for the present approach to examine the upper limit in constructing embedded fdtd methods as natural generalizations of the high order fdtd schemes the proposed derivative matching methods automatically reduce to the standard fdtd schemes when the material interfaces are absent an interesting feature of the present approach is that it encompasses a variety of schemes of different orders in a single code another feature of the present approach is that it can be robustly implemented with other high accuracy time domain approaches such as the multiresolution time domain method and the local spectral time domain method to cope with material interfaces numerical experiments on both 1d and 2d problems are carried out to test the convergence examine the stability access the efficiency and explore the limitation of the proposed methods it is found that operating at their best capacity the proposed high order schemes could be over 2000 times more efficient than their fourth order versions in 2d in conclusion the present work indicates that the proposed hierarchical derivative matching methods might lead to practical high order schemes for numerical solution of time domain maxwell s equations with material interfaces

closed form expressions for the finite approximations of first and higher derivatives based on taylor series numerical differentiation formulas based on interpolating polynomials operators and lozenge diagrams can be simplified to one of the finite difference approximations based on taylor series in this paper we have presented closed form expressions of these approximations of arbitrary order for first and higher derivatives a comparison of the three types of approximations is given with an ideal digital differentiator by comparing their frequency responses the comparison reveals that the central difference approximations can be used as digital differentiators because they do not introduce any phase distortion and their amplitude response is closer to that of an ideal differentiator it is also observed that central difference approximations are in fact the same as maximally flat digital differentiators in the appendix a computer program written in mathematica is presented which can give the approximation of any order to the derivative of a function at a certain mesh point

numerically induced phase shift in the kdv soliton abstract when using a finite difference scheme to study the motion of a kdv solition a shift in the position of the solition from the exact solution is detected in this paper we retain the lowest order terms in the truncation error and treat them analytically as a perturbation of the kdv equation it is found that perturbation theory can be used to determine the numerically induced shift

a numerical energy conserving method for the dnls equation an implicit numerical energy conserving method is developed for the derivative nonlinear schrodinger dnls equation for periodic boundary conditions we find no numerical high frequency modulational instabilities in addition to the modulational instability from a linear analysis around a nonlinear state for the dnls equation if the modulation is small and k0 a22 2 t k0 is the wavenumber and a the amplitude the numerical scheme is used to follow the nonlinear behavior of the dnls modulational instability the numerical code is also tested by the evolution for one soliton initial data these tests show that if the modulation is not small compared to the background wave amplitude new nonlinear numerical instabilities are introduced

mathematical proof of closed form expressions for finite difference approximations based on taylor series taylor series based finite difference approximations of derivatives of a function have already been presented in closed forms with explicit formulas for their coefficients however those formulas were not derived mathematically and were based on observation of numerical results in this paper we provide a mathematical proof of those formulas by deriving them mathematically from the taylor series

a second order splitting method for the cahn hilliard equation a semi discrete finite element method requiring only continuous element is presented for the approximation of the solution of the evolutionary fourth order in space cahn hilliard equation optimal order error bounds are derived in various norms for an implementation which uses mass lumping the continuous problem has an energy based lyapunov functional it is proved that this property holds for the discrete problem

multidimensional optimization of finite difference schemes for computational aeroacoustics because of the long propagation distances computational aeroacoustics schemes must propagate the waves at the correct wave speeds and lower the isotropy error as much as possible the spatial differencing schemes are most frequently analyzed and optimized for one dimensional test cases therefore in multidimensional problems such optimized schemes may not have isotropic behavior in this work optimized finite difference schemes for multidimensional computational aeroacoustics are derived which are designed to have improved isotropy compared to existing schemes the derivation is performed based on both taylor series expansion and fourier analysis various explicit centered finite difference schemes and the associated boundary stencils have been derived and analyzed the isotropy corrector factor a parameter of the schemes can be determined by minimizing the integrated error between the phase or group velocities on different spatial directions the order of accuracy of the optimized schemes is the same as that of the classical schemes the advantage being in reducing the isotropy error the present schemes are restricted to equally spaced cartesian grids so the generalized curvilinear transformation method and cartesian grid methods are good candidates the optimized schemes are tested by solving various multidimensional problems of aeroacoustics

conservative numerical methods for x f x abstract two distinctly different numerical methods are developed for solving the conservative initial value problem x f x x o dot x o both methods conserve exactly the same total energy as does the given differential equation computer examples are described and

three dimensional simulation of unstable gravity driven infiltration of water into a porous medium infiltration of water in dry porous media is subject to a powerful gravity driven instability although the phenomenon of unstable infiltration is well known its description using continuum mathematical models has posed a significant challenge for several decades the classical model of water flow in the unsaturated flow the richards equation is unable to reproduce the instability here we present a computational study of a model of unsaturated flow in porous media that extends the richards equation and is capable of predicting the instability and captures the key features of gravity fingering quantitatively the extended model is based on a phase field formulation and is fourth order in space the new model poses a set of challenges for numerical discretizations such as resolution of evolving interfaces stiffness in space and time treatment of singularly perturbed equations and discretization of higher order spatial partial differential operators we develop a numerical algorithm based on isogeometric analysis a generalization of the finite element method that permits the use of globally smooth basis functions leading to a simple and efficient discretization of higher order spatial operators in variational form we illustrate the accuracy efficiency and robustness of our method with several examples in two and three dimensions in both homogeneous and strongly heterogeneous media we simulate for the first time unstable gravity driven infiltration in three dimensions and confirm that the new theory reproduces the fundamental features of water infiltration into a porous medium our results are consistent with classical experimental observations that demonstrate a transition from stable to unstable fronts depending on the infiltration flux

on line versus off line learning from random examples general results i propose a general model of on line learning from random examples which when applied to a smooth realizable stochastic rule yields the same asymptotic generalization error rate as optimal batch algorithms the approach is based on an iterative gaussian approximation to the posterior gibbs distribution of rule parameters

nonlinear aspects of the cahn hilliard equation abstract this paper treats phase separation within the context of the phenomenological cahn hilliard equation c t m c f c k 2 c where c is the concentration m c is the mobility and f c is homogeneou s free energy which is assumed here to be a fourth degree polynomial natural boundary conditions are introduced the full set of equilibrium solutions is specified a comparison theorem for stability criteria which was postulated by langer is proved here within the framework of the natural boundary conditions energy methods are used to define and estimate the limit of monotonic global stability it is pointed out that within the parameter region where the uniform homogeneous state is the only equilibrium solution there may still exist some internal excitable region in which the homogeneous solution possesses growing fluctuations furthermore a periodic instability is shown to exist in the metastable region in addition to the well known nucleation instability

a decomposed immersed interface method for variable coefficient elliptic equations with non smooth and discontinuous solutions a second order accurate finite difference method is presented for solving two dimensional variable coefficient elliptic equations on cartesian grids in which the coefficients the source term the solution and its derivatives may be nonsmooth or discontinuous across an interface a correction term is introduced to the standard central difference stencil so that the numerical discretization is well defined across the interface we also propose a new method to approximate the correction term as part of the iterative procedure the method is easy to implement since the correction term only needs to be added to the right hand side of the system therefore the coefficient matrix remains symmetric and diagonally dominant allowing for most standard solvers to be used numerical examples show good agreements with exact solutions and the order of accuracy is comparable with other immersed interface methods

glm ie generalised linear models inference estimation toolbox the glm ie toolbox contains functionality for estimation and inference in generalised linear models over continuous valued variables besides a variety of penalised least squares solvers for estimation it offers inference based on convex variational bounds on expectation propagation and on factorial mean field scalable and efficient inference in fully connected undirected graphical models or markov random fields with gaussian and non gaussian potentials is achieved by casting all the computations as matrix vector multiplications we provide a wide choice of penalty functions for estimation potential functions for inference and matrix classes with lazy evaluation for convenient modelling we designed the glm ie package to be simple generic and easily expansible most of the code is written in matlab including some mex files to be fully compatible to both matlab 7 x and gnu octave 3 3 x large scale probabilistic classification as well as sparse linear modelling can be performed in a common algorithmical framework by the glm ie toolkit

analysis and approximation of a fractional cahn hilliard equation we derive a fractional cahn hilliard equation fche by considering a gradient flow in the negative order sobolev space h alpha alpha in 0 1 where the choice alpha 1 corresponds to the classical cahn hilliard equation while the choice alpha 0 recovers the allen cahn equation the existence of a unique solution is established and it is shown that the equation preserves mass for all positive values of fractional order alpha and that it indeed reduces the free energy we then turn to the delicate question of the l infty boundedness of the solution and establish an l infty bound for the fche in the case where the nonlinearity is a quartic polynomial as a consequence of the estimates we are able to show that the fourier galerkin method delivers a spectral rate of convergence for the fche in the case of a semidiscrete approximation scheme finally we present results obtained using computational simulation of the fche for a variety of choices of fractional order alpha it is

a second order accurate linearized difference scheme for the two dimensional cahn hilliard equation the cahn hilliard equation is a nonlinear evolutionary equation that is of fourth order in space in this paper a linearized finite difference scheme is derived by the method of reduction of order it is proved that the scheme is uniquely solvable and convergent with the convergence rate of order two in a discrete l2 norm the coefficient matrix of the difference system is symmetric and positive definite so many well known iterative methods e g gauss seidel sor can be used to solve the system

information geometry and plefka s mean field theory an alternate derivation of plefka s method is presented which is obtained from minimizing gibbs energy it is shown that this method can be rederived using a perturbation expansion of kullback leibler divergence the latter having a nice information geometric interpretation

the immersed interface multigrid methods for interface problems new multigrid methods are developed for the maximum principle preserving immersed interface method applied to second order linear elliptic and parabolic pdes that involve interfaces and discontinuities for elliptic interface problems the multigrid solver developed in this paper works while some other multigrid solvers do not for linear parabolic equations we have developed the second order maximum principle preserving finite difference scheme in this paper we use the crank nicolson scheme to deal with the diffusion part and an explicit scheme for the first order derivatives numerical examples are also presented

numerical integrators that preserve symmetries and reversing symmetries we consider properties of flows the relationships between them and whether numerical integrators can be made to preserve these properties this is done in the context of automorphisms and antiautomorphisms of a certain group generated by maps associated to vector fields this new framework unifies several known constructions we also use the concept of covariance of a numerical method with respect to a group of coordinate transformations the main application is to explore the relationship between spatial symmetries reversing symmetries and time symmetry of flows and numerical integrators

susceptibility propagation by using diagonal consistency a susceptibility propagation that is constructed by combining a belief propagation and a linear response method is used for approximate computation for markov random fields herein we formulate an improved susceptibility propagation by using the concept of a diagonal matching method that is based on mean field approaches to inverse ising problems the proposed susceptibility propagation is robust for various network structures and it is reduced to the ordinary susceptibility propagation and to the adaptive thouless anderson palmer equation in special cases

the fourier collocation method for the cahn hilliard equation abstract in this paper a fourier collocation method for numerically solving cahn hilliard equations with periodic boundary conditions is developed we establish their semidiscrete and fully discrete schemes that inherit the energy dissipation property and mass conservation property from the associated continuous problem we prove existence and uniqueness of the numerical solution and derive the optimal error bounds we perform some numerical experiments which confirm our results

memory free dynamics for the thouless anderson palmer equations of ising models with arbitrary rotation invariant ensembles of random coupling matrices we propose an iterative algorithm for solving the thouless anderson palmer equations of ising models with arbitrary rotation invariant random coupling matrices in the thermodynamic limit we prove by means of the dynamical functional method that the proposed algorithm converges when the so called de almeida thouless criterion is fulfilled moreover we give exact analytical expressions for the rate of the convergence

adaptive and self averaging thouless anderson palmer mean field theory for probabilistic modeling we develop a generalization of the thouless anderson palmer tap mean field approach of disorder physics which makes the method applicable to the computation of approximate averages in probabilistic models for real data in contrast to the conventional tap approach where the knowledge of the distribution of couplings between the random variables is required our method adapts to the concrete set of couplings we show the significance of the approach in two ways our approach reproduces replica symmetric results for a wide class of toy models assuming a nonglassy phase with given disorder distributions in the thermodynamic limit on the other hand simulations on a real data model demonstrate that the method achieves more accurate predictions as compared to conventional tap approaches

a nonconforming finite element method for the cahn hilliard equation this paper reports a fully discretized scheme for the cahn hilliard equation the method uses a convexity splitting scheme to discretize in the temporal variable and a nonconforming finite element method to discretize in the spatial variable and the scheme can preserve the mass conservation and energy dissipation properties of the original problem some typical phase transition phenomena are also observed through the numerical examples

the fourier spectral method for the cahn hilliard equation in this paper a fourier spectral method for numerically solving cahn hilliard equation with periodic boundary conditions is developed we establish their semi discrete and fully discrete schemes that inherit the energy dissipation property and mass conservation property from the associated continuous problem we prove existence and uniqueness of the numerical solution and derive the optimal error bounds we perform some numerical experiments which confirm our results

isogeometric analysis of the cahn hilliard phase field model abstract the cahn hilliard equation involves fourth order spatial derivatives finite element solutions are not common because primal variational formulations of fourth order operators are only well defined and integrable if the finite element basis functions are piecewise smooth and globally c 1 continuous there are a very limited number of two dimensional finite elements possessing c 1 continuity applicable to complex geometries but none in three dimensions we propose isogeometric analysis as a technology that possesses a unique combination of attributes for complex problems involving higher order differential operators namely higher order accuracy robustness two and three dimensional geometric flexibility compact support and most importantly the possibility of c 1 and higher order continuity a nurbs based variational formulation for the cahn hilliard equation was tested on two and three dimensional problems we present steady state solutions in two dimensions and for the first time in three dimensions to achieve these results an adaptive time stepping method is introduced we also present a technique for desensitizing calculations to dependence on mesh refinement this enables the calculation of topologically correct solutions on coarse meshes opening the way to practical engineering applications of phase field methodology

stability and numerical dispersion analysis of a fourth order accurate fdtd method in order to obtain high order accuracy a fourth order accurate finite difference time domain fdtd method is presented by kyu pyung hwang unlike conventional fdtd methods a staggered backward differentiation scheme instead of the leapfrog scheme is used to approximate the temporal partial differential operator however the high order of its characteristic equation derived by the von neumann method makes the analysis of its numerical dispersion and stability very difficult in automatic control theory there are two effective methods for the stability analysis i e the routh hurwitz test and the jury test the combination of the von neumann method with each of the two can strictly derive the stability condition which only makes use of the coefficients of its characteristic equation without numerically solving it the method of analysis in this paper is also applicable in the stability and numerical analysis of other high order accurate fdtd methods

the space of interactions in neural networks gardner s computation with the cavity method gardner s 1987 1988 computation of the number of n bit patterns which can be stored in an optical neural network used as an associative memory is derived without replicas using the cavity method this allows for a unified presentation whatever the basic measure in the space of coupling constants but above all it gives the clear physical content of the assumption of replica symmetry tap equations are also derived

rigorous dynamics of expectation propagation based signal recovery from unitarily invariant measurements signal recovery from unitarily invariant measurements is investigated in this paper a message passing algorithm is formulated on the basis of expectation propagation ep a rigorous analysis is presented for the dynamics of the algorithm in the large system limit where both input and output dimensions tend to infinity while the compression rate is kept constant the main result is the justification of state evolution se equations conjectured by ma and ping this result implies that the ep based algorithm achieves the bayes optimal performance that was originally derived via a non rigorous tool in statistical physics and proved partially in a recent paper when the compression rate is larger than a threshold the proof is based on an extension of a conventional conditioning technique for the standard gaussian matrix to the case of the haar matrix

error estimates with smooth and nonsmooth data for a finite element method for the cahn hilliard equation a finite element method for the cahn hilliard equation a semilinear parabolic equation of fourth order is analyzed both in a spatially semidiscrete case and in a completely discrete case based on the backward euler method error bounds of optimal order over a finite time interval are obtained for solutions with smooth and nonsmooth initial data a detailed study of the regularity of the exact solution is included the analysis is based on local lipschitz conditions for the nonlinearity with respect to sobolev norms and the existence of a ljapunov functional for the exact and the discretized equations is essential a result concerning the convergence of the attractor of the corresponding approximate nonlinear semigroup upper semicontinuity with respect to the discretization parameters is obtained as a simple application of the nonsmooth data error estimate

continuous finite element methods which preserve energy properties for nonlinear problems abstract continuous finite elements are used to develop numerical methods which preserve energy properties for nonlinear ordinary and partial differential equations schemes are constructed that give high orders of accuracy time discretization algorithms for the cahn hilliard klein gordon korteweg de vries and schrodinger equations are presented comparisons are made with existing schemes

a nonconforming finite element method for the two dimensional cahn hilliard equation the cahn hilliard equation is a nonlinear evolutionary equation that is fourth order in space in this paper a continuous in time finite element galerkin approximation is considered we use the nonconforming morley element and derive optimal order error bounds in l 2

hybrid approximate message passing gaussian and quadratic approximations of message passing algorithms on graphs have attracted considerable recent attention due to their computational simplicity analytic tractability and wide applicability in optimization and statistical inference problems this paper presents a systematic framework for incorporating such approximate message passing amp methods in general graphical models the key concept is a partition of dependencies of a general graphical model into strong and weak edges with the weak edges representing small linearizable couplings of variables amp approximations based on the central limit theorem can be readily applied to aggregates of many weak edges and integrated with standard message passing updates on the strong edges the resulting algorithm which we call hybrid generalized approximate message passing hygamp can yield significantly simpler implementations of sum product and max sum loopy belief propagation by varying the partition of strong and weak edges a performance complexity tradeoff can be achieved group sparsity and multinomial logistic regression problems are studied as examples of the proposed methodology

an explicit finite difference scheme with exact conservation properties abstract a finite difference scheme for the numerical study of the korteweg de vries equation is constructed it is explicit and yet conserves exactly the energy of the computed solutions the underlying idea can also be applied to more general equations or systems numerical experiments are included

comparison of taylor finite difference and window finite difference and their application in fdtd the finite difference time domain fdtd method is an important tool in numerical electromagnetic simulation there are many ways to construct a finite difference approximation such as the taylor series expansion theorem the filtering theory etc this paper aims to provide the comparison between the taylor finite difference tfd scheme based on the taylor series expansion theorem and the window finite difference wfd scheme based on the filtering theory their properties have been examined in detail separately in addition the formula of the generalized finite difference gfd scheme is presented which can include both the tfd scheme and the wfd scheme furthermore their application in the numerical solution of maxwell s equations is presented the formulas for the stability criterion and the numerical dispersion relation are derived and analyzed in order to evaluate their performance more accurately a new definition of error is presented upon it the effect of several factors including the grid resolution the courant number and the aspect ratio of the cell on the performance of the numerical dispersion is examined

gaussian processes for classification mean field algorithms we derive a mean field algorithm for binary classification with gaussian processes that is based on the tap approach originally proposed in statistical physics of disordered systems the theory also yields an approximate leave one out estimator for the generalization error which is computed with no extra computational cost we show that from the tap approach it is possible to derive both a simpler naive mean field theory and support vector machines svms as limiting cases for both mean field algorithms and support vector machines simulation results for three small benchmark data sets are presented they show that one may get state of the art performance by using the leave one out estimator for model selection and the built in leave one out estimators are extremely precise when compared to the exact leave one out estimate the second result is taken as strong support for the internal consistency of the mean field approach

vector approximate message passing the standard linear regression slr problem is to recover a vector mathrm x 0 from noisy linear observations mathrm y mathrm ax 0 mathrm w the approximate message passing amp algorithm proposed by donoho maleki and montanari is a computationally efficient iterative approach to slr that has a remarkable property for large i i d sub gaussian matrices a its per iteration behavior is rigorously characterized by a scalar state evolution whose fixed points when unique are bayes optimal the amp algorithm however is fragile in that even small deviations from the i i d sub gaussian model can cause the algorithm to diverge this paper considers a vector amp vamp algorithm and shows that vamp has a rigorous scalar state evolution that holds under a much broader class of large random matrices a those that are right orthogonally invariant after performing an initial singular value decomposition svd of a the per iteration complexity of vamp is similar to that of amp in addition the fixed points of vamp s state evolution are consistent with the replica prediction of the minimum mean squared error derived by tulino caire verdu and shamai numerical experiments are used to confirm the effectiveness of vamp and its consistency with state evolution predictions

accelerating cross validation in multinomial logistic regression with l 1 regularization we develop an approximate formula for evaluating a cross validation estimator of predictive likelihood for multinomial logistic regression regularized by an l1 norm this allows us to avoid repeated optimizations required for literally conducting cross validation hence the computational time can be significantly reduced the formula is derived through a perturbative approach employing the largeness of the data size and the model dimensionality an extension to the elastic net regularization is also addressed the usefulness of the approximate formula is demonstrated on simulated data and the isolet dataset from the uci machine learning repository matlab and python codes implementing the approximate formula are distributed in obuchi 2017 takahashi and obuchi 2017

a wavenumber based extrapolation and interpolation method for use in conjunction with high order finite difference schemes the errors incurred in using extrapolation and interpolation in large scale computations are analyzed and quantified in the wavenumber space if a large extrapolation stencil is used the errors in the low wavenumbers can be significantly reduced however the errors in the high wavenumbers are at the same time greatly increased the opposite is true if the stencil size is reduced based on the wavenumber analysis an optimized extrapolation and interpolation method is proposed the optimization is carried out over a selected band of wavenumbers it is known that extrapolation often leads to numerical instability the instability is the result of large error amplification in the high wavenumber range to reduce the tendency to trigger numerical instability it is proposed that an extra constraint be imposed on the optimized extrapolation method the added constraint aims to reduce error amplification over the high wavenumbers numerical examples are provided to illustrate that accurate and stable numerical results can be obtained in large scale simulation using a high order finite difference scheme and the proposed optimized extrapolation method when the same problems are recomputed using the familiar high order polynomials extrapolation method in the lagrange form in one case the numerical results are plagued by large errors and ultimately instability in another problem it is found that the use of the lagrange polynomials extrapolation method would lead immediately to numerical instability

bayesian analysis of the scatterometer wind retrieval inverse problem some new approaches the retrieval of wind vectors from satellite scatterometer observations is a non linear inverse problem a common approach to solving inverse problems is to adopt a bayesian framework and to infer the posterior distribution of the parameters of interest given the observations by using a likelihood model relating the observations to the parameters and a prior distribution over the parameters we show how gaussian process priors can be used efficiently with a variety of likelihood models using local forward observation models and direct inverse models for the scatterometer we present an enhanced markov chain monte carlo method to sample from the resulting multimodal posterior distribution we go on to show how the computational complexity of the inference can be controlled by using a sparse sequential bayes algorithm for estimation with gaussian processes this helps to overcome the most serious barrier to the use of probabilistic gaussian process methods in remote sensing inverse problems which is the prohibitively large size of the data sets we contrast the sampling results with the approximations that are found by using the sparse sequential bayes algorithm

regular article finite difference schemes for u t x α δg δu that inherit energy conservation or dissipation property we propose a new procedure for designing by rote finite difference schemes that inherit energy conservation or dissipation property from nonlinear partial differential equations such as the korteweg de vries kdv equation and the cahn hilliard equation the most important feature of our procedure is a rigorous discretization of variational derivatives using summation by parts which implies that the inherited properties are satisfied exactly since the inherited properties are kept even if the time mesh size changes in the time evolution process we can use some appropriate time mesh adaptive methods to obtain numerical solutions through the derived schemes because of these properties the derived schemes are expected to be numerically stable and yield solutions converging to pde solutions and sufficiently flexible to treat the inheritance of the energy conservation and dissipation properties are verified numerically for the kdv equation and the cahn hilliard equation

fourier analysis of numerical algorithms for the maxwell equations the fourier method is used to analyze the dispersive dissipative and isotropy errors of various spatial and time discretizations applied to the maxwell equations on multi dimensional grids both cartesian grids and non cartesian grids based on hexagons and tetradecahedra are studied and compared the numerical errors are quantitatively determined in terms of phase speed wavenumber propagation direction gridspacings and cfl number the study shows that centered schemes are more efficient and accurate than upwind schemes and the non cartesian grids yield superior isotropy than the cartesian ones for the centered schemes the staggered grids produce less errors than the unstaggered ones a new unstaggered algorithm which has all the best properties is introduced using an optimization technique to determine the nodal weights the new algorithm provides the highest accuracy among all the schemes discussed the study also demonstrates that a proper choice of time discretization can reduce the overall numerical errors due to the spatial discretization

convergence condition of the tap equation for the infinite ranged ising spin glass model it is shown that the power expansion of the gibbs potential of the sk model up to second order in the exchange couplings leads to the tap equation this result remains valid for the general including a ferromagnetic exchange sk model theorems of power expansions and resolvent techniques are employed to solve the convergence problem the convergence condition is presented for the whole temperature range and for general distributions of the local magnetisations

chebyshev collocation method for the constant mobility cahn hilliard equation in a square domain abstract chebyshev collocation method was developed for constant mobility cahn hilliard equation the accuracy of the method was indirectly checked with the aid of two separate calculations of temporal derivative of total free energy we found that the derivatives of total free energy by two different ways of calculation coincided exactly when the number of grid points in the transitional layer within mixing region was 16 or more when the coefficient of gradient energy 2 was chosen correspondingly in the given domain we also investigated the application of kosloff and tal ezer mapping for the chebyshev collocation grid points in space and showed the possibility of the use of much larger timestep size than that of the chebyshev collocation method without mapping

numerical analysis of a continuum model of phase transition a method for solution of the cahn hilliard equation is presented unlike previous work the discrete equations for the new method possess a lyapunov function this makes it possible to prove convergence of the approximate solutions without assumptions beyond those necessary for existence and uniqueness of the differential equation several consequences are explored

analysis of mean field annealing in substactive interference cancellation in this contribution we derive the cost function corresponding to the linear complexity subtractive interference cancellation with tangent hyperbolic tentative decisions we use the cost function to analyse the fix points of solving the subtractive interference cancellation equations the analysis show that we can control the slope of the tangent hyperbolic functions so that the corresponding cost function is convex we also show that increasing the slope can make the cost non convex going from the convex regime into the non convex regime we prove that the bifurcation of the fix points for non singular signal correlation matrices consist of the fix point of interest together with a saddle node bifurcation this proves that tracking the solution from low slopes with a convex cost gradually increasing to higher slopes with non convex cost can bring us to the best solution being very close to the optimal determined by enumeration this tracking is the idea behind annealing we show monte carlo studies with a substantial signal to noise ratio gain compared to not using annealing we also show how annealing can be used to increase capacity at a given target bit error rate in fact this capacity gain is the same obtained by improved parallel interference cancellation making us believe that the latter includes a mechanism to avoid local minima similar to annealing

efficient learning in boltzmann machines using linear response theory the learning process in boltzmann machines is computationally very expensive the computational complexity of the exact algorithm is exponential in the number of neurons we present a new approximate learning algorithm for boltzmann machines based on mean field theory and the linear response theorem the computational complexity of the algorithm is cubic in the number of neurons in the absence of hidden units we show how the weights can be directly computed from the fixed point equation of the learning rules thus in this case we do not need to use a gradient descent procedure for the learning process we show that the solutions of this method are close to the optimal solutions and give a significant improvement when correlations play a significant role finally we apply the method to a pattern completion task and show good performance for networks up to 100 neurons

mean field equations for spin models with orthogonal interaction matrices we study the metastable states in ising spin models with orthogonal interaction matrices we focus on three realizations of this model the random case and two non random cases i e the fully frustrated model on an infinite dimensional hypercube and the so called sine model we use the mean field or tap equations which we derive by resumming the high temperature expansion of the gibbs free energy in some special non random cases we can find the absolute minimum of the free energy for the random case we compute the average number of solutions to the tap equations we find that the configurational entropy or complexity is extensive in the range trsb t tm we also present an apparently unrelated replica calculation which reproduces the analytical expression for the total number of tap solutions

finite difference calculus invariant structure of a class of algorithms for the nonlinear klein gordon equation in a previous work the authors have presented a formalism for deriving systematically invariant symmetric finite difference algorithms for nonlinear evolution differential equations that admit conserved quantities this formalism is herein cast in the context of exact finite difference calculus the algorithms obtained from the proposed formalism are shown to derive exactly from discrete scalar potential functions using finite difference calculus in the same sense as that of the corresponding differential equation being derivable from its associated energy function a conserved quantity a clear ramification of this result is that the derived algorithms preserve certain discrete invariant quantities which are the consistent counterpart of the invariant quantities in the continuous case results on the nonlinear stability of a class of algorithms that are derived using the proposed formalism and that preserve energy or linear momentum are discussed in the context of finite difference calculus some

an approximate analytical approach to resampling averages using a novel reformulation we develop a framework to compute approximate resampling data averages analytically the method avoids multiple retraining of statistical models on the samples our approach uses a combination of the replica trick of statistical physics and the tap approach for approximate bayesian inference we demonstrate our approach on regression with gaussian processes a comparison with averages obtained by monte carlo sampling shows that our method achieves good accuracy

mean field method with correlations determined by linear response we introduce a mean field approximation based on the reconciliation of maximum entropy and linear response for correlations in the cluster variation method within a general formalism that includes previous mean field methods we derive formulas improving on e g the bethe approximation and the sessak monasson result at high temperature applying the method to direct and inverse ising problems we find improvements over standard implementations

mean field theory of boltzmann machine learning i present a mean field theory for boltzmann machine learning derived by employing thouless anderson palmer free energy formalism to a full extent using the plefka expansion an extended theory that takes higher order correction to mean field free energy formalism into consideration is presented from which the mean field approximation of general orders along with the linear response correction are derived by truncating the plefka expansion up to desired orders a theoretical foundation for an effective trick of using diagonal weights introduced by kappen and rodr i guez is also given because of the finite system size and a lack of scaling assumptions on interaction coefficients the truncated free energy formalism cannot provide an exact description in the case of boltzmann machines accuracies of mean field approximations of several orders are compared by computer simulations

a theory of solving tap equations for ising models with general invariant random matrices we consider the problem of solving tap mean field equations by iteration for ising models with coupling matrices that are drawn at random from general invariant ensembles we develop an analysis of iterative algorithms using a dynamical functional approach that in the thermodynamic limit yields an effective dynamics of a single variable trajectory our main novel contribution is the expression for the implicit memory term of the dynamics for general invariant ensembles by subtracting these terms that depend on magnetizations at previous time steps the implicit memory terms cancel making the iteration dependent on a gaussian distributed field only the tap magnetizations are stable fixed points if a de almeida thouless stability criterion is fulfilled we illustrate our method explicitly for coupling matrices drawn from the random orthogonal ensemble

the explicit jump immersed interface method finite difference methods for pdes with piecewise smooth solutions many boundary value problems bvps or initial bvps have nonsmooth solutions with jumps along lower dimensional interfaces the explicit jump immersed interface method ejiim was developed following li s fast iterative immersed interface method fiiim recognizing that the foundation for the efficient solution of many such problems is a good solver for elliptic bvps ejiim generalizes the class of problems for which fiiim is applicable it handles interfaces between constant and variable coefficients and extends the immersed interface method iim to bvps on irregular domains with neumann and dirichlet boundary conditions proofs of second order convergence for a one dimensional 1d problem with piecewise constant coefficients and for two dimensional 2d problems with singular sources are given other problems are reduced to the singular sources case with additional equations determining the source strengths the advantages of ejiim are high quality of solutions even on coarse grids and easy adaptation to many problems with complicated geometries while still maintaining the efficiency of the fiiim

accuracy and conservation properties in numerical integration the case of the korteweg de vries equation when numerically integrating time dependent differential equations it is often recommended to employ methods that preserve some of the invariant quantities mass energy etc of the problem being considered this recommendation is usually justified on the grounds that conservation of invariant quantities may ensure that the numerical solution possesses some important qualitative features however there are cases where schemes that preserve invariants are also advantageous in that they possess favourable error propagation mechanisms that render them superior from a quantitative point of view in the present paper we consider the korteweg de vries equation as a case study we show rigorously that for soliton problems and at leading order the error of conservative schemes consists of a phase error that grows linearly with time plus a complementary term that is bounded in the h 1 norm uniformly in time for general nonconservative schemes the error involves a linearly growing amplitude error a quadratically growing phase error and a complementary term that grows linearly in the h 1 norm numerical experiments are presented

analytical and numerical aspects of certain nonlinear evolution equations 1v numerical modified korteweg de vries equation various numerical methods are used in order to approximate the korteweg de vries equation namely i zabusky kruskal scheme ii hopscotch method iii a scheme due to goda iv a proposed local scheme v a proposed global scheme vi a scheme suggested by kruskal vii split step fourier method by tappert viii an improved split step fourier method and ix pseudospectral method by fornberg and whitham comparisons between our proposed scheme which is developed using notions of the inverse scattering transform and the other utilized schemes are obtained

three new unconditionally stable fdtd methods with high order accuracy three novel finite difference time domain fdtd methods based on the split step ss scheme with high order accuracy are presented which are proven to be unconditionally stable in the first novel method symmetric operator and uniform splitting are adopted simultaneously to split the matrix derived from the classical maxwell s equations into six sub matrices accordingly the time step is divided into six sub steps the second and third proposed methods are obtained by adjusting the sequence of the sub matrices deduced in the first method so all the novel methods presented in the paper have similar formulations of which the numerical dispersion errors and the anisotropic errors are lower than the alternating direction implicit finite difference time domain adi fdtd method the initial ss fdtd method and the modified ss fdtd method based on the strang splitting scheme specifically for the second method corresponding to a certain cell per wavelength cpw there is a courant number value making the numerical anisotropic error to be zero while in the third novel method corresponding to a certain courant number value there exists a cpw making the numerical anisotropic error to be zero in order to demonstrate the high order accuracy and efficiency of the proposed methods numerical results are presented

statistical mechanical analysis of the kronecker channel model for multiple input multiple output wireless communication the kronecker channel model of wireless communication is analyzed using statistical mechanics methods in the model spatial proximities among transmission reception antennas are taken into account as certain correlation matrices which generally yield nontrivial dependence among symbols to be estimated this prevents accurate assessment of the communication performance by naively using a previously developed analytical scheme based on a matrix integration formula in order to resolve this difficulty we develop a formalism that can formally handle the correlations in kronecker models based on the known scheme unfortunately direct application of the developed scheme is in general practically difficult however the formalism is still useful indicating that the effect of the correlations generally increase after the fourth order with respect to correlation strength therefore the known analytical scheme offers a good approximation in performance evaluation when the correlation strength is sufficiently small for a class of specific correlation we show that the performance analysis can be mapped to the problem of one dimensional spin systems in random fields which can be investigated without approximation by the belief propagation algorithm