on inverse problems with unknown operators consider a problem of recovery of a smooth function signal image f spl isin spl fscr spl isin l sub 2 0 1 sup d passed through an unknown filter and then contaminated by a noise a typical model discussed in the paper is described by a stochastic differential equation dy sub f sup spl epsi t hf t dt spl epsi dw t t spl isin 0 1 sup d spl epsi 0 where h is a linear operator modeling the filter and w is a brownian motion sheet modeling a noise the aim is to recover f with asymptotically as spl epsi spl rarr 0 minimax mean integrated squared error traditionally the problem is studied under the assumption that the operator h is known then the ill posedness of the problem is the main concern in this paper a more complicated and more realistic case is considered where the operator is unknown instead a training set of n pairs e sub l y e sub l sup spl sigma l 1 2 n where e sub l is an orthonormal system in l sub 2 and y e sub l sup spl sigma denote the solutions of stochastic differential equations of the above type with f e sub l and spl epsi spl sigma is available an optimal in a minimax sense over considered operators and signals data driven recovery of the signal is suggested the influence of spl epsi spl sigma and n on the recovery is thoroughly studied in particular we discuss an interesting case of a larger noise during the training and present formulas for threshold levels for n beyond which no improvement in recovery of input signals occurs we also discuss the case where h is an unknown perturbation of a known operator we describe a class of perturbations for which the accuracy of recovery of the signal is asymptotically the same up to a constant as in the case of precisely known operator

reduction of boundary artifacts in image restoration the abrupt boundary truncation of an image introduces artifacts in the restored image the traditional solution is to smooth the image data using special window functions such as hamming or trapezoidal windows this is followed by zero padding and linear convolution with the restoration filter this method improves the results but still distorts the image especially at the margins instead of the above method we propose a different procedure this procedure is simple and exploits the natural property of circular or periodic convolution of the discrete fourier transform dft instead of padding the image by zeros it is padded by a reflected version of it this is followed by circular convolution with the restoration filter this procedure is shown to lead to better restoration results than the windowing and linear convolution techniques the computational effort is also improved since our method requires half the number of computations required by the conventional linear deconvolution method

expectation propagation detection for high order high dimensional mimo systems modern communications systems use multiple input multiple output mimo and high order qam constellations for maximizing spectral efficiency however as the number of antennas and the order of the constellation grow the design of efficient and low complexity mimo receivers possesses big technical challenges for example symbol detection can no longer rely on maximum likelihood detection or sphere decoding methods as their complexity increases exponentially with the number of transmitters receivers in this paper we propose a low complexity high accuracy mimo symbol detector based on the expectation propagation ep algorithm ep allows approximating iteratively at polynomial time the posterior distribution of the transmitted symbols we also show that our ep mimo detector outperforms classic and state of the art solutions reducing the symbol error rate at a reduced computational complexity

modal coupling in the vibration of fluid loaded cylindrical shells an approach is presented to evaluate the pressure field and vibratory response of a finite fluid loaded cylindrical shell with infinite rigid extensions which are connected to the shell the approach combines a generalized fourier series or in vacuo eigenfunction expansion of the velocity field of the shell with a green s function and integral equation representation of the acoustic loading on the shell although modes with different circumferential wavenumbers are decoupled all modes with identical circumferential wavenumbers are coupled via the fluid the algebraic equations which account for this coupling include both shell impedances and acoustic impedances which are mode dependent general integral expressions are presented for the acoustic impedances which include both self radiation and interaction impedances in order to illustrate the general characteristics of these acoustic impedances the impedances are investigated for a particular set of eigenfunctions low frequency and asymptotic expressi

nested expectation propagation for gaussian process classification this paper considers probabilistic multinomial probit classification using gaussian process gp priors challenges with multiclass gp classification are the integration over the non gaussian posterior distribution and the increase of the number of unknown latent variables as the number of target classes grows expectation propagation ep has proven to be a very accurate method for approximate inference but the existing ep approaches for the multinomial probit gp classification rely on numerical quadratures or independence assumptions between the latent values associated with different classes to facilitate the computations in this paper we propose a novel nested ep approach which does not require numerical quadratures and approximates accurately all between class posterior dependencies of the latent values but still scales linearly in the number of classes the predictive accuracy of the nested ep approach is compared to laplace variational bayes and markov chain monte carlo mcmc approximations with various benchmark data sets in the experiments nested ep was the most consistent method compared to mcmc sampling but in terms of classification accuracy the differences between all the methods were small from a practical point of view

gaussian kullback leibler approximate inference we investigate gaussian kullback leibler g kl variational approximate inference techniques for bayesian generalised linear models and various extensions in particular we make the following novel contributions sufficient conditions for which the g kl objective is differentiable and convex are described constrained parameterisations of gaussian covariance that make g kl methods fast and scalable are provided the lower bound to the normalisation constant provided by g kl methods is proven to dominate those provided by local lower bounding methods complexity and model applicability issues of g kl versus other gaussian approximate inference methods are discussed numerical results comparing g kl and other deterministic gaussian approximate inference methods are presented for robust gaussian process regression models with either student t or laplace likelihoods large scale bayesian binary logistic regression models and bayesian sparse linear models for sequential experimental design

image denoising using scale mixtures of gaussians in the wavelet domain we describe a method for removing noise from digital images based on a statistical model of the coefficients of an overcomplete multiscale oriented basis neighborhoods of coefficients at adjacent positions and scales are modeled as the product of two independent random variables a gaussian vector and a hidden positive scalar multiplier the latter modulates the local variance of the coefficients in the neighborhood and is thus able to account for the empirically observed correlation between the coefficient amplitudes under this model the bayesian least squares estimate of each coefficient reduces to a weighted average of the local linear estimates over all possible values of the hidden multiplier variable we demonstrate through simulations with images contaminated by additive white gaussian noise that the performance of this method substantially surpasses that of previously published methods both visually and in terms of mean squared error

embedding of meshes in mobius cubes the hypercube is one of the most popular interconnection networks since it has simple structure and is easy to implement mobius cubes form a class of hypercube variants that give better performance with the same number of edges and vertices in this paper we consider embedding of meshes in mobius cubes the main results obtained in this paper are 1 for n 1 there exists a 2x2 n 1 mesh that can be embedded in the n dimensional mobius cube with dilation 1 and expansion 1 2 for n 4 there exists a 4x2 n 2 mesh that can be embedded in the n dimensional mobius cube with dilation 2 and expansion 1 3 for n 4 there are two disjoint 4x2 n 3 meshes that can be embedded in the 0 type n dimensional mobius cube with dilation 1 4 for n 4 there are two disjoint 4x2 n 3 meshes that can be embedded in the 1 type n dimensional mobius cube with dilation 2 results of 1 and 3 are optimal in the sense that the dilations of the embeddings are 1

beyond inverse ising model structure of the analytical solution i consider the problem of deriving couplings of a statistical model from measured correlations a task which generalizes the well known inverse ising problem after reminding that such problem can be mapped on the one of expressing the entropy of a system as a function of its corresponding observables i show the conditions under which this can be done without resorting to iterative algorithms i find that inverse problems are local the inverse fisher information is sparse whenever the corresponding models have a factorized form and the entropy can be split in a sum of small cluster contributions i illustrate these ideas through two examples the ising model on a tree and the one dimensional periodic chain with arbitrary order interaction and support the results with numerical simulations the extension of these methods to more general scenarios is finally discussed

memory free dynamics for the thouless anderson palmer equations of ising models with arbitrary rotation invariant ensembles of random coupling matrices we propose an iterative algorithm for solving the thouless anderson palmer equations of ising models with arbitrary rotation invariant random coupling matrices in the thermodynamic limit we prove by means of the dynamical functional method that the proposed algorithm converges when the so called de almeida thouless criterion is fulfilled moreover we give exact analytical expressions for the rate of the convergence

expectation propagation for continuous time stochastic processes we consider the inverse problem of reconstructing the posterior measure over the trajectories of a diffusion process from discrete time observations and continuous time constraints we cast the problem in a bayesian framework and derive approximations to the posterior distributions of single time marginals using variational approximate inference giving rise to an expectation propagation type algorithm for non linear diffusion processes this is achieved by leveraging moment closure approximations we then show how the approximation can be extended to a wide class of discrete state markov jump processes by making use of the chemical langevin equation our empirical results show that the proposed method is computationally efficient and provides good approximations for these classes of inverse problems

rigorous dynamics of expectation propagation based signal recovery from unitarily invariant measurements signal recovery from unitarily invariant measurements is investigated in this paper a message passing algorithm is formulated on the basis of expectation propagation ep a rigorous analysis is presented for the dynamics of the algorithm in the large system limit where both input and output dimensions tend to infinity while the compression rate is kept constant the main result is the justification of state evolution se equations conjectured by ma and ping this result implies that the ep based algorithm achieves the bayes optimal performance that was originally derived via a non rigorous tool in statistical physics and proved partially in a recent paper when the compression rate is larger than a threshold the proof is based on an extension of a conventional conditioning technique for the standard gaussian matrix to the case of the haar matrix

hybrid approximate message passing gaussian and quadratic approximations of message passing algorithms on graphs have attracted considerable recent attention due to their computational simplicity analytic tractability and wide applicability in optimization and statistical inference problems this paper presents a systematic framework for incorporating such approximate message passing amp methods in general graphical models the key concept is a partition of dependencies of a general graphical model into strong and weak edges with the weak edges representing small linearizable couplings of variables amp approximations based on the central limit theorem can be readily applied to aggregates of many weak edges and integrated with standard message passing updates on the strong edges the resulting algorithm which we call hybrid generalized approximate message passing hygamp can yield significantly simpler implementations of sum product and max sum loopy belief propagation by varying the partition of strong and weak edges a performance complexity tradeoff can be achieved group sparsity and multinomial logistic regression problems are studied as examples of the proposed methodology

gaussian processes for classification mean field algorithms we derive a mean field algorithm for binary classification with gaussian processes that is based on the tap approach originally proposed in statistical physics of disordered systems the theory also yields an approximate leave one out estimator for the generalization error which is computed with no extra computational cost we show that from the tap approach it is possible to derive both a simpler naive mean field theory and support vector machines svms as limiting cases for both mean field algorithms and support vector machines simulation results for three small benchmark data sets are presented they show that one may get state of the art performance by using the leave one out estimator for model selection and the built in leave one out estimators are extremely precise when compared to the exact leave one out estimate the second result is taken as strong support for the internal consistency of the mean field approach

bayesian analysis of the scatterometer wind retrieval inverse problem some new approaches the retrieval of wind vectors from satellite scatterometer observations is a non linear inverse problem a common approach to solving inverse problems is to adopt a bayesian framework and to infer the posterior distribution of the parameters of interest given the observations by using a likelihood model relating the observations to the parameters and a prior distribution over the parameters we show how gaussian process priors can be used efficiently with a variety of likelihood models using local forward observation models and direct inverse models for the scatterometer we present an enhanced markov chain monte carlo method to sample from the resulting multimodal posterior distribution we go on to show how the computational complexity of the inference can be controlled by using a sparse sequential bayes algorithm for estimation with gaussian processes this helps to overcome the most serious barrier to the use of probabilistic gaussian process methods in remote sensing inverse problems which is the prohibitively large size of the data sets we contrast the sampling results with the approximations that are found by using the sparse sequential bayes algorithm

periodic boxcar deconvolution and diophantine approximation we consider the nonparametric estimation of a periodic function that is observed in additive gaussian white noise after convolution with a boxcar the indicator function of an interval this is an idealized model for the problem of recovery of noisy signals and images observed with motion blur if the length of the boxcar is rational then certain frequencies are irretreviably lost in the periodic model we consider the rate of convergence of estimators when the length of the boxcar is irrational using classical results on approximation of irrationals by continued fractions a basic question of interest is whether the minimax rate of convergence is slower than for nonperiodic problems with 1 f like convolution filters the answer turns out to depend on the type and smoothness of functions being estimated in a manner not seen with homogeneous filters

wavelet based statistical signal processing using hidden markov models wavelet based statistical signal processing techniques such as denoising and detection typically model the wavelet coefficients as independent or jointly gaussian these models are unrealistic for many real world signals we develop a new framework for statistical signal processing based on wavelet domain hidden markov models hmms that concisely models the statistical dependencies and non gaussian statistics encountered in real world signals wavelet domain hmms are designed with the intrinsic properties of the wavelet transform in mind and provide powerful yet tractable probabilistic signal models efficient expectation maximization algorithms are developed for fitting the hmms to observational signal data the new framework is suitable for a wide range of applications including signal estimation detection classification prediction and even synthesis to demonstrate the utility of wavelet domain hmms we develop novel algorithms for signal denoising classification and detection

efficient learning in boltzmann machines using linear response theory the learning process in boltzmann machines is computationally very expensive the computational complexity of the exact algorithm is exponential in the number of neurons we present a new approximate learning algorithm for boltzmann machines based on mean field theory and the linear response theorem the computational complexity of the algorithm is cubic in the number of neurons in the absence of hidden units we show how the weights can be directly computed from the fixed point equation of the learning rules thus in this case we do not need to use a gradient descent procedure for the learning process we show that the solutions of this method are close to the optimal solutions and give a significant improvement when correlations play a significant role finally we apply the method to a pattern completion task and show good performance for networks up to 100 neurons

a novel algorithm to embed a multi dimensional torus into a locally twisted cube this paper addresses how to embed a multi dimensional torus of maximal size of into an n dimensional locally twisted cube the major contribution of this paper is that for n 4 every k dimensional torus of size 2 s 1x2 s 2x x2 s k satisfying i 1 ks i n can be embedded into an n dimensional locally twisted cube with dilation 2 and unit expansion further an embedding algorithm can be constructed based on our embedding method and the time complexity of this algorithm is linear with respect to the size of the locally twisted cube the embedding is optimal in the sense that it has unit expansion

sparse on line gaussian processes we develop an approach for sparse representations of gaussian process gp models which are bayesian types of kernel machines in order to overcome their limitations for large data sets the method is based on a combination of a bayesian on line algorithm together with a sequential construction of a relevant subsample of the data that fully specifies the prediction of the gp model by using an appealing parameterization and projection techniques in a reproducing kernel hilbert space recursions for the effective parameters and a sparse gaussian approximation of the posterior process are obtained this allows for both a propagation of predictions and bayesian error measures the significance and robustness of our approach are demonstrated on a variety of experiments

estimation of finite mixture distributions through bayesian sampling summary a formal bayesian analysis of a mixture model usually leads to intractable calculations since the posterior distribution takes into account all the partitions of the sample we present approximation methods which evaluate the posterior distribution and bayes estimators by gibbs sampling relying on the missing data structure of the mixture model the data augmentation method is shown to converge geometrically since a duality principle transfers properties from the discrete missing data chain to the parameters the fully conditional gibbs alternative is shown to be ergodic and geometric convergence is established in the normal case we also consider non informative approximations associated with improper priors assuming that the sample corresponds exactly to a k component mixture

expectation propagation with factorizing distributions a gaussian approximation and performance results for simple models we discuss the expectation propagation ep algorithm for approximate bayesian inference using a factorizing posterior approximation for neural network models we use a central limit theorem argument to make ep tractable when the number of parameters is large for two types of models we show that ep can achieve optimal generalization performance when data are drawn from a simple distribution

wavelet decomposition approaches to statistical inverse problems summary a wide variety of scientific settings involve indirect noisy measurements where one faces a linear inverse problem in the presence of noise primary interest is in some function f t but data are accessible only about some linear transform corrupted by noise the usual linear methods for such inverse problems do not perform satisfactorily when f t is spatially inhomogeneous one existing nonlinear alternative is the wavelet vaguelette decomposition method based on the expansion of the unknown f t in wavelet series in the vaguelette wavelet decomposition method proposed here the observed data are expanded directly in wavelet series the performances of various methods are compared through exact risk calculations in the context of the estimation of the derivative of a function observed subject to noise a result is proved demonstrating that with a suitable universal threshold somewhat larger than that used for standard denoising problems both the wavelet based approaches have an ideal spatial adaptivity property

sound radiation by a baffled cylindrical shell a numerical technique based on boundary integral equations part 2 abstract in part 1 a method was proposed to evaluate the sound pressure radiated by a baffled cylindrical shell here the numerical aspect of the method is examined in some detail as an example it is applied to the evaluation of the sound pressure radiated at infinity examples of directivity patterns are presented

improving variational methods via pairwise linear response identities inference methods are often formulated as variational approximations these approximations allow easy evaluation of statistics by marginalization or linear response but these estimates can be inconsistent we show that by introducing constraints on covariance one can ensure consistency of linear response with the variational parameters and in so doing inference of marginal probability distributions is improved for the bethe approximation and its generalizations improvements are achieved with simple choices of the constraints the approximations are presented as variational frameworks iterative procedures related to message passing are provided for finding the minima

the mathematical structure of the approximate linear response relation in this paper we study the mathematical structures of the linear response relation based on plefka s expansion and the cluster variation method in terms of the perturbation expansion and we show how this linear response relation approximates the correlation functions of the specified system moreover by comparing the perturbation expansions of the correlation functions estimated by the linear response relation based on these approximation methods with exact perturbative forms of the correlation functions we are able to explain why the approximate techniques using the linear response relation work well

modeling tangential vector fields on a sphere abstractphysical processes that manifest as tangential vector fields on a sphere are common in geophysical and environmental sciences these naturally occurring vector fields are often subject to physical constraints such as being curl free or divergence free we start with constructing parametric models for curl free and divergence free vector fields that are tangential to the unit sphere through applying the surface gradient or the surface curl operator to a scalar random potential field on the unit sphere using the helmholtz hodge decomposition we then construct a class of simple but flexible parametric models for general tangential vector fields which are represented as a sum of a curl free and a divergence free components we propose a likelihood based parameter estimation procedure and show that fast computation is possible even for large data sets when the observations are on a regular latitude longitude grid characteristics and practical utility of the proposed methodology are illustrated th

fitting continuous arma models to unequally spaced spatial data abstract methods for fitting continuous spatial autoregressive moving average arma models to unequally spaced observations in two dimensions are reviewed and extended these are models with rational two dimensional spectra assuming gaussian input noise and observational errors maximum likelihood methods are used to estimate the arma parameters and the regression coefficients of the deterministic trend when the number of observations is too large for exact maximum likelihood estimation approximate maximum likelihood estimation is used based on nearest neighbors comparisons of nearest neighbor methods with exact likelihood methods are presented predictions of the height of the field at unobserved points can be calculated with confidence intervals

new tight frames of curvelets and optimal representations of objects with piecewise c2 singularities this paper introduces new tight frames of curvelets to address the problem of finding optimally sparse representations of objects with discontinuities along piecewise c 2 edges conceptually the curvelet transform is a multiscale pyramid with many directions and positions at each length scale and needle shaped elements at fine scales these elements have many useful geometric multiscale features that set them apart from classical multiscale representations such as wavelets for instance curvelets obey a parabolic scaling relation which says that at scale 2 j each element has an envelope that is aligned along a ridge of length 2 j 2 and width 2 j we prove that curvelets provide an essentially optimal representation of typical objects f that are c 2 except for discontinuities along piecewise c 2 curves such representations are nearly as sparse as if f were not singular and turn out to be far more sparse than the wavelet decomposition of the object for instance the n term partial reconstruction f c n obtained by selecting the n largest terms in the curvelet series obeys f f c n 2 l2 c n 2 log n 3 n this rate of convergence holds uniformly over a class of functions that are c 2 except for discontinuities along piecewise c 2 curves and is essentially optimal in comparison the squared error of n term wavelet approximations only converges as n 1 as n which is considerably worse than the optimal behavior

mimo detection for high order qam based on a gaussian tree approximation this paper proposes a new detection algorithm for mimo communication systems employing high order qam constellations the factor graph that corresponds to this problem is very loopy in fact it is a complete graph hence a straightforward application of the belief propagation bp algorithm yields very poor results our algorithm is based on an optimal tree approximation of the gaussian density of the unconstrained linear system the finite set constraint is then applied to obtain a cycle free discrete distribution simulation results show that even though the approximation is not directly applied to the exact discrete distribution applying the bp algorithm to the cycle free factor graph outperforms current methods in terms of both performance and complexity the improved performance of the proposed algorithm is demonstrated on the problem of mimo detection

the empirical mode decomposition and the hilbert spectrum for nonlinear and non stationary time series analysis a new method for analysing nonlinear and non stationary data has been developed the key part of the method is the empirical mode decomposition method with which any complicated data set can be dec

susceptibility propagation by using diagonal consistency a susceptibility propagation that is constructed by combining a belief propagation and a linear response method is used for approximate computation for markov random fields herein we formulate an improved susceptibility propagation by using the concept of a diagonal matching method that is based on mean field approaches to inverse ising problems the proposed susceptibility propagation is robust for various network structures and it is reduced to the ordinary susceptibility propagation and to the adaptive thouless anderson palmer equation in special cases

sparse approximate inference for spatio temporal point process models abstractspatio temporal log gaussian cox process models play a central role in the analysis of spatially distributed systems in several disciplines yet scalable inference remains computationally challenging both due to the high resolution modeling generally required and the analytically intractable likelihood function here we exploit the sparsity structure typical of spatially discretized log gaussian cox process models by using approximate message passing algorithms the proposed algorithms scale well with the state dimension and the length of the temporal horizon with moderate loss in distributional accuracy they hence provide a flexible and faster alternative to both nonlinear filtering smoothing type algorithms and to approaches that implement the laplace method or expectation propagation on block sparse latent gaussian models we infer the parameters of the latent gaussian model using a structured variational bayes approach we demonstrate the proposed framework on simulation studies with bo

tree structure expectation propagation for ldpc decoding over the bec we present the tree structure expectation propagation tree ep algorithm to decode low density parity check ldpc codes over discrete memoryless channels dmcs expectation propagation generalizes belief propagation bp in two ways first it can be used with any exponential family distribution over the cliques in the graph second it can impose additional constraints on the marginal distributions we use this second property to impose pairwise marginal constraints over pairs of variables connected to a check node of the ldpc code s tanner graph thanks to these additional constraints the tree ep marginal estimates for each variable in the graph are more accurate than those provided by bp we also reformulate the tree ep algorithm for the binary erasure channel bec as a peeling type algorithm tep and we show that the algorithm has the same computational complexity as bp and it decodes a higher fraction of errors we describe the tep decoding process by a set of differential equations that represents the expected residual graph evolution as a function of the code parameters the solution of these equations is used to predict the tep decoder performance in both the asymptotic regime and the finite length regimes over the bec while the asymptotic threshold of the tep decoder is the same as the bp decoder for regular and optimized codes we propose a scaling law for finite length ldpc codes which accurately approximates the tep improved performance and facilitates its optimization

sound radiation from a finite fluid filled submerged cylindrical shell with porous material sandwich a theoretical model is developed to predict the far field sound radiation from a finite fluid filled submerged cylindrical thin shell with porous material sandwich a combination of the wave number domain approach and the transfer matrix method is presented which is convenient to analyze the vibratory responses in terms of wave number expressions for the spectral radial velocity of the outer surface of the shell are also formulated a prediction model for sound radiation at far field is given by using the boundary integral equation extensive numerical results are also presented to illustrate the general characteristics of the far field sound pressure as a function of frequency an experimental verification is performed and a good correlation between the theoretical results and the experimental results shows that the theoretical study work in this paper is correct

systems of convolution equations deconvolution shannon sampling and the wavelet and gabor transforms linear translation invariant systems e g sensors linear filters are modeled by the convolution equation s f mu where f is the input signal mu is the system impulse response function or more generally impulse response distribution and s is the output signal in many applications the output s is an inadequate approximation of f which motivates solving the convolution equation for f i e deconvolving f from mu if the function mu is time limited compactly supported and nonsingular it is proven that this deconvolution problem is ill posed a theory of solving such equations has been developed by berenstein et al it circumvents ill posedness by using a multichannel system if the signal f is overdetermined by using a system of convolution equations s i f mu i i 1 ldots n the problem of solving for f is well posed if the set of convolvers mu i satisfies the condition of being what is called strongly coprime in this case there exist compactly suppor

the concave convex procedure the concave convex procedure cccp is a way to construct discrete time iterative dynamical systems that are guaranteed to decrease global optimization and energy functions monotonically this procedure can be applied to almost any optimization problem and many existing algorithms can be interpreted in terms of it in particular we prove that all expectation maximization algorithms and classes of legendre minimization and variational bounding algorithms can be reexpressed in terms of cccp we show that many existing neural network and mean field theory algorithms are also examples of cccp the generalized iterative scaling algorithm and sinkhorn s algorithm can also be expressed as cccp by changing variables cccp can be used both as a new way to understand and prove the convergence of existing optimization algorithms and as a procedure for generating new algorithms

how to expand around mean field theory using high temperature expansions high temperature expansions performed at a fixed order parameter provide a simple and systematic way to derive and correct mean field theories for statistical mechanical models for models like spin glasses which have general couplings between spins the authors show that these expansions generate the thouless anderson palmer equations at low order they explicitly calculate the corrections to tap theory for these models for ferromagnetic models they show that their expansions can easily be converted into 1 d expansions around mean field theory where d is the number of spatial dimensions only a small finite number of graphs need to be calculated to generate each order in 1 d for thermodynamic quantities like free energy or magnetization unlike previous 1 d expansions the expansions are valid in the low temperature phases of the models considered they consider alternative ways to expand around mean field theory besides 1 d expansions in contrast to the 1 d expansion for the critical temperature which is presumably asymptotic these schemes can be used to devise convergent expansions for the critical temperature they also appear to give convergent series for thermodynamic quantities and critical exponents they test the schemes using the spherical model where their properties can be studied using exact expressions

approximate inference in boltzmann machines inference in boltzmann machines is np hard in general as a result approximations are often necessary we discuss first order mean field and second order onsager truncations of the plefka expansion of the gibbs free energy the bethe free energy is introduced and rewritten as a gibbs free energy from there a convergent belief optimization algorithm is derived to minimize the bethe free energy an analytic expression for the linear response estimate of the covariances is found which is exact on boltzmann trees finally a number of theorems is proven concerning the plefka expansion relating the first order mean field and the second order onsager approximation to the bethe approximation experiments compare mean field approximation onsager approximation belief propagation and belief optimization

forward fourier wavelet regularized deconvolution for ill conditioned systems we propose an efficient hybrid fourier wavelet regularized deconvolution forward algorithm that performs noise regularization via scalar shrinkage in both the fourier and wavelet domains the fourier shrinkage exploits the fourier transform s economical representation of the colored noise inherent in deconvolution whereas the wavelet shrinkage exploits the wavelet domain s economical representation of piecewise smooth signals and images we derive the optimal balance between the amount of fourier and wavelet regularization by optimizing an approximate mean squared error mse metric and find that signals with more economical wavelet representations require less fourier shrinkage forward is applicable to all ill conditioned deconvolution problems unlike the purely wavelet based wavelet vaguelette deconvolution wvd moreover its estimate features minimal ringing unlike the purely fourier based wiener deconvolution even in problems for which the wvd was designed we prove that forward s mse decays with the optimal wvd rate as the number of samples increases further we demonstrate that over a wide range of practical sample lengths forward improves on wvd s performance

correctness of local probability propagation in graphical models with loops graphical models such as bayesian networks and markov networks represent joint distributions over a set of variables by means of a graph when the graph is singly connected local propagation rules of the sort proposed by pearl 1988 are guaranteed to converge to the correct posterior probabilities recently a number of researchers have empirically demonstrated good performance of these same local propagation schemes on graphs with loops but a theoretical understanding of this performance has yet to be achieved for graphical models with a single loop we derive an analytical relationship between the probabilities computed using local propagation and the correct marginals using this relationship we show a category of graphical models with loops for which local propagation gives rise to provably optimal maximum a posteriori assignments although the computed marginals will be incorrect we also show how nodes can use local information in the messages they receive in order to correct their computed marginals we discuss how these results can be extended to graphical models with multiple loops and show simulation results suggesting that some properties of propagation on single loop graphs may hold for a larger class of graphs specifically we discuss the implication of our results for understanding a class of recently proposed error correcting codes known as turbo codes

vector approximate message passing the standard linear regression slr problem is to recover a vector mathrm x 0 from noisy linear observations mathrm y mathrm ax 0 mathrm w the approximate message passing amp algorithm proposed by donoho maleki and montanari is a computationally efficient iterative approach to slr that has a remarkable property for large i i d sub gaussian matrices a its per iteration behavior is rigorously characterized by a scalar state evolution whose fixed points when unique are bayes optimal the amp algorithm however is fragile in that even small deviations from the i i d sub gaussian model can cause the algorithm to diverge this paper considers a vector amp vamp algorithm and shows that vamp has a rigorous scalar state evolution that holds under a much broader class of large random matrices a those that are right orthogonally invariant after performing an initial singular value decomposition svd of a the per iteration complexity of vamp is similar to that of amp in addition the fixed points of vamp s state evolution are consistent with the replica prediction of the minimum mean squared error derived by tulino caire verdu and shamai numerical experiments are used to confirm the effectiveness of vamp and its consistency with state evolution predictions

accelerating cross validation in multinomial logistic regression with l 1 regularization we develop an approximate formula for evaluating a cross validation estimator of predictive likelihood for multinomial logistic regression regularized by an l1 norm this allows us to avoid repeated optimizations required for literally conducting cross validation hence the computational time can be significantly reduced the formula is derived through a perturbative approach employing the largeness of the data size and the model dimensionality an extension to the elastic net regularization is also addressed the usefulness of the approximate formula is demonstrated on simulated data and the isolet dataset from the uci machine learning repository matlab and python codes implementing the approximate formula are distributed in obuchi 2017 takahashi and obuchi 2017

on the uniqueness of loopy belief propagation fixed points we derive sufficient conditions for the uniqueness of loopy belief propagation fixed points these conditions depend on both the structure of the graph and the strength of the potentials and naturally extend those for convexity of the bethe free energy we compare them with a strengthened version of conditions derived elsewhere for pairwise potentials we discuss possible implications for convergent algorithms as well as for other approximate free energies

approximations for binary gaussian process classification we provide a comprehensive overview of many recent algorithms for approximate inference in gaussian process models for probabilistic binary classification the relationships between several approaches are elucidated theoretically and the properties of the different algorithms are corroborated by experimental results we examine both 1 the quality of the predictive distributions and 2 the suitability of the different marginal likelihood approximations for model selection selecting hyperparameters and compare to a gold standard based on mcmc interestingly some methods produce good predictive distributions although their marginal likelihood approximations are poor strong conclusions are drawn about the methods the expectation propagation algorithm is almost always the method of choice unless the computational budget is very tight we also extend existing methods in various ways and provide unifying code implementing all approaches

perturbative corrections for approximate inference in gaussian latent variable models expectation propagation ep provides a framework for approximate inference when the model under consideration is over a latent gaussian field with the approximation being gaussian we show how these approximations can systematically be corrected a perturbative expansion is made of the exact but intractable correction and can be applied to the model s partition function and other moments of interest the correction is expressed over the higher order cumulants which are neglected by ep s local matching of moments through the expansion we see that ep is correct to first order by considering higher orders corrections of increasing polynomial complexity can be applied to the approximation the second order provides a correction in quadratic time which we apply to an array of gaussian process and ising models the corrections generalize to arbitrarily complex approximating families which we illustrate on tree structured ising model approximations furthermore they provide a polynomial time assessment of the approximation error we also provide both theoretical and practical insights on the exactness of the ep solution

wavelet based image denoising using a markov random field a priori model this paper describes a new method for the suppression of noise in images via the wavelet transform the method relies on two measures the first is a classic measure of smoothness of the image and is based on an approximation of the local holder exponent via the wavelet coefficients the second novel measure takes into account geometrical constraints which are generally valid for natural images the smoothness measure and the constraints are combined in a bayesian probabilistic formulation and are implemented as a markov random field mrf image model the manipulation of the wavelet coefficients is consequently based on the obtained probabilities a comparison of quantitative and qualitative results for test images demonstrates the improved noise suppression performance with respect to previous wavelet based image denoising methods

constructing edge disjoint spanning trees in locally twisted cubes the use of edge disjoint spanning trees for data broadcasting and scattering problem in networks provides a number of advantages including the increase of bandwidth and fault tolerance in this paper we present an algorithm for constructing n edge disjoint spanning trees in an n dimensional locally twisted cube since the n dimensional locally twisted cube is regular with the common degree n the number of constructed trees is optimal

expectation consistent approximate inference we propose a novel framework for approximations to intractable probabilistic models which is based on a free energy formulation the approximation can be understood as replacing an average over the original intractable distribution with a tractable one it requires two tractable probability distributions which are made consistent on a set of moments and encode different features of the original intractable distribution in this way we are able to use gaussian approximations for models with discrete or bounded variables which allow us to include non trivial correlations these are neglected in many other methods we test the framework on toy benchmark problems for binary variables on fully connected graphs and 2d grids and compare with other methods such as loopy belief propagation good performance is already achieved by using single nodes as tractable substructures significant improvements are obtained when a spanning tree is used instead

inferring hidden states in langevin dynamics on large networks average case performance we present average performance results for dynamical inference problems in large networks where a set of nodes is hidden while the time trajectories of the others are observed examples of this scenario can occur in signal transduction and gene regulation networks we focus on the linear stochastic dynamics of continuous variables interacting via random gaussian couplings of generic symmetry we analyze the inference error given by the variance of the posterior distribution over hidden paths in the thermodynamic limit and as a function of the system parameters and the ratio ensuremath alpha between the number of hidden and observed nodes by applying kalman filter recursions we find that the posterior dynamics is governed by an effective drift that incorporates the effect of the observations we present two approaches for characterizing the posterior variance that allow us to tackle respectively equilibrium and nonequilibrium dynamics the first appeals to random matrix theory and reveals average spectral properties of the inference error and typical posterior relaxation times the second is based on dynamical functionals and yields the inference error as the solution of an algebraic equation

real nonparametric regression using complex wavelets summary wavelet shrinkage is an effective nonparametric regression technique especially when the underlying curve has irregular features such as spikes or discontinuities the basic idea is simple take the discrete wavelet transform dwt of data consisting of a signal corrupted by noise shrink or remove the wavelet coefficients to remove the noise and then invert the dwt to form an estimate of the true underlying curve various authors have proposed increasingly sophisticated methods of doing this using real valued wavelets complex valued wavelets exist but are rarely used we propose two new shrinkage techniques which use complex valued wavelets extensive simulations show that our methods almost always give significantly more accurate estimates when compared with methods based on real valued wavelets further one of our methods is both simpler and dramatically faster than its competitors in an attempt to understand the excellent performance of this latter method we present a new risk bound on its hard thresholded coefficients

embedding meshes into crossed cubes crossed cubes are important variants of hypercubes in this paper we consider embeddings of meshes in crossed cubes the major research findings in this paper are 1 for any integer n 1 a 2x2 n 1 mesh can be embedded in the n dimensional crossed cube with dilation 1 and expansion 1 2 for any integer n 4 two node disjoint 4x2 n 3 meshes can be embedded in the n dimensional crossed cube with dilation 1 and expansion 2 the obtained results are optimal in the sense that the dilations of the embeddings are 1 the embedding of the 2x2 n 1 mesh is also optimal in terms of expansion because it has the smallest expansion 1

variational inference a review for statisticians abstractone of the core problems of modern statistics is to approximate difficult to compute probability densities this problem is especially important in bayesian statistics which frames all inference about unknown quantities as a calculation involving the posterior density in this article we review variational inference vi a method from machine learning that approximates probability densities through optimization vi has been used in many applications and tends to be faster than classical methods such as markov chain monte carlo sampling the idea behind vi is to first posit a family of densities and then to find a member of that family which is close to the target density closeness is measured by kullback leibler divergence we review the ideas behind mean field variational inference discuss the special case of vi applied to exponential family models present a full example with a bayesian mixture of gaussians and derive a variant that uses stochastic optimization to scale up to massive data

approximate learning algorithm in boltzmann machines boltzmann machines can be regarded as markov random fields for binary cases they are equivalent to the ising spin model in statistical mechanics learning systems in boltzmann machines are one of the np hard problems thus in general we have to use approximate methods to construct practical learning algorithms in this context in this letter we propose new and practical learning algorithms for boltzmann machines by using the belief propagation algorithm and the linear response approximation which are often referred as advanced mean field methods finally we show the validity of our algorithm using numerical experiments

paths and cycles identifying vertices in twisted cubes the hypercube is one of the most popular interconnection networks since it has simple structure and is easy to implement the twisted cube is an important variation of the hypercube and preserves many of its desirable properties karpovsky et al introduced the concept of identifying codes to model fault detection in multiprocessor systems and honkala et al developed an identifying code by using cycles to identify the faulty processors in the hypercube in this paper we study the vertex identification problem on the twisted cube we first propose an interesting construction scheme to build paths and cycles and furthermore apply a minimum number of paths and cycles to identify the faulty processors of the twisted cube

on the universality of noiseless linear estimation with respect to the measurement matrix in a noiseless linear estimation problem one aims to reconstruct a vector x from the knowledge of its linear projections y x there have been many theoretical works concentrating on the case where the matrix is a random i i d one but a number of heuristic evidence suggests that many of these results are universal and extend well beyond this restricted case here we revisit this problematic through the prism of development of message passing methods and consider not only the universality of the l1 transition but also the one of the optimal bayesian reconstruction we observed that the universality extends to the bayes optimal minimum mean squared mmse error and to complicated matrices such as those appearing in randomly statistical retrieval problems

fixed points of belief propagation an analysis via polynomial homotopy continuation belief propagation bp is an iterative method to perform approximate inference on arbitrary graphical models whether bp converges and if the solution is a unique fixed point depends on both the structure and the parametrization of the model to understand this dependence it is interesting to find all fixed points in this work we formulate a set of polynomial equations the solutions of which correspond to bp fixed points to solve such a nonlinear system we present the numerical polynomial homotopy continuation nphc method experiments on binary ising models and on error correcting codes show how our method is capable of obtaining all bp fixed points on ising models with fixed parameters we show how the structure influences both the number of fixed points and the convergence properties we further asses the accuracy of the marginals and weighted combinations thereof weighting marginals with their respective partition function increases the accuracy in all experiments contrary to the conjecture that uniqueness of bp fixed points implies convergence we find graphs for which bp fails to converge even though a unique fixed point exists moreover we show that this fixed point gives a good approximation and the nphc method is able to obtain this fixed point

finite element formulation for a baffled fluid loaded finite cylindrical shell a new method for obtaining the response of a baffled fluid loaded finite cylindrical shell using the finite element method is presented the galerkin finite element formulation for this problem utilizes a neumann green s function representation of the pressure loading on the shell by representing the pressure loading in this manner only the structure need be discretized both c0 and c1 finite element discretizations of the shell are considered the finite element results are compared with the analytic solutions of the fluid loaded cylinder which are obtained using an expansion of the displacement in terms of the in vacuo modes of the naghdi cooper and flugge shell theories

sound radiation by baffled plates and related boundary integral equations abstract the displacement of the plate is described by its green s representation involving the infinite fluid loaded plate kernel the boundary conditions lead to a system of two integral equations along the plate boundary from a numerical point of view the method is efficient because this kernel is represented in terms of hankel functions and laplace type integrals which can be computed very fast numerical methods are described and an example is given

fitting time series models to nonstationary processes a general minimum distance estimation procedure is presented for nonstationary time series models that have an evolutionary spectral representation the asymptotic properties of the estimate are derived under the assumption of possible model misspecification for autoregressive processes with time varying coefficients the estimate is compared to the least squares estimate furthermore the behavior of estimates is explained when a stationary model is fitted to a nonstationary process

expectation propagation in the large data limit summary expectation propagation ep is a widely successful algorithm for variational inference ep is an iterative algorithm used to approximate complicated distributions typically to find a gaussian approximation of posterior distributions in many applications of this type ep performs extremely well surprisingly despite its widespread use there are very few theoretical guarantees on gaussian ep and it is quite poorly understood to analyse ep we first introduce a variant of ep averaged ep which operates on a smaller parameter space we then consider averaged ep and ep in the limit of infinite data where the overall contribution of each likelihood term is small and where posteriors are almost gaussian in this limit we prove that the iterations of both averaged ep and ep are simple they behave like iterations of newton s algorithm for finding the mode of a function we use this limit behaviour to prove that ep is asymptotically exact and to obtain other insights into the dynamic behaviour of ep e g that it may diverge under poor initialization exactly like newton s method ep is a simple algorithm to state but a difficult one to study our results should facilitate further research into the theoretical properties of this important method

an approximate analytical approach to resampling averages using a novel reformulation we develop a framework to compute approximate resampling data averages analytically the method avoids multiple retraining of statistical models on the samples our approach uses a combination of the replica trick of statistical physics and the tap approach for approximate bayesian inference we demonstrate our approach on regression with gaussian processes a comparison with averages obtained by monte carlo sampling shows that our method achieves good accuracy

extended plefka expansion for stochastic dynamics we propose an extension of the plefka expansion which is well known for the dynamics of discrete spins to stochastic differential equations with continuous degrees of freedom and exhibiting generic nonlinearities the scenario is sufficiently general to allow application to e g biochemical networks involved in metabolism and regulation the main feature of our approach is to constrain in the plefka expansion not just first moments akin to magnetizations but also second moments specifically two time correlations and responses for each degree of freedom the end result is an effective equation of motion for each single degree of freedom where couplings to other variables appear as a self coupling to the past i e memory term and a coloured noise this constitutes a new mean field approximation that should become exact in the thermodynamic limit of a large network for suitably long ranged couplings for the analytically tractable case of linear dynamics we establish this exactness explicitly by appeal to spectral methods of random matrix theory for gaussian couplings with arbitrary degree of symmetry

cccp algorithms to minimize the bethe and kikuchi free energies convergent alternatives to belief propagation this article introduces a class of discrete iterative algorithms that are provably convergent alternatives to belief propagation bp and generalized belief propagation gbp our work builds on recent results by yedidia freeman and weiss 2000 who showed that the fixed points of bp and gbp algorithms correspond to extrema of the bethe and kikuchi free energies respectively we obtain two algorithms by applying cccp to the bethe and kikuchi free energies respectively cccp is a procedure introduced here for obtaining discrete iterative algorithms by decomposing a cost function into a concave and a convex part we implement our cccp algorithms on two and three dimensional spin glasses and compare their results to bp and gbp our simulations show that the cccp algorithms are stable and converge very quickly the speed of cccp is similar to that of bp and gbp unlike cccp bp will often not converge for these problems gbp usually but not always converges the results found by cccp applied to the bethe or kikuchi free energies are equivalent or slightly better than those found by bp or gbp respectively when bp and gbp converge note that for these and other problems bp and gbp give very accurate results see yedidia et al 2000 and failure to converge is their major error mode finally we point out that our algorithms have a large range of inference and learning applications

sharp adaptation for inverse problems with random noise we consider a heteroscedastic sequence space setup with polynomially increasing variances of observations that allows to treat a number of inverse problems in particular multivariate ones we propose an adaptive estimator that attains simultaneously exact asymptotic minimax constants on every ellipsoid of functions within a wide scale that includes ellipoids with polynomially and exponentially decreasing axes and at the same time satisfies asymptotically exact oracle inequalities within any class of linear estimates having monotone non increasing weights the construction of the estimator is based on a properly penalized blockwise stein s rule with weakly geometically increasing blocks as an application we construct sharp adaptive estimators in the problems of deconvolution and tomography

emulation of higher order tensors in manifold monte carlo methods for bayesian inverse problems the bayesian approach to inverse problems relies predominantly on markov chain monte carlo methods for posterior inference the typical nonlinear concentration of posterior measure observed in many such inverse problems presents severe challenges to existing simulation based inference methods motivated by these challenges the exploitation of local geometric information in the form of covariant gradients metric tensors levi civita connections and local geodesic flows have been introduced to more effectively locally explore the configuration space of the posterior measure however obtaining such geometric quantities usually requires extensive computational effort and despite their effectiveness affects the applicability of these geometrically based monte carlo methods in this paper we explore one way to address this issue by the construction of an emulator of the model from which all geometric objects can be obtained in a much more computationally feasible manner the main concept is to approximate the geometric quantities using a gaussian process emulator which is conditioned on a carefully chosen design set of configuration points which also determines the quality of the emulator to this end we propose the use of statistical experiment design methods to refine a potentially arbitrarily initialized design online without destroying the convergence of the resulting markov chain to the desired invariant measure the practical examples considered in this paper provide a demonstration of the significant improvement possible in terms of computational loading suggesting this is a promising avenue of further development

accuracy of the bethe approximation for hyperparameter estimation in probabilistic image processing we investigate the accuracy of statistical mechanical approximations for the estimation of hyperparameters from observable data in probabilistic image processing which is based on bayesian statistics and maximum likelihood estimation hyperparameters in statistical science correspond to interactions or external fields in the statistical mechanics context in this paper hyperparameters in the probabilistic model are determined so as to maximize a marginal likelihood a practical algorithm is described for grey level image restoration based on a gaussian graphical model and the bethe approximation the algorithm corresponds to loopy belief propagation in artificial intelligence we examine the accuracy of hyperparameter estimation when we use the bethe approximation it is well known that a practical algorithm for probabilistic image processing can be prescribed analytically when a gaussian graphical model is adopted as a prior probabilistic model in bayes formula we are therefore able to compare in a numerical study results obtained through mean field type approximations with those based on exact calculation

wavelet processes and adaptive estimation of the evolutionary wavelet spectrum this article defines and studies a new class of non stationary random processes constructed from discrete non decimated wavelets which generalizes the cramer fourier representation of stationary time series we define an evolutionary wavelet spectrum ews which quantifies how process power varies locally over time and scale we show how the ews may be rigorously estimated by a smoothed wavelet periodogram and how both these quantities may be inverted to provide an estimable time localized autocovariance we illustrate our theory with a pedagogical example based on discrete non decimated haar wavelets and also a real medical time series example

perturbation corrections in approximate inference mixture modelling applications bayesian inference is intractable for many interesting models making deterministic algorithms for approximate inference highly desirable unlike stochastic methods which are exact in the limit the accuracy of these approaches cannot be reasonably judged in this paper we show how low order perturbation corrections to an expectation consistent ec approximation can provide the necessary tools to ameliorate inference accuracy and to give an indication of the quality of approximation without having to resort to monte carlo methods further comparisons are given with variational bayes and parallel tempering pt combined with thermodynamic integration on a gaussian mixture model to obtain practical results we further generalize pt to temper from arbitrary distributions rather than a prior in bayesian inference

simple method for inference in inverse ising problem using full data abstract we consider inference in the inverse ising problem using full data which means incorporating sets of spin configurations we approximate the boltzmann distribution of the system to generate a frequency distribution derived from the given data then the ratio between two boltzmann distributions with different spin configurations eliminates the partition function and we obtain linear equations which can be solved to yield statistical parameters our method is applicable to cases where the absolute values of the coupling parameters and external fields are large compared to pseudolikelihood maximization the accuracy of the inference obtained from our method is similar although our approach is less labor intensive

adaptive and self averaging thouless anderson palmer mean field theory for probabilistic modeling we develop a generalization of the thouless anderson palmer tap mean field approach of disorder physics which makes the method applicable to the computation of approximate averages in probabilistic models for real data in contrast to the conventional tap approach where the knowledge of the distribution of couplings between the random variables is required our method adapts to the concrete set of couplings we show the significance of the approach in two ways our approach reproduces replica symmetric results for a wide class of toy models assuming a nonglassy phase with given disorder distributions in the thermodynamic limit on the other hand simulations on a real data model demonstrate that the method achieves more accurate predictions as compared to conventional tap approaches

mean field approaches to independent component analysis we develop mean field approaches for probabilistic independent component analysis ica the sources are estimated from the mean of their posterior distribution and the mixing matrix and noise level is estimated by maximum a posteriori map the latter requires the computation of a good approximation to the correlations between sources for this purpose we investigate three increasingly advanced mean field methods the variational also known as naive mean field approach linear response corrections and an adaptive version of the thouless anderson and palmer 1977 tap mean field approach which is due to opper and winther 2001 the resulting algorithms are tested on a number of problems on synthetic data the advanced mean field approaches are able to recover the correct mixing matrix in cases where the variational mean field theory fails for handwritten digits sparse encoding is achieved using nonnegative source and mixing priors for speech the mean field method is able to separate in the underdetermined overcomplete case of two sensors and three sources one major advantage of the proposed method is its generality and algorithmic simplicity finally we point out several possible extensions of the approaches developed here

embedding a family of disjoint 3d meshes into a crossed cube crossed cubes are an important class of hypercube variants this paper addresses how to embed a family of disjoint 3d meshes into a crossed cube two major contributions of this paper are 1 for n 4 a family of two disjoint 3d meshes of size 2x2x2 n 3 can be embedded in an n d crossed cube with unit dilation and unit expansion and 2 for n 6 a family of four disjoint 3d meshes of size 4x2x2 n 5 can be embedded in an n d crossed cube with unit dilation and unit expansion these results mean that a family of two or four 3d mesh structured parallel algorithms can be executed on a same crossed cube efficiently and in parallel our work extends the results recently obtained by fan and jia j fan x jia embedding meshes into crossed cubes information sciences 177 15 2007 3151 3160

harmonic decomposition of audio signals with matching pursuit we introduce a dictionary of elementary waveforms called harmonic atoms that extends the gabor dictionary and fits well the natural harmonic structures of audio signals by modifying the standard matching pursuit we define a new pursuit along with a fast algorithm namely the fast harmonic matching pursuit to approximate n dimensional audio signals with a linear combination of m harmonic atoms our algorithm has a computational complexity of o mkn where k is the number of partials in a given harmonic atom the decomposition method is demonstrated on musical recordings and we describe a simple note detection algorithm that shows how one could use a harmonic matching pursuit to detect notes even in difficult situations e g very different note durations lots of reverberation and overlapping notes

multiscale image segmentation using wavelet domain hidden markov models we introduce a new image texture segmentation algorithm hmtseg based on wavelets and the hidden markov tree hmt model the hmt is a tree structured probabilistic graph that captures the statistical properties of the coefficients of the wavelet transform since the hmt is particularly well suited to images containing singularities edges and ridges it provides a good classifier for distinguishing between textures utilizing the inherent tree structure of the wavelet hmt and its fast training and likelihood computation algorithms we perform texture classification at a range of different scales we then fuse these multiscale classifications using a bayesian probabilistic graph to obtain reliable final segmentations since hmtseg works on the wavelet transform of the image it can directly segment wavelet compressed images without the need for decompression into the space domain we demonstrate the performance of hmtseg with synthetic aerial photo and document image segmentations

convergence condition of the tap equation for the infinite ranged ising spin glass model it is shown that the power expansion of the gibbs potential of the sk model up to second order in the exchange couplings leads to the tap equation this result remains valid for the general including a ferromagnetic exchange sk model theorems of power expansions and resolvent techniques are employed to solve the convergence problem the convergence condition is presented for the whole temperature range and for general distributions of the local magnetisations

on learning vector valued functions in this letter we provide a study of learning in a hilbert space of vectorvalued functions we motivate the need for extending learning theory of scalar valued functions by practical considerations and establish some basic results for learning vector valued functions that should prove useful in applications specifically we allow an output space y to be a hilbert space and we consider a reproducing kernel hilbert space of functions whose values lie in y in this setting we derive the form of the minimal norm interpolant to a finite set of data and apply it to study some regularization functionals that are important in learning theory we consider specific examples of such functionals corresponding to multiple output regularization networks and support vector machines for both regression and classification finally we provide classes of operator valued kernels of the dot product and translation invariant type

expectation propagation for nonlinear inverse problems with an application to electrical impedance tomography in this paper we study a fast approximate inference method based on expectation propagation for exploring the posterior probability distribution arising from the bayesian formulation of nonlinear inverse problems it is capable of efficiently delivering reliable estimates of the posterior mean and covariance thereby providing an inverse solution together with quantified uncertainties some theoretical properties of the iterative algorithm are discussed and the efficient implementation for an important class of problems of projection type is described the method is illustrated with one typical nonlinear inverse problem electrical impedance tomography with complete electrode model under sparsity constraints numerical results for real experimental data are presented and compared with that by markov chain monte carlo the results indicate that the method is accurate and computationally very efficient

sufficient conditions for convergence of the sum product algorithm novel conditions are derived that guarantee convergence of the sum product algorithm also known as loopy belief propagation or simply belief propagation bp to a unique fixed point irrespective of the initial messages for parallel synchronous updates the computational complexity of the conditions is polynomial in the number of variables in contrast with previously existing conditions our results are directly applicable to arbitrary factor graphs with discrete variables and are shown to be valid also in the case of factors containing zeros under some additional conditions the conditions are compared with existing ones numerically and if possible analytically for binary variables with pairwise interactions sufficient conditions are derived that take into account local evidence i e single variable factors and the type of pair interactions attractive or repulsive it is shown empirically that this bound outperforms existing bounds

the radiation of sound from a finite ring forced cylindrical elastic shell i wiener hopf analysis the radiation of sound from a finite fluid loaded cylindrical elastic shell is examined by means of the modified wiener hopf technique the shell is freely submerged in an inviscid compressible stationary fluid of infinite extent and axisymmetrically excited by a ring force the problem is formulated as a three part wiener hopf equation with three unknown functions and is recast as two uncoupled fredholm integral equations of the second kind by utilizing a factorization and decomposition procedure when the product of the cylinder length l and the wave number k is large asymptotic analysis can be applied to the integral equations which reduces them to a finite system of algebraic equations corresponding to a set of unknown constants including the edge displacements the method provides a straightforward formulation for the solution and is valid for arbitrary fluid loading numerical results are presented for the radiated far field velocity potential this requires a numerical decomposition of the wiener hopf kernel and the solution of the set of simultaneous equations