relative entropy as a universal metric for multiscale errors we show that the relative entropy s text rel suggests a fundamental indicator of the success of multiscale studies in which coarse grained cg models are linked to first principles fp ones we demonstrate that s text rel inherently measures fluctuations in the differences between cg and fp potential energy landscapes and develop a theory that tightly and generally links it to errors associated with coarse graining we consider two simple case studies substantiating these results and suggest that s text rel has important ramifications for evaluating and designing coarse grained models

stochastic variational inference we develop stochastic variational inference a scalable algorithm for approximating posterior distributions we develop this technique for a large class of probabilistic models and we demonstrate it with two probabilistic topic models latent dirichlet allocation and the hierarchical dirichlet process topic model using stochastic variational inference we analyze several large collections of documents 300k articles from nature 1 8m articles from the new york times and 3 8m articles from wikipedia stochastic inference can easily handle data sets of this size and outperforms traditional variational inference which can only handle a smaller subset we also show that the bayesian nonparametric topic model outperforms its parametric counterpart stochastic variational inference lets us apply complex bayesian models to massive data sets

the zeroes of the partition function of the random energy model abstract the random energy model is one of the simplest disordered models containing some of the physics of spin glasses the zeroes of the partition function in the complex temperature plane are determined for this model they either lie on simple lines or are dense in some regions of the complex plane

coarse graining schemes for stochastic lattice systems with short and long range interactions we develop coarse graining schemes for stochastic many particle microscopic models with competing short and long range interactions on a dimensional lattice we focus on the coarse graining of equilibrium gibbs states and by using cluster expansions we analyze the corresponding renormalization group map we quantify the approximation properties of the coarse grained terms arising from different types of interactions and present a hierarchy of correction terms we derive semi analytical numerical coarse graining schemes that are accompanied by a posteriori error estimates for lattice systems with short and long range interactions

statistical mechanics of conducting phase transitions the critical behavior of the effective conductivity of the random resistor network in zd near its percolation threshold is considered the network has bonds assigned the conductivities 1 and e 0 in the volume fractions p and 1 p motivated by the statistical mechanics of an ising ferromagnet at temperature t in a field h we introduce a partition function and free energy for the resistor network which establishes a direct correspondence between the two problems in particular we show that the free energies for the resistor network and the ising model both have the same type of integral representation which has the interpretation of the complex potential due to a charge distribution on 0 1 in the s 1 1 e plane for the resistor network and on the unit circle in the z exp 2 h plane for the ferromagnet based on this correspondence we develop a yang lee picture of the onset of nonanalytic behavior of the effective conductivity so that the percolation threshold p pc is characterized as a

an estimator for the relative entropy rate of path measures for stochastic differential equations we address the problem of estimating the relative entropy rate rer for two stochastic processes described by stochastic differential equations for the case where the drift of one process is known analytically but one has only observations from the second process we use a variational bound on the rer to construct an estimator

language based performance prediction for distributed and mobile systems we present a framework for performance prediction of distributed and mobile systems we rely on process calculi and their structural operational semantics the dynamic behaviour is described through transition systems whose transitions are labelled by encodings of their proofs that we then map into stochastic processes we enhance related works by allowing general continuous distributions resorting to a notion of enabling between transitions we also discuss how the number of resources available affects the overall model finally we introduce a notion of bisimulation that takes stochastic information into account and prove it to be a congruence when only exponential distributions are of interest our equivalence induces a lumpable partition on the underlying markov process

path space variational inference for non equilibrium coarse grained systems in this paper we discuss information theoretic tools for obtaining optimized coarse grained molecular models for both equilibrium and non equilibrium molecular simulations the latter are ubiquitous in physicochemical and biological applications where they are typically associated with coupling mechanisms multi physics and or boundary conditions in general the non equilibrium steady states are not known explicitly as they do not necessarily have a gibbs structure the presented approach can compare microscopic behavior of molecular systems to parametric and non parametric coarse grained models using the relative entropy between distributions on the path space and setting up a corresponding path space variational inference problem the methods can become entirely data driven when the microscopic dynamics are replaced with corresponding correlated data in the form of time series furthermore we present connections and generalizations of force matching methods in coarse graining with path space information methods we demonstrate the enhanced transferability of information based parameterizations to different observables at a specific thermodynamic point due to information inequalities we discuss methodological connections between information based coarse graining of molecular systems and variational inference methods primarily developed in the machine learning community however we note that the work presented here addresses variational inference for correlated time series due to the focus on dynamics the applicability of the proposed methods is demonstrated on high dimensional stochastic processes given by overdamped and driven langevin dynamics of interacting particles

spatial two level interacting particle simulations and information theory based error quantification we propose a hierarchy of two level kinetic monte carlo methods for sampling high dimensional stochastic lattice particle dynamics with complex interactions the method is based on the efficient coupling of different spatial resolution levels taking advantage of the low sampling cost in a coarse space and developing local reconstruction strategies from coarse grained dynamics furthermore a natural extension to a multilevel kinetic coarse grained monte carlo is presented microscopic reconstruction corrects possibly significant errors introduced through coarse graining leading to the controlled error approximation of the sampled stochastic process in this manner the proposed algorithm overcomes known shortcomings of coarse graining of particle systems with complex interactions such as combined long and short range particle interactions and or complex lattice geometries specifically we provide error analysis for the approximation of long time stationary dynamics in terms of relative entropy measu

some congruence properties for p calculus bisimilarities both for interleaving and for non interleaving semantics several variants of a pi calculus bisimilarity can be given which differ on the requirements imposed on name instantiations examples are the em late early open and em ground variants the ground variant is the simplest because it places no requirements on name instantiations with the exception of open bisimilarities none of the bisimilarity considered in the literature is a congruence relation on the full pi calculus language we show that in the case of certain forms of the non interleaving em causal bisimulation the late early open and ground variants coincide and are congruence relations in the sublanguage of the pi calculus without matching we also show that to obtain the same results in the case of the interleaving bisimilarity in addition to forbidding matching it is necessary to constrain the output prefix

natural gradient works efficiently in learning when a parameter space has a certain underlying structure the ordinary gradient of a function does not represent its steepest direction but the natural gradient does information geometry is used for calculating the natural gradients in the parameter space of perceptrons the space of matrices for blind source separation and the space of linear dynamical systems for blind source deconvolution the dynamical behavior of natural gradient online learning is analyzed and is proved to be fisher efficient implying that it has asymptotically the same performance as the optimal batch estimation of parameters this suggests that the plateau phenomenon which appears in the backpropagation learning algorithm of multilayer perceptrons might disappear or might not be so serious when the natural gradient is used an adaptive method of updating the learning rate is proposed and analyzed

the effective conductivity of strongly heterogeneous composites we deal with the effective conductivity m m z of two phase ordered or disordered mixtures consisting of particles of material of conductivity z inserted in a matrix of conductivity 1 we focus on finding bounds on the set of values of z for which the function m is singular or vanishes and we apply our results to the estimation of the effective conductivity of high contrast mixtures z 0 or z we find that the zeroes and singularities of the function m lie on an interval of the negative real axis which depends on the shape of the particles and the interparticle distances our results agree with previous numerical calculations for periodic arrays of spheres in some cases we show that our estimates are optimal we apply our results about the zeroes and singularities together with the complex variable method and find bounds on the effective conductivity of matrix particle random composites these bounds give good estimations even in cases of high contrast and in many cases they improve substantially over the bounds obtained by other methods for the same types of high contrast mixtures

scalable information inequalities for uncertainty quantification abstract in this paper we demonstrate the only available scalable information bounds for quantities of interest of high dimensional probabilistic models scalability of inequalities allows us to a obtain uncertainty quantification bounds for quantities of interest in the large degree of freedom limit and or at long time regimes b assess the impact of large model perturbations as in nonlinear response regimes in statistical mechanics c address model form uncertainty i e compare different extended models and corresponding quantities of interest we demonstrate some of these properties by deriving robust uncertainty quantification bounds for phase diagrams in statistical mechanics models

probabilistic and nondeterministic aspects of anonymity the concept of anonymity comes into play in a wide range of situations varying from voting and anonymous donations to postings on bulletin boards and sending emails the protocols for ensuring anonymity often use random mechanisms which can be described probabilistically while the agents behavior may be totally unpredictable irregular and hence expressible only nondeterministically formal definitions of the concept of anonymity have been investigated in the past either in a totally nondeterministic framework or in a purely probabilistic one in this paper we investigate a notion of anonymity which combines both probability and nondeterminism and which is suitable for describing the most general situation in which the protocol and the users can have both probabilistic and nondeterministic behavior we also investigate the properties of the definition for the particular cases of purely nondeterministic users and purely probabilistic users we formulate the notions of anonymity in terms of probabilistic automata and we describe protocols and users as processes in the probabilistic p calculus whose semantics is again based on probabilistic automata throughout the paper we illustrate our ideas by using the example of the dining cryptographers

p calculus internal mobility and agent passing calculi abstract the calculus is a process algebra which originates from ccs and permits a natural modelling of mobility i e dynamic reconfigurations of the process linkage using communication of names previous research has shown that the calculus has a much greater expressiveness than ccs but it also has a much more complex mathematical theory the primary goal of this work is to understand the reasons for this gap another goal is to compare the expressiveness of name passing calculi i e calculi like calculus where mobility is achieved via exchange of names and that of agent passing calculi i e calculi where mobility is achieved via exchange of agents we divide the mobility mechanisms of the calculus into internal and external mobility mechanisms the study of the subcalculus which only uses internal mobility called i suggests that internal mobility is responsible for much of the expressiveness of the calculus whereas external mobility is responsible for many of the semantic complications a pleasant property of i is the full symmetry between input and output constructs internal mobility is strongly related to agent passing mobility by imposing bounds on the order of the types of i and of the higher order calculus sangiorgi 1992 we define a hierarchy of name passing calculi based on internal mobility and one of agent passing calculi we show that there is an exact correspondence in terms of expressiveness between the two hierarchies

tau laws for pi calculus the paper investigates the nonsymbolic algebraic semantics of the weak bisimulation congruences on finite pi processes the weak bisimulation congruences are studied both in the absence and in the presence of the mismatch operator some interesting phenomena about the open congruences are revealed several new tau laws are discovered and their relationship is discussed the contributions of the paper are mainly as follows 1 it is proved that milner s three tau laws fail to lift a complete system for the strong open congruence to a complete system for the weak open congruence in the absence of both the mismatch operator and the restriction operator a fourth tau law is proposed to deal with the match operator under the prefix operation it is shown that for this calculus a complete system for the strong open congruence extended with all the four tau laws is complete for the weak open congruence 2 it is verified that the four tau laws are also enough for the weak open congruence of the pi calculus without the mismatch operator a complete system using distinctions is given 3 it is pointed out that the standard definition of the weak open congruence gives rise to a bad equivalence relation in the presence of the mismatch operator two alternatives are proposed these are the late open congruence and the early open congruence their difference is similar to that between the weak late congruence and the weak early congruence complete axiomatic systems for the two weak open congruences are given

wavelets as basis functions to represent the coarse graining potential in multiscale coarse graining approach in this paper we apply multiresolution analysis mra to develop sparse but accurate representations for the multiscale coarse graining mscg approximation to the many body potential of mean force we rigorously framed the mscg method into mra so that all the instruments of this theory become available together with a multitude of new basis functions namely the wavelets the coarse grained cg force field is hierarchically decomposed at different resolution levels enabling to choose the most appropriate wavelet family for each physical interaction without requiring an a priori knowledge of the details localization the representation of the cg potential in this new efficient orthonormal basis leads to a compression of the signal information in few large expansion coefficients the multiresolution property of the wavelet transform allows to isolate and remove the noise from the cg force field reconstruction by thresholding the basis function coefficients from each frequency band independently we discuss the implementation of our wavelet based mscg approach and demonstrate its accuracy using two different condensed phase systems i e liquid water and methanol simulations of liquid argon have also been performed using a one to one mapping between atomistic and cg sites the latter model allows to verify the accuracy of the method and to test different choices of wavelet families furthermore the results of the computer simulations show that the efficiency and sparsity of the representation of the cg force field can be traced back to the mathematical properties of the chosen family of wavelets this result is in agreement with what is known from the theory of multiresolution analysis of signals

a new framework for extracting coarse grained models from time series with multiscale structure in many applications it is desirable to infer coarse grained models from observational data the observed process often corresponds only to a few selected degrees of freedom of a high dimensional dynamical system with multiple time scales in this work we consider the inference problem of identifying an appropriate coarse grained model from a single time series of a multiscale system it is known that estimators such as the maximum likelihood estimator or the quadratic variation of the path estimator can be strongly biased in this setting here we present a novel parametric inference methodology for problems with linear parameter dependency that does not suffer from this drawback furthermore we demonstrate through a wide spectrum of examples that our methodology can be used to derive appropriate coarse grained models from time series of partial observations of a multiscale system in an effective and systematic fashion

error analysis of coarse graining for stochastic lattice dynamics the coarse grained monte carlo cgmc algorithm was originally proposed in the series of works m a katsoulakis a j majda and d g vlachos j comput phys 186 2003 pp 250 278 m a katsoulakis a j majda and d g vlachos proc natl acad sci usa 100 2003 pp 782 787 m a katsoulakis and d g vlachos j chem phys 119 2003 pp 9412 9427 in this paper we further investigate the approximation properties of the coarse graining procedure and provide both analytical and numerical evidence that the hierarchy of the coarse models is built in a systematic way that allows for error control in both transient and long time simulations we demonstrate that the numerical accuracy of the cgmc algorithm as an approximation of stochastic lattice spin flip dynamics is of order two in terms of the coarse graining ratio and that the natural small parameter is the coarse graining ratio over the range of particle particle interactions the error estimate is shown to hold in the weak convergence sense we employ the derived analytical results to guide cgmc algorithms and demonstrate a cpu speed up in demanding computational regimes that involve nucleation phase transitions and metastability

from mesoscale back to microscale reconstruction schemes for coarse grained stochastic lattice systems starting from a microscopic stochastic lattice spin system and the corresponding coarse grained model we introduce a mathematical strategy to recover microscopic information given the coarse grained data we define reconstructed microscopic measures satisfying two conditions i they are close in specific relative entropy to the initial microscopic equilibrium measure conditioned on the coarse grained data and ii their sampling is computationally advantageous when compared to sampling directly from the conditioned microscopic equilibrium measure by using different techniques we consider the cases of both short and long range microscopic models

strong normalisation in the π calculus we introduce a typed calculus where strong normalisation is ensured by typability strong normalisation is a useful property in many computational contexts including distributed systems in spite of its simplicity our type discipline captures a wide class of converging name passing interactive behaviour the proof of strong normalisability combines methods from typed calculi and linear logic with process theoretic reasoning it is adaptable to systems involving state non determinism polymorphism control and other extensions strong normalisation is shown to have significant consequences including finite axiomatisation of weak bisimilarity a fully abstract embedding of the simply typed calculus with products and sums and basic liveness in interaction strong normalisability has been extensively studied as a fundamental property in functional calculi term rewriting and logical systems this work is one of the first steps to extend theories and proof methods for strong normalisability to the context of name passing processes