on the computational complexity of ising spin glass models in a spin glass with ising spins the problems of computing the magnetic partition function and finding a ground state are studied in a finite two dimensional lattice these problems can be solved by algorithms that require a number of steps bounded by a polynomial function of the size of the lattice in contrast to this fact the same problems are shown to belong to the class of np hard problems both in the two dimensional case within a magnetic field and in the three dimensional case np hardness of a problem suggests that it is very unlikely that a polynomial algorithm could exist to solve it

shape from recognition a novel approach for 3 d face shape recovery in this paper we develop a novel framework for robust recovery of three dimensional 3 d surfaces of faces from single images the underlying principle is shape from recognition i e the idea that pre recognizing face parts can constrain the space of possible solutions to the image irradiance equation thus allowing robust recovery of the 3 d structure of a specific part parts of faces like nose lips and eyes are recognized and localized using robust expansion matching filter templates under varying pose and illumination specialized backpropagation based neural networks are then employed to recover the 3 d shape of particular face parts representation using principal components allows to efficiently encode classes of objects such as nose lips etc the specialized networks are designed and trained to map the principal component coefficients of the part images to another set of principal component coefficients that represent the corresponding 3 d surface shapes to achieve robustness to viewing conditions the network is trained with a wide range of illumination and viewing directions a method for merging recovered 3 d surface regions by minimizing the sum squared error in overlapping areas is also derived quantitative analysis of the reconstruction of the surface parts in varying illumination and pose show relatively small errors indicating that the method is robust and accurate several examples showing recovery of the complete face also illustrate the efficacy of the approach

improving variational methods via pairwise linear response identities inference methods are often formulated as variational approximations these approximations allow easy evaluation of statistics by marginalization or linear response but these estimates can be inconsistent we show that by introducing constraints on covariance one can ensure consistency of linear response with the variational parameters and in so doing inference of marginal probability distributions is improved for the bethe approximation and its generalizations improvements are achieved with simple choices of the constraints the approximations are presented as variational frameworks iterative procedures related to message passing are provided for finding the minima

on the macwilliams identity for classical and quantum convolutional codes the weight generating functions associated with convolutional codes ccs are based on state space realizations or the weight adjacency matrices wams the macwilliams identity for ccs on the wams was first established by gluesing luerssen and schneider in the case of minimal encoders and generalized by forney we consider this problem in the viewpoint of constraint codes and obtain a simple and direct proof of this macwilliams identity in the case of minimal encoders for our purpose we choose a different representation for the exact weight generating function ewgf of a block code by defining it as a linear combination of orthonormal vectors in dirac bra ket notation this representation provides great flexibility so that general split weight generating functions and their macwilliams identities can be easily obtained from the macwilliams identity for ewgfs as a result we also obtain the macwilliams identity for the input parity wams of a systematic convolutional code and its dual finally paralleling the development of the classical case we establish the macwilliams identity for quantum ccs

on the rapid computation of various polylogarithmic constants we give algorithms for the computation of the d th digit of certain transcendental numbers in various bases these algorithms can be easily implemented multiple precision arithmetic is not needed require virtually no memory and feature run times that scale nearly linearly with the order of the digit desired they make it feasible to compute for example the billionth binary digit of log 2 or on a modest work station in a few hours run time

belief propagation algorithm and the ising model on networks with arbitrary distributions of motifs we generalize the belief propagation algorithm to sparse random networks with arbitrary distributions of motifs triangles loops etc each vertex in these networks belongs to a given set of motifs generalization of the configuration model these networks can be treated as sparse uncorrelated hypergraphs in which hyperedges represent motifs here a hypergraph is a generalization of a graph where a hyperedge can connect any number of vertices these uncorrelated hypergraphs are treelike hypertrees which crucially simplifies the problem and allows us to apply the belief propagation algorithm to these loopy networks with arbitrary motifs as natural examples we consider motifs in the form of finite loops and cliques we apply the belief propagation algorithm to the ferromagnetic ising model with pairwise interactions on the resulting random networks and obtain an exact solution of this model we find an exact critical temperature of the ferromagnetic phase transition and demonstrate that with increasing the clustering coefficient and the loop size the critical temperature increases compared to ordinary treelike complex networks however weak clustering does not change the critical behavior qualitatively our solution also gives the birth point of the giant connected component in these loopy networks

surface topography using shape from shading abstract this paper demonstrates how a recently reported shape from shading scheme can be used to extract topographic information from 2d intensity imagery worthington and hancock ieee trans pattern anal mach intell 21 1999 1250 1267 the shape from shading scheme has two important features which enhance the recovered surface description firstly it uses a geometric update procedure which allows the image irradiance equation to be satisfied as a hard constraint this not only improves the data closeness of the recovered needle map but also removes the necessity for extensive parameter tuning secondly we use curvature information to impose topographic constraints on the recovered needle map the topographic information is captured using the shape index of koenderink and van doorn image vision comput 10 1992 557 565 and consistency is imposed using a robust error function we show that the new shape from shading scheme leads to a meaningful topographic labelling of 3d surface structures moreover the resulting topographic information proves to be useful in a simple histogram based object recognition scheme

loop calculus in statistical physics and information science considering a discrete and finite statistical model of a general position we introduce an exact expression for the partition function in terms of a finite series the leading term in the series is the bethe peierls belief propagation bp contribution the rest are expressed as loop contributions on the factor graph and calculated directly using the bp solution the series unveils a small parameter that often makes the bp approximation so successful applications of the loop calculus in statistical physics and information science are discussed

belief propagation and loop calculus for the permanent of a non negative matrix we consider computation of the permanent of a positive n n non negative matrix p pji i j 1 n or equivalently the problem of weighted counting of the perfect matchings over the complete bipartite graph kn n the problem is known to be of likely exponential complexity stated as the partition function z of a graphical model the problem allows for exact loop calculus representation chertkov m and chernyak v 2006 phys rev e 72 065102 in terms of an interior minimum of the bethe free energy functional over non integer doubly stochastic matrix of marginal beliefs ji i j 1 n also correspondent to a fixed point of the iterative message passing algorithm of the belief propagation bp type our main result is an explicit expression of the exact partition function permanent in terms of the matrix of bp marginals as z perm p zbp perm ji 1 ji i j 1 ji where zbp is the bp expression for the permanent stated explicitly in terms of we give two derivations of the formula a direct one based on the bethe free energy and an alternative one combining the ihara graph function and the loop calculus approaches assuming that the matrix of the bp marginals is calculated we provide two lower bounds and one upper bound to estimate the multiplicative term two complementary lower bounds are based on the gurvits van der waerden theorem and on a relation between the modified permanent and determinant respectively

distances and riemannian metrics for multivariate spectral densities the paper is concerned with the practical problem of how to compare power spectral densities of multivariable time series for scalar time series several notions of distance divergences have been proposed and studied starting from the early 1970s while multivariable ones have only recently began to receive any attention in the paper two classes of divergence measures inspired by classical prediction theory are introduced these divergences naturally induce riemannian metrics on the cone of multivariable densities the metrics amount to the quadratic term in the divergence between infinitesimally close to each other power spectra for one of the two we provide explicit formulae for the corresponding geodesics and geodesic distance a close connection between the geometry of power spectra and the geometry of the fisher rao metric is noted

factor graphs for quantum probabilities a factor graph representation of quantum mechanical probabilities involving any number of measurements is proposed unlike standard statistical models the proposed representation uses auxiliary variables state variables that are not random variables all joint probability distributions are marginals of some complex valued function q and it is demonstrated how the basic concepts of quantum mechanics relate to factorizations and marginals of q

metrics for power spectra an axiomatic approach we present an axiomatic framework for seeking distances between power spectral density functions the axioms require that the sought metric respects the effects of additive and multiplicative noise in reducing our ability to discriminate spectra as well as they require continuity of statistical quantities with respect to perturbations measured in the metric we then present a particular metric which abides by these requirements the metric is based on the monge kantorovich transportation problem and is contrasted with an earlier riemannian metric based on the minimum variance prediction geometry of the underlying time series it is also being compared with the more traditional itakura saito distance measure as well as the aforementioned prediction metric on two representative examples

quasi cyclic unit memory convolutional codes unit memory convolutional codes with generator matrices which are composed of circulant submatrices are introduced this structure facilitates the analysis of efficient search for good codes equivalences among such codes and some of the basic structural properties are discussed in particular catastrophic encoders and minimal encoders are characterized and dual codes treated further various distance measures are discussed and a number of good codes some of which result from efficient computer search and some of which result from known block codes are presented

the maximum entropy principle in the absence of a time arrow fractional pole models the maximum entropy me principle as it is often invoked in the context of time series analysis suggests the selection of a power spectrum which is consistent with autocorrelation data and corresponds to a random process least predictable from past observations we introduce and compare a class of spectra with the property that the underlying random process is least predictable at any given point from the complete set of past and future observations in this context randomness is quantified by the size of the corresponding smoothing error and deterministic processes are characterized by integrability of the inverse of their power spectral densities as opposed to the log integrability in the classical setting the power spectrum which is consistent with a partial autocorrelation sequence and corresponds to the most random mr process in this new sense is no longer rational but generated by finitely many fractional poles

statistical approach to shape from shading reconstruction of three dimensional face surfaces from single two dimensional images the human visual system is proficient in perceiving three dimensional shape from the shading patterns in a two dimensional image how it does this is not well understood and continues to be a question of fundamental and practical interest in this paper we present a new quantitative approach to shape from shading that may provide some answers we suggest that the brain through evolution or prior experience has discovered that objects can be classified into lower dimensional object classes as to their shape extraction of shape from shading is then equivalent to the much simpler problem of parameter estimation in a low dimensional space we carry out this proposal for an important class of three dimensional 3d objects human heads from an ensemble of several hundred laser scanned 3d heads we use principal component analysis to derive a low dimensional parameterization of head shape space an algorithm for solving shape from shading using this representation is presented it works well even on real images where it is able to recover the 3d surface for a given person maintaining facial detail and identity from a single 2d image of his face this algorithm has applications in face recognition and animation

local statistics of realizable vertex models we study planar vertex models which are probability measures on edge subsets of a planar graph satisfying certain constraints at each vertex examples including the dimer model and 1 2 model which we will define we express the local statistics of a large class of vertex models on a finite hexagonal lattice as a linear combination of the local statistics of dimers on the corresponding fisher graph with the help of a generalized holographic algorithm using an n n torus to approximate the periodic infinite graph we give an explicit integral formula for the free energy and local statistics for configurations of the vertex model on an infinite bi periodic graph as an example we simulate the 1 2 model by the technique of glauber dynamics

a recursive approach to low complexity codes a method is described for constructing long error correcting codes from one or more shorter error correcting codes referred to as subcodes and a bipartite graph a graph is shown which specifies carefully chosen subsets of the digits of the new codes that must be codewords in one of the shorter subcodes lower bounds to the rate and the minimum distance of the new code are derived in terms of the parameters of the graph and the subeodes both the encoders and decoders proposed are shown to take advantage of the code s explicit decomposition into subcodes to decompose and simplify the associated computational processes bounds on the performance of two specific decoding algorithms are established and the asymptotic growth of the complexity of decoding for two types of codes and decoders is analyzed the proposed decoders are able to make effective use of probabilistic information supplied by the channel receiver e g reliability information without greatly increasing the number of computations required it is shown that choosing a transmission order for the digits that is appropriate for the graph and the subcodes can give the code excellent burst error correction abilities the construction principles

fast facial shape recovery from a single image with general unknown lighting by using tensor representation in this paper we propose a fast 3 d facial shape recovery algorithm from a single image with general unknown lighting in order to derive the algorithm we formulate a nonlinear least square problem with two parameter vectors which are related to personal identity and light conditions we then combine the spherical harmonics for the surface normals of a human face with tensor algebra and show that in a certain condition the dimensionality of the least square problem can be further reduced to one tenth of the regular subspace based model by using tensor decomposition n mode svd which greatly speeds up the computations in order to enhance the shape recovery performance we have incorporated prior information in updating the parameters in the experiment the proposed algorithm takes less than 0 4s to reconstruct a face and shows a significant performance improvement over other reported schemes

sample mean versus sample frechet mean for combining complex wishart matrices a statistical study the space of covariance matrices is a non euclidean space the matrices form a manifold which if equipped with a riemannian metric becomes a riemannian manifold and recently this idea has been used for comparison and clustering of complex valued spectral matrices which at a given frequency are typically modeled as complex wishart distributed random matrices identically distributed sample complex wishart matrices can be combined via a standard sample mean to derive a more stable overall estimator however using the riemannian geometry their so called sample frechet mean can also be found we derive the expected value of the determinant of the sample frechet mean and the expected value of the sample frechet mean itself the population frechet mean is shown to be a scaled version of the true covariance matrix the risk under convex loss functions for the standard sample mean is never larger than for the frechet mean in simulations the sample mean also performs better for the estimation of an important functional derived from the estimated covariance matrix namely partial coherence

the dynamics of group codes dual abelian group codes and systems fundamental results concerning the dynamics of abelian group codes behaviors and their duals are developed duals of sequence spaces over locally compact abelian lca groups may be defined via pontryagin duality dual group codes are orthogonal subgroups of dual sequence spaces the dual of a complete code or system is finite and the dual of a laurent code or system is anti laurent if c and c sup spl perp are dual codes then the state spaces of c act as the character groups of the state spaces of c sup spl perp the controllability properties of c are the observability properties of c sup spl perp in particular c is strongly controllable if and only if c sup spl perp is strongly observable and the controller memory of c is the observer memory of c sup spl perp the controller granules of c act as the character groups of the observer granules of c sup spl perp examples of minimal observer form encoder and syndrome former constructions are given finally every observer granule of c is an end around controller granule of c

normal factor graphs and holographic transformations this paper stands at the intersection of two distinct lines of research one line is holographic algorithms a powerful approach introduced by valiant for solving various counting problems in computer science the other is normal factor graphs an elegant framework proposed by forney for representing codes defined on graphs we introduce the notion of holographic transformations for normal factor graphs and establish a very general theorem called the generalized holant theorem which relates a normal factor graph to its holographic transformation we show that the generalized holant theorem on the one hand underlies the principle of holographic algorithms and on the other hand reduces to a general duality theorem for normal factor graphs a special case of which was first proved by forney in the course of our development we formalize a new semantics for normal factor graphs which highlights various linear algebraic properties that potentially enable the use of normal factor graphs as a linear algebraic tool

loop series expansion with propagation diagrams the bethe approximation is a successful method for approximating partition functions of probabilistic models associated with a graph recently chertkov and chernyak derived an interesting formula called loop series expansion which is an expansion of the partition function the main term of the series is the bethe approximation while other terms are labelled by subgraphs called generalized loops in this paper we derive a loop series expansion of binary pairwise markov random fields with propagation diagrams which describes rules as to how first messages and secondary messages propagate our approach allows us to express the loop series in the form of a polynomial with coefficients positive integers using the propagation diagrams we establish a new formula that shows a relation between the exact marginal probabilities and their bethe approximations

geometric methods for spectral analysis this paper explores a geometric framework for modeling nonstationary but slowly varying time series based on the assumption that short windowed power spectra capture their spectral character and that energy transference in the frequency domain has a physical significance the framework relies on certain notions of transportation distance and their respective geodesics to model possible nonparametric changes in the power spectral density with respect to time we discuss the relevance of this framework to applications in spectral tracking spectral averaging and speech morphing

characteristic generators and dualization for tail biting trellises this paper focuses on dualizing tail biting trellises particularly kv trellises these trellises are based on characteristic generators as introduced by koetter vardy 2003 and may be regarded as a natural generalization of minimal conventional trellises even though they are not necessarily minimal two dualization techniques will be investigated the local dualization introduced by forney 2001 for general normal graphs and a linear algebra based dualization tailored to the specific class of tail biting bahl cocke jelinek raviv bcjr trellises introduced by nori shankar 2006 it turns out that in general the bcjr dual is a subtrellis of the local dual while for kv trellises these two coincide furthermore making use of both the bcjr construction and the local dualization it will be shown that for each complete set of characteristic generators of a code there exists a complete set of characteristic generators of the dual code such that their resulting kv trellises are dual to each other if paired suitably this proves a stronger version of a conjecture formulated by koetter vardy

a robust real time algorithm for facial shape recovery from a single image containing cast shadow under general unknown lighting it is difficult to find a non iterative solution for statistical shape from shading sfs for a single image containing cast shadow under unknown arbitrary light conditions this is because it is not trivial to find a mapping from a convex cone in the image space to a point in the depth space to find a non iterative solution for the statistical sfs which has not been done yet to our knowledge we show that it is possible to approximate a cone of images illumination cone to a polytope by a nonlinear function which is a transform from cartesian coordinates to hyperspherical coordinates the images of a subject form a convex cone in the image space and it can be better to use the direction of an image in hyperspherical coordinates as an input feature for the mapping rather than the image itself the maximum error occurs on one of the vertices of the polytope if we choose the objective function to be the squared error of a linear function in the hyperspherical space hence we can solve the least square problem using the cost only for the vertices in deriving the solution we use the generalized rayleigh quotient canonical correlation analysis cca and prior information on input images to reduce the solution space and improve 3 d reconstruction performance in experiments the proposed scheme performs robustly under light variations and is fast enough for real time applications

on the capacity of two dimensional run length constrained channels two dimensional binary patterns that satisfy one dimensional d k run length constraints both horizontally and vertically are considered for a given d and k the capacity c sub d k is defined as c sub d k lim sub m n spl rarr spl infin log sub 2 n sub m n sup d k mn where n sub m n sup d k denotes the number of m spl times n rectangular patterns that satisfy the two dimensional d k run length constraint bounds on c sub d k are given and it is proven for every d spl ges 1 and every k d that c sub d k 0 if and only if k d 1 encoding algorithms are also discussed

on the geometry of maximum entropy problems we show that a simple geometric result suffices to derive the form of the optimal solution in a large class of finite and infinite dimensional maximum entropy problems concerning probability distributions spectral densities and covariance matrices these include burg s spectral estimation method and dempster s covariance completion as well as various recent generalizations of the above we then apply this orthogonality principle to the new problem of completing a block circulant covariance matrix when an a priori estimate is available

on unit constraint length convolutional codes unit constraint length convolutional codes are codes that can be generated by encoders with a single delay element the maximal free distances of these codes are determined in terms of the maximal minimum distances of linear block codes a formula for the weight enumerator of unit constraint length convolution codes is derived and a macwilliams identity to determine the weight enumerator state diagram of a code from the weight enumerator state diagram of its dual code is given

a general coupling based model framework for wideband mimo channels a general coupling based model framework for wideband multiple input multiple output mimo channels is presented in this paper under this framework the channel state information csi tensor can be expressed by the product of a coupling tensor a complex gaussian tensor and three unitary matrices the unitary matrices can be either eigenbases or steering matrices in different domains the coupling tensor reflects the relationship between the column vectors of these unitary matrices the complex gaussian tensor is used to describe the small scale fading several realizations of this framework are introduced including the wideband kronecker based wkb model the wideband eigenvalue decomposition based web model the wideband virtual presentation wvp model and the wideband hybrid why model to evaluate the performance of these models channel measurements were carried out in different indoor scenarios both at tsinghua university and lund university the results show that these models have good agreement with the measured data furthermore we can see that the why model can provide a tradeoff between complexity and accuracy in channel synthesis

a factor graph approach to algebraic topology with applications to kramers wannier duality algebraic topology studies topological spaces with the help of tools from abstract algebra the main focus of this paper is to show that many concepts from algebraic topology can be conveniently expressed in terms of normal factor graphs as an application we give an alternative proof of a classical duality result of kramers and wannier which expresses the partition function of the 2 d ising model at a low temperature in terms of the partition function of the 2 d ising model at a high temperature moreover we discuss analogous results for the 3 d ising model and the potts model

a coupled statistical model for face shape recovery from brightness images we focus on the problem of developing a coupled statistical model that can be used to recover facial shape from brightness images of faces we study three alternative representations for facial shape these are the surface height function the surface gradient and a fourier basis representation we jointly capture variations in intensity and the surface shape representations using a coupled statistical model the model is constructed by performing principal components analysis on sets of parameters describing the contents of the intensity images and the facial shape representations by fitting the coupled model to intensity data facial shape is implicitly recovered from the shape parameters experiments show that the coupled model is able to generate accurate shape from out of training sample intensity images

efficiently computing the permanent and hafnian of some banded toeplitz matrices we present a new efficient method for computing the permanent and hafnian of certain banded toeplitz matrices the method covers non trivial cases for which previous known methods do not apply the main idea is to use the elements of the first row and column which determine the entire toeplitz matrix to construct a digraph in which certain paths correspond to permutations that the permanent and hafnian count since counting paths can be done efficiently the permanent and hafnian for those matrices is easily obtainable

symmetric shape from shading using self ratio image in this paper we present a symmetric shape from shading sfs approach to recover both shape and albedo for symmetric objects lambertian surfaces with unknown varying albedo and orthographic projections are assumed in our formulation of symmetric sfs we have two image irradiance equations one is the standard equation used in sfs and the other is a self ratio image irradiance equation this new image irradiance equation relates the self ratio image which is defined as the ratio of two halves of the input image to light source and surface shape the introduction of the self ratio image facilitates the direct use of symmetry cue based on the self ratio image a new model based symmetric source from shading algorithm is also presented we then propose symmetric sfs algorithms to recover both shape and albedo from a single image and present experimental results the new symmetric sfs scheme has one important property the existence of a unique global solution which consists of unique local solutions at each point simultaneously obtained using the intensity information at that point and the surrounding local region and the assumption of a cal c 2 surface proofs for the existence of a unique solution in the cases of unknown constant and non constant albedos are provided

a method for enforcing integrability in shape from shading algorithms an approach for enforcing integrability a particular implementation of the approach an example of its application to extending an existing shape from shading algorithm and experimental results showing the improvement that results from enforcing integrability are presented a possibly nonintegrable estimate of surface slopes is represented by a finite set of basis functions and integrability is enforced by calculating the orthogonal projection onto a vector subspace spanning the set of integrable slopes the integrability projection constraint was applied to extending an iterative shape from shading algorithm of m j brooks and b k p horn 1985 experimental results show that the extended algorithm converges faster and with less error than the original version good surface reconstructions were obtained with and without known boundary conditions and for fairly complicated surfaces

loop calculus for nonbinary alphabets using concepts from information geometry the bethe approximation is a well known approximation of the partition function used in statistical physics recently an equality relating the partition function and its bethe approximation was obtained for graphical models with binary variables by chertkov and chernyak in this equality the multiplicative error in the bethe approximation is represented as a weighted sum over all generalized loops in the graphical model in this paper the equality is generalized to graphical models with nonbinary alphabet using concepts from information geometry

local irreducibility of tail biting trellises this paper investigates tail biting trellis realizations for linear block codes intrinsic trellis properties are used to characterize irreducibility on given intervals of the time axis it proves beneficial to always consider the trellis and its dual simultaneously a major role is played by trellis properties that amount to observability and controllability of trellis fragments of various lengths for fragments of length less than the minimum span length of the code it is shown that fragment observability and fragment controllability are equivalent to irreducibility for reducible trellises a constructive reduction procedure is presented the considerations also lead to a characterization for when the dual of a trellis allows a product factorization into elementary atomic trellises

statistical physics of loopy interactions independent loop approximation and beyond we consider an interacting system of spin variables on a loopy interaction graph identified by a tree graph and a set of loopy interactions we start from a high temperature expansion for loopy interactions represented by a sum of non negative contributions from all the possible frustration free loop configurations we then compute the loop corrections using different approximations for the nonlocal loop interactions induced by the spin correlations in the tree graph for distant loopy interactions we can exploit the exponential decay of correlations in the tree interaction graph to compute loop corrections within an independent loop approximation higher orders of the approximation are obtained by considering the correlations between the nearby loopy interactions involving larger number of spin variables in particular the sum over the loop configurations can be computed exactly by the belief propagation algorithm in the low orders of the approximation as long as the loopy interactions have a tree structure these results might be useful in developing more accurate and convergent message passing algorithms exploiting the structure of loopy interactions

new constraints on data closeness and needle map consistency for shape from shading this paper makes two contributions to the problem of needle map recovery using shape from shading first we provide a geometric update procedure which allows the image irradiance equation to be satisfied as a hard constraint this not only improves the data closeness of the recovered needle map but also removes the necessity for extensive parameter tuning second we exploit the improved ease of control of the new shape from shading process to investigate various types of needle map consistency constraint the first set of constraints are based on needle map smoothness the second avenue of investigation is to use curvature information to impose topographic constraints third we explore ways in which the needle map is recovered so as to be consistent with the image gradient field in each case we explore a variety of robust error measures and consistency weighting schemes that can be used to impose the desired constraints on the recovered needle map we provide an experimental assessment of the new shape from shading framework on both real world images and synthetic images with known ground truth surface normals the main conclusion drawn from our analysis is that the data closeness constraint improves the efficiency of shape from shading and that both the topographic and gradient consistency constraints improve the fidelity of the recovered needle map

monte carlo methods for the ferromagnetic potts model using factor graph duality normal factor graph duality offers new possibilities for monte carlo algorithms in graphical models specifically we consider the problem of estimating the partition function of the ferromagnetic ising and potts models by monte carlo methods which are known to work well at high temperatures but to fail at low temperatures we propose monte carlo methods uniform sampling and importance sampling in the dual normal factor graph and demonstrate that they behave differently they work particularly well at low temperatures by comparing the relative error in estimating the partition function we show that the proposed importance sampling algorithm significantly outperforms the state of the art deterministic and monte carlo methods for the ferromagnetic ising model in an external field we show the equivalence between the valid configurations in the dual normal factor graph and the terms that appear in the high temperature series expansion of the partition function following this result we discuss connections with jerrum sinclair s polynomial randomized approximation scheme the subgraphs world process for evaluating the partition function of ferromagnetic ising models

quantum circuits that can be simulated classically in polynomial time a model of quantum computation based on unitary matrix operations was introduced by feynman and deutsch it has been asked whether the power of this model exceeds that of classical turing machines we show here that a significant class of these quantum computations can be simulated classically in polynomial time in particular we show that two bit operations characterized by 4 4 matrices in which the sixteen entries obey a set of five polynomial relations can be composed according to certain rules to yield a class of circuits that can be simulated classically in polynomial time this contrasts with the known universality of two bit operations and demonstrates that efficient quantum computation of restricted classes is reconcilable with the polynomial time turing hypothesis therefore it is possible that the techniques introduced bring the quantum computational model within the realm of algebraic complexity theory in a manner consistent with one view of quantum physics the wave function is simulated deterministically and randomization arises only in the course of making measurements the results generalize the quantum model in that they do not require the matrices to be unitary in a different direction these techniques also yield deterministic polynomial time algorithms for the decision and parity problems for certain classes of read twice boolean formulae all our results are based on the use of gates that are defined in terms of their graph matching properties

partition function loop series for a general graphical model free energy corrections and message passing equations a loop series expansion for the partition function of a general statistical model on a graph is carried out if the auxiliary probability distributions of the expansion are chosen to be a fixed point of the belief propagation equation the first term of the loop series gives the bethe peierls bp free energy functional at the replica symmetric level of the mean field spin glass theory and corrections are contributed only by subgraphs that are free of dangling edges this result generalizes the early work of chertkov and chernyak on binary statistical models if the belief propagation equation has multiple fixed points a loop series expansion is performed for the grand partition function the first term of this series gives the bp free energy functional at the first step replica symmetry breaking rsb level of the mean field spin glass theory and corrections again come only from subgraphs that are free of dangling edges provided that the auxiliary probability distributions of the expansion are chosen to be a fixed point of the survey propagation equation the same loop series expansion can be performed for higher level partition functions obtaining the higher level rsb bp free energy functionals and the correction terms and message passing equations without using the bp approximation

fixed point algorithms for estimating power means of positive definite matrices estimating means of data points lying on the riemannian manifold of symmetric positive definite spd matrices has proved of great utility in applications requiring interpolation extrapolation smoothing signal detection and classification the power means of spd matrices with exponent p in the interval 1 1 interpolate in between the harmonic mean p 1 and the arithmetic mean p 1 while the geometric cartan karcher mean which is the one currently employed in most applications corresponds to their limit evaluated at 0 in this paper we treat the problem of estimating power means along the continuum p 1 1 given noisy observed measurement we provide a general fixed point algorithm mpm and we show that its convergence rate for p 0 5 deteriorates very little with the number and dimension of points given as input along the whole continuum mpm is also robust with respect to the dispersion of the points on the manifold noise much more than the gradient descent algorithm usually employed to estimate the geometric mean thus mpm is an efficient algorithm for the whole family of power means including the geometric mean which by mpm can be approximated with a desired precision by interpolating two solutions obtained with a small p value we also present an approximated version of the mpm algorithm with very low computational complexity for the special case p finally we show the appeal of power means through the classification of brain computer interface event related potentials data

constrained codes as networks of relations we address the well known problem of determining the capacity of constrained coding systems while the one dimensional case is well understood to the extent that there are techniques for rigorously deriving the exact capacity in contrast computing the exact capacity of a two dimensional constrained coding system is still an elusive research challenge the only known exception in the two dimensional case is an exact however not rigorous solution to the run length limited rll system on the hexagonal lattice furthermore only exponential time algorithms are known for the related problem of counting the exact number of constrained two dimensional information arrays we present the first known rigorous technique that yields an exact capacity of a two dimensional constrained coding system in addition we devise an efficient polynomial time algorithm for counting the exact number of constrained arrays of any given size our approach is a composition of a number of ideas and techniques describing the capacity problem as a solution to a counting problem in networks of relations graph theoretic tools originally developed in the field of statistical mechanics techniques for efficiently simulating quantum circuits as well as ideas from the theory related to the spectral distribution of toeplitz matrices using our technique we derive a closed form solution to the capacity related to the path cover constraint in a two dimensional triangular array the resulting calculated capacity is path cover is a generalization of the well known one dimensional rll constraint for which the capacity is known to be

codes on graphs duality and macwilliams identities a conceptual framework involving partition functions of normal factor graphs is introduced paralleling a similar recent development by al bashabsheh and mao the partition functions of dual normal factor graphs are shown to be a fourier transform pair whether or not the graphs have cycles the original normal graph duality theorem follows as a corollary within this framework macwilliams identities are found for various local and global weight generating functions of general group or linear codes on graphs this generalizes and provides a concise proof of the macwilliams identity for linear time invariant convolutional codes that was recently found by gluesing luerssen and schneider further macwilliams identities are developed for terminated convolutional codes particularly for tail biting codes similar to those studied recently by bocharova hug johannesson and kudryashov

characterization of quasi stationarity regions for vehicle to vehicle radio channels we analyze the nonwide sense stationarity non wss of vehicle to vehicle v2v radio channels using three metrics 1 the correlation matrix distance cmd 2 the wideband spectral divergence sd and 3 the shadow fading correlation the analysis is based on measurements carried out at 5 3 ghz using a mathbf 30 times mathbf 4 mimo system in suburban urban and underground parking areas several factors such as the existence of a line of sight los the speed of cars and the antenna array size and configuration are considered in the analysis of non wss it is found that quasi stationarity region ranges from 3 to 80 m in different v2v scenarios and is strongly affected by the above factors based on the comparison of the equivalent quasi stationarity region size estimated by the three metrics it is suggested to use sd and shadowing correlation metrics for systems with small electrical array apertures and to use cmd metric only for arrays with large electrical apertures

conal distances between rational spectral densities this paper generalizes thompson and hilbert metrics to the space of spectral densities the resulting complete metric space has the differentiable structure of a finsler manifold with explicit geodesics the corresponding distances are filtering invariant can be computed efficiently and admit geodesic paths that preserve rationality these are properties of fundamental importance in many engineering applications

the dynamics of group codes state spaces trellis diagrams and canonical encoders a group code c over a group g is a set of sequences of group elements that itself forms a group under a component wise group operation a group code has a well defined state space sigma sub k at each time k each code sequence passes through a well defined state sequence the set of all state sequences is also a group code the state code of c the state code defines an essentially unique minimal realization of c the trellis diagram of c is defined by the state code of c and by labels associated with each state transition the set of all label sequences forms a group code the label code of c which is isomorphic to the state code of c if c is complete and strongly controllable then a minimal encoder in controller canonical feedbackfree form may be constructed from certain sets of shortest possible code sequences called granules the size of the state space sigma sub k is equal to the size of the state space of this canonical encoder which is given by a decomposition of the input groups of c at each time k if c is time invariant and nu controllable then mod sigma sub k mod pi sub 1

distances and riemannian metrics for spectral density functions we introduce a differential geometric structure for spectral density functions of discrete time random processes this is quite analogous to the riemannian structure of information geometry which is used to study perturbations of probability density functions and which is based on the fisher information metric herein we introduce an analogous riemannian metric which we motivate with a problem in prediction theory it turns out that this problem also provides a prediction theoretic interpretation to the itakura distortion measure which relates to our metric geodesies and geodesic distances are characterized in closed form and hence the geodesic distance between two spectral density functions provides an explicit intrinsic pseudo metric on the cone of density functions certain other distortion measures that involve generalized means of spectral density functions are shown to lead to the same riemannian metric finally an alternative riemannian metric is introduced which is motivated by an analogous problem involving smoothing instead of prediction

counting in graph covers a combinatorial characterization of the bethe entropy function we present a combinatorial characterization of the bethe entropy function of a factor graph such a characterization being in contrast to the original analytical definition of this function we achieve this combinatorial characterization by counting valid configurations in finite graph covers of the factor graph analogously we give a combinatorial characterization of the bethe partition function whose original definition was also of an analytical nature as we point out our approach has similarities to the replica method but also stark differences the above findings are a natural backdrop for introducing a decoder for graph based codes that we will call symbolwise graph cover decoding a decoder that extends our earlier work on blockwise graph cover decoding both graph cover decoders are theoretical tools that help toward a better understanding of message passing iterative decoding namely blockwise graph cover decoding links max product min sum algorithm decoding with linear programming decoding and symbolwise graph cover decoding links sum product algorithm decoding with bethe free energy function minimization at temperature one in contrast to the gibbs entropy function which is a concave function the bethe entropy function is in general not concave everywhere in particular we show that every code picked from an ensemble of regular low density parity check codes with minimum hamming distance growing with high probability linearly with the block length has a bethe entropy function that is convex in certain regions of its domain

characterization of non stationary channels using mismatched wiener filtering a common simplification in the statistical treatment of linear time varying ltv wireless channels is the approximation of the channel as a stationary random process inside certain time frequency regions we develop a methodology for the determination of local quasi stationarity lqs regions i e local regions in which a channel can be treated as stationary contrary to previous results relying on to some extent heuristic measures and thresholds we consider a finite length wiener filter as realistic channel estimator and relate the size of lqs regions in time to the degradation of the mean square error mse of the estimate due to outdated and thus mismatched channel statistics we show that for certain power spectral densities psds of the channel a simplified but approximate evaluation of the matched mse based on the assumption of an infinite filtering length yields a lower bound on the actual matched mse moreover for such psds the actual mse degradation is upper bounded and the size of the actual lqs regions is lower bounded by the approximate evaluation using channel measurements we compare the evolution of the lqs regions based on the actual and the approximate mse they show strong similarities

approximating the permanent with fractional belief propagation we discuss schemes for exact and approximate computations of permanents and compare them with each other specifically we analyze the belief propagation bp approach and its fractional belief propagation fbp generalization for computing the permanent of a non negative matrix known bounds and conjectures are verified in experiments and some new theoretical relations bounds and conjectures are proposed the fractional free energy ffe function is parameterized by a scalar parameter 1 1 where 1 corresponds to the bp limit and 1 corresponds to the exclusion principle but ignoring perfect matching constraints mean field mf limit ffe shows monotonicity and continuity with respect to for every non negative matrix we define its special value 1 0 to be the for which the minimum of the parameterized ffe function is equal to the permanent of the matrix where the lower and upper bounds of the interval corresponds to respective bounds for the permanent our experimental analysis suggests that the distribution of varies for different ensembles but always lies within the 1 1 2 interval moreover for all ensembles considered the behavior of is highly distinctive offering an empirical practical guidance for estimating permanents of non negative matrices via the ffe approach

mixing time of markov chains for the 1 2 model a 1 2 model configuration is a subset of edges of a hexagonal lattice satisfying the constraint that each vertex is incident to 1 or 2 edges we introduce markov chains to sample the 1 2 model configurations on the 2d hexagonal lattice and prove that the mixing time of these chains is polynomial in the sizes of the graphs for a large class of probability measures

on the macwilliams identity for convolutional codes the adjacency matrix associated with a convolutional code collects in a detailed manner information about the weight distribution of the code a macwilliams identity conjecture stating that the adjacency matrix of a code fully determines the adjacency matrix of the dual code will be formulated and an explicit formula for the transformation will be stated the formula involves the macwilliams matrix known from complete weight enumerators of block codes the conjecture will be proven for the class of convolutional codes where either the code itself or its dual does not have forney indices bigger than one for the general case the conjecture is backed up by many examples and a weaker version will be established

a convex optimization approach to arma modeling we formulate a convex optimization problem for approximating any given spectral density with a rational one having a prescribed number of poles and zeros n poles and m zeros inside the unit disc and their conjugates the approximation utilizes the kullback leibler divergence as a distance measure the stationarity condition for optimality requires that the approximant matches n 1 covariance moments of the given power spectrum and m cepstral moments of the corresponding logarithm although the latter with possible slack the solution coincides with one derived by byrnes enqvist and lindquist who addressed directly the question of covariance and cepstral matching thus the present paper provides an approximation theoretic justification of such a problem since the approximation requires only moments of spectral densities and of their logarithms it can also be used for system identification

a macwilliams identity for convolutional codes the general case a macwilliams identity for convolutional codes will be established it makes use of the weight adjacency matrices of the code and its dual based on state space realizations the controller canonical form of the codes in question the macwilliams identity applies to various notions of duality appearing in the literature on convolutional coding theory

incorporating illumination constraints in deformable models for shape from shading and light direction estimation we present a method for the integration of nonlinear holonomic constraints in deformable models and its application to the problems of shape and illuminant direction estimation from shading experimental results demonstrate that our method performs better than previous shape from shading algorithms applied to images of lambertian objects under known illumination it is also more general as it can be applied to non lambertian surfaces and it does not require knowledge of the illuminant direction in this paper 1 we first develop a theory for the numerically robust integration of nonlinear holonomic constraints within a deformable model framework in this formulation we use lagrange multipliers and a baumgarte stabilization approach 1972 2 we also describe a fast new method for the computation of constraint based forces in the case of high numbers of local parameters 3 we demonstrate how any type of illumination constraint from the simple lambertian model to more complex highly nonlinear models can be incorporated in a deformable model framework 4 we extend our method to work when the direction of the light source is not known we couple our shape estimation method with a method for light estimation in an iterative process where improved shape estimation results in improved light estimation and vice versa 5 we perform a series of experiments

on the convergence of an efficient algorithm for kullback leibler approximation of spectral densities this paper deals with a method for the approximation of a spectral density function among the solutions of a generalized moment problem a la byrnes georgiou lindquist the approximation is pursued with respect to the kullback leibler pseudo distance which gives rise to a convex optimization problem after developing the variational analysis we discuss the properties of an efficient algorithm for the solution of the corresponding dual problem based on the iteration of a nonlinear map in a bounded subset of the dual space our main result is the proof of local convergence of the latter established as a consequence of the central manifold theorem supported by numerical evidence we conjecture that in the mentioned bounded set the convergence is actually global

mathematical programming decoding of binary linear codes theory and algorithms mathematical programming is a branch of applied mathematics and has recently been used to derive new decoding approaches challenging established but often heuristic algorithms based on iterative message passing concepts from mathematical programming used in the context of decoding include linear integer and nonlinear programming network flows notions of duality as well as matroid and polyhedral theory this paper reviews and categorizes decoding methods based on mathematical programming approaches for binary linear codes over binary input memoryless symmetric channels

mean and median of psd matrices on a riemannian manifold application to detection of narrow band sonar signals the rich information in the power spectral density psd matrix of a received signal can be extracted in different ways for various purposes in signal processing care however must be taken when distances between psd matrices are considered because these matrices form a riemannian manifold in the signal space in this paper two riemannian distances rd are introduced for the measurement of distances between psd matrices on the manifold the principle by which the geodesics on the manifold can be lifted to a euclidean subspace isometric with the tangent space of the manifold is also explained and emphasized this leads to the concept that any optimization involving the rd on the manifold can be equivalently performed in this euclidean subspace the application of this principle is illustrated by the development of iterative algorithms to find the riemannian means and riemannian medians according to the two rd these distance measures are then applied to the detection of narrow band sonar signals exploration of the translation of measure reference as well as the application of optimum weighting show substantial improvement in detection performance over classical detection method

facial shape from shading and recognition using principal geodesic analysis and robust statistics the aim in this paper is to use principal geodesic analysis to model the statistical variations for sets of facial needle maps we commence by showing how to represent the distribution of surface normals using the exponential map shape deformations are described using principal geodesic analysis on the exponential map using ideas from robust statistics we show how this deformable model may be fitted to facial images in which there is significant self shadowing moreover we demonstrate that the resulting shape from shading algorithm can be used to recover accurate facial shape and albedo from real world images in particular the algorithm can effectively fill in the facial surface when more than 30 of its area is subject to self shadowing to investigate the utility of the shape parameters delivered by the method we conduct experiments with illumination insensitive face recognition we present a novel recognition strategy in which similarity is measured in the space of the principal geodesic parameters we also use the recovered shape information to generate illumination normalized prototype images on which recognition can be performed finally we show that from a single input image we are able to generate the basis images employed by a number of well known illumination insensitive recognition algorithms we also demonstrate that the principal geodesics provide an efficient parameterization of the space of harmonic basis images

recovering facial shape using a statistical model of surface normal direction in this paper we show how a statistical model of facial shape can be embedded within a shape from shading algorithm we describe how facial shape can be captured using a statistical model of variations in surface normal direction to construct this model we make use of the azimuthal equidistant projection to map the distribution of surface normals from the polar representation on a unit sphere to cartesian points on a local tangent plane the distribution of surface normal directions is captured using the covariance matrix for the projected point positions the eigenvectors of the covariance matrix define the modes of shape variation in the fields of transformed surface normals we show how this model can be trained using surface normal data acquired from range images and how to fit the model to intensity images of faces using constraints on the surface normal direction provided by lambert s law we demonstrate that the combination of a global statistical constraint and local irradiance constraint yields an efficient and accurate approach to facial shape recovery and is capable of recovering fine local surface details we assess the accuracy of the technique on a variety of images with ground truth and real world images

gender discriminating models from facial surface normals in this paper we show how to use facial shape information to construct discriminating models for gender classification we represent facial shapes using 2 5d fields of facial surface normals and investigate three different methods to improve the gender discriminating capacity of the model constructed using the standard eigenspace method the three methods are novel variants of principal geodesic analysis pga namely a weighted pga b supervised weighted pga and c supervised pga our starting point is to define a weight map over the facial surface that indicates the importance of different locations in discriminating gender we show how to compute the relevant weights and how to incorporate the weights into the 2 5d model construction we evaluate the performance of the alternative methods using facial surface normals extracted from 3d range images or recovered from brightness images experimental results demonstrate the effectiveness of our methods moreover the classification accuracy which is as high as 97 demonstrates the effectiveness of using facial shape information for gender classification

the relation between information theory and the differential geometry approach to statistics abstract cencov has shown that the riemannian metric on the probability simplex xi 1 defined by ds 2 dx i 2 x i has an invariance property under certain probabilistically natural mappings no other riemannian metric has the same property the geometry associated with this metric is shown to lead almost automatically to measures of divergence between probability distributions which are associated with kullback bhattacharyya and matusita certain vector fields are associated in a natural way with random variables the integral curves of these vector fields yield the maximum entropy or minimum divergence estimates of probabilities some other consequences of this geometric view are also explored

error correction on a tree an instanton approach we introduce a method that allows analytical or semianalytical estimating of the post error correction bit error rate ber when a forward error correction is utilized for transmitting information through a noisy channel the generic method that applies to a variety of error correction schemes in the regimes where the ber is low is illustrated using the example of a finite size code approximated by a treelike structure exploring the statistical physics formulation of the problem we find that the ber decreases with the signal to noise ratio nonuniformly i e crossing over through a sequence of phases the higher the signal to noise ratio the lower the symmetry of the phase dominating ber the last 50 years have witnessed a tremendous increase in the amount of data transmitted through various communication systems this also leads to tightening of the conditions for error free data transfer in the presence of noise and other transmission related impairments the problem of dealing with errors in information flow has a fundamental importance and has been studied extensively in information and coding theory in 1948 shannon 1 proved that applying an error corrected code can result in an error free communication in the thermodynamic limit of an infinitely long message as long as the rate of transmitted information is kept below a certain value known as the channel capacity constructing well performing capacity approaching yet practical codes has been a challenge until the discovery that some classes of codes based on random construction 2 can achieve near optimum performance when used for transmission over white additive gaussian noise channels 3 in the past few years several codes have been designed with performances very close to this limit 4 generally these codes are referred to as codes on graphs and their prime examples are low density parity check ldpc codes and turbo codes a linear block code for which ldpc codes are an example can be represented as the set

new bounds on the capacity of multidimensional run length constraints we examine the well known problem of determining the capacity of multidimensional run length limited constrained systems by recasting the problem which is essentially a combinatorial counting problem into a probabilistic setting we are able to derive new lower and upper bounds on the capacity of 0 k rll systems these bounds are better than all previously known analytical bounds for k 2 and are tight asymptotically thus we settle the open question what is the rate at which the capacity of 0 k rll systems converges to 1 as k we also provide the first nontrivial upper bound on the capacity of general d k rll systems

woven convolutional codes i encoder properties encoders for convolutional codes with large free distances can be constructed by combining several less powerful convolutional encoders this paper is devoted to constructions in which the constituent convolutional codes are woven together in a manner that resembles the structure of a fabric the general construction is called twill and it is described together with two special cases viz woven convolutional encoders with outer warp and with inner warp the woven convolutional encoders inherit many of their structural properties such as minimality and catastrophicity from their constituent encoders for all three types of woven convolutional codes upper and lower bounds on their free distances as well as lower bounds on the active distances of their encoders are derived

mean field method with correlations determined by linear response we introduce a mean field approximation based on the reconciliation of maximum entropy and linear response for correlations in the cluster variation method within a general formalism that includes previous mean field methods we derive formulas improving on e g the bethe approximation and the sessak monasson result at high temperature applying the method to direct and inverse ising problems we find improvements over standard implementations

time and spectral domain relative entropy a new approach to multivariate spectral estimation the concept of spectral relative entropy rate is introduced for jointly stationary gaussian processes using classical information theoretic results we establish a remarkable connection between time and spectral domain relative entropy rates this naturally leads to a new spectral estimation technique where a multivariate version of the itakura saito distance is employed it may be viewed as an extension of the approach called three introduced by byrnes georgiou and lindquist in 2000 which in turn followed in the footsteps of the burg jaynes maximum entropy method spectral estimation is here recast in the form of a constrained spectrum approximation problem where the distance is equal to the processes relative entropy rate the corresponding solution entails a complexity upper bound which improves on the one so far available in the multichannel framework indeed it is equal to the one featured by three in the scalar case the solution is computed via a globally convergent matricial newton type algorithm simulations suggest the effectiveness of the new technique in tackling multivariate spectral estimation tasks especially in the case of short data records

codes on graphs observability controllability and local reducibility this paper investigates properties of realizations of linear or group codes on general graphs that lead to local reducibility trimness and properness are dual properties of constraint codes a linear or group realization with a constraint code that is not both trim and proper is locally reducible a linear or group realization on a finite cycle free graph is minimal if and only if every local constraint code is trim and proper a realization is called observable if there is a one to one correspondence between codewords and configurations and controllable if it has independent constraints a linear or group realization is observable if and only if its dual is controllable a simple counting test for controllability is given an unobservable or uncontrollable realization is locally reducible parity check realizations are controllable if and only if they have independent parity checks in an uncontrollable tail biting trellis realization the behavior partitions into disconnected sub behaviors but this property does not hold for nontrellis realizations on a general graph the support of an unobservable configuration is a generalized cycle

permanents pfaffian orientations and even directed circuits given a 0 1 square matrix a when can some of the 1 s be changed toi1 s in such a way that the permanent of a equals the determinant of the modifled matrix when does a real square matrix have the property that every real matrix with the same sign pattern that is the corresponding entries either have the same sign or are both zero is nonsingular when is a hypergraph with n vertices and n hyperedges minimally nonbipartite when does a bipartite graph have a pfa an orientation given a digraph does it have no directed circuit of even length given a digraph does it have a subdivision with no even directed circuit it is known that all of the above problems are equivalent we prove a structural characterization of the feasible instances which implies a polynomial time algorithm to solve all of the above problems the structural characterization says roughly speaking that a bipartite graph has a pfa an orientation if and only if it can be obtained by piecing together in a specifled way planar bipartite graphs and one sporadic nonplanar bipartite graph