Parameter Estimation for Undirected Graphical Models With Hard Constraints

The hardcore model on a graph <inline-formula> <tex-math notation="LaTeX">$G$ </tex-math></inline-formula> with parameter <inline-formula> <tex-math notation="LaTeX">$\lambda > 0$ </tex-math></inline-formula> is a probability measure on the collection of all independent sets of <inline-formula> <tex-math notation="LaTeX">$G$ </tex-math></inline-formula>, that assigns to each independent set <inline-formula> <tex-math notation="LaTeX">$I$ </tex-math></inline-formula> a probability proportional to <inline-formula> <tex-math notation="LaTeX">$\lambda ^{|I|}$ </tex-math></inline-formula>. In this paper we consider the problem of estimating the parameter <inline-formula> <tex-math notation="LaTeX">$\lambda $ </tex-math></inline-formula> given a single sample from the hardcore model on a graph <inline-formula> <tex-math notation="LaTeX">$G$ </tex-math></inline-formula>. To bypass the computational intractability of the maximum likelihood method, we use the maximum pseudo-likelihood (MPL) estimator, which for the hardcore model has a surprisingly simple closed form expression. We show that for any sequence of graphs <inline-formula> <tex-math notation="LaTeX">$\{G_{N}\}_{N \geq 1}$ </tex-math></inline-formula>, where <inline-formula> <tex-math notation="LaTeX">$G_{N}$ </tex-math></inline-formula> is a graph on <inline-formula> <tex-math notation="LaTeX">$N$ </tex-math></inline-formula> vertices, the MPL estimate of <inline-formula> <tex-math notation="LaTeX">$\lambda $ </tex-math></inline-formula> is <inline-formula> <tex-math notation="LaTeX">$\sqrt {N}$ </tex-math></inline-formula>-consistent (that is, it converges to the true parameter at rate <inline-formula> <tex-math notation="LaTeX">$1/\sqrt {N}$ </tex-math></inline-formula>), whenever the graph sequence has uniformly bounded average degree. We then extend our methods to obtain estimates for the vector of activity parameters in general <inline-formula> <tex-math notation="LaTeX">$H$ </tex-math></inline-formula>-coloring models, in which restrictions between adjacent colors are encoded by a constraint graph <inline-formula> <tex-math notation="LaTeX">$H$ </tex-math></inline-formula>. These constitute an important class of Markov random fields that includes all hard-constraint models, which arise in a broad array of fields including combinatorics, statistical physics, and communication networks. Given a single sample from an <inline-formula> <tex-math notation="LaTeX">$H$ </tex-math></inline-formula>-coloring model, we derive sufficient conditions under which the MPL estimate is <inline-formula> <tex-math notation="LaTeX">$\sqrt {N}$ </tex-math></inline-formula>-consistent. Moreover, we verify the sufficient conditions for <inline-formula> <tex-math notation="LaTeX">$H$ </tex-math></inline-formula>-coloring models for which there is at least one ‘unconstrained’ color (that is, there exists at least one vertex in the constraint graph <inline-formula> <tex-math notation="LaTeX">$H$ </tex-math></inline-formula> that is connected to all vertices), as long as the graph sequence has uniformly bounded average degree. This applies to many <inline-formula> <tex-math notation="LaTeX">$H$ </tex-math></inline-formula>-coloring examples such as the Widom-Rowlinson and multi-state hard-core models. On the other hand, for the <inline-formula> <tex-math notation="LaTeX">$q$ </tex-math></inline-formula>-coloring model, which falls outside this class, we show that the condition can fail and consistent estimation may be impossible even for graphs with bounded average degree. Nevertheless, we show that the MPL estimate is <inline-formula> <tex-math notation="LaTeX">$\sqrt {N}$ </tex-math></inline-formula>-consistent in the <inline-formula> <tex-math notation="LaTeX">$q$ </tex-math></inline-formula>-coloring model when <inline-formula> <tex-math notation="LaTeX">$\{G_{N}\}_{N \geq 1}$ </tex-math></inline-formula> has bounded average double neighborhood. The presence of hard constraints, as opposed to soft constraints, leads to new challenges, and our proofs entail applications of the method of exchangeable pairs as well as combinatorial arguments that employ the probabilistic method.