Testing Probabilistic Circuits

1 Introduction . Probabilistic modeling is at the heart of modern computer science , with applications ranging from image recognition and image generation [ 29 , 30 ] to weather forecasting [ 3 ] . Probabilistic models have a multitude of representations , such as probabilistic circuits ( PCs ) [ 9 ] , graphical models [ 19 ] , generative networks [ 16 ] , and determinantal point processes [ 20 ] . Of particular interest to us are PCs , which are known to support guaranteed inference and thus have applications in safety-critical fields such as healthcare [ 2 , 25 ] . In this work , we will focus on PCs that are fragments of the Negation Normal Form ( NNF ) , specifically DNNFs , d-DNNFs , SDNNFs , and PIs [ 13 ] . We refer to the survey by Choi et al . [ 9 ] for more details regarding PCs . Given two distributions P and Q , a fundamental problem is to determine whether they are close . Closeness between distributions is frequently quantified using the total variation ( TV ) distance , dTV ( P , Q ) = ( 1/2 ) ‖P −Q‖1 , where ‖ · ‖ is the ` 1 norm [ 21 , 6 ] . Thus , stated formally , closeness testing is the problem of deciding whether dTV ( P , Q ) ≤ ε or dTV ( P , Q ) ≥ η for 0 ≤ ε < η ≤ 1 . Determining the closeness of models has applications in AI planning [ 13 ] , bioinformatics [ 31 , 33 , 35 ] and probabilistic program verification [ 15 , 23 ] . Equivalence testing is a special case of closeness testing , where one tests if dTV ( P , Q ) = 0 . Darwiche and Huang [ 13 ] initiated the study of equivalence testing of PCs by designing an equivalence test for d-DNNFs . An equivalence test is , however , of little use in contexts where the PCs under test ∗The accompanying tool , available open source , can be found at https : //github.com/meelgroup/teq . The Appendix is available in the accompanying supplementary material . †The authors decided to forgo the old convention of alphabetical ordering of authors in favor of a randomized ordering , denoted by r© . The publicly verifiable record of the randomization is available at https : //www.aeaweb.org/journals/policies/random-author-order/search with confirmation code : icoxrj0sNB2L . For citation of the work , authors request that the citation guidelines by AEA for random author ordering be followed . 35th Conference on Neural Information Processing Systems ( NeurIPS 2021 ) . encode non-identical distributions that are nonetheless close enough for practical purposes . Such situations may arise due to the use of approximate PC compilation [ 10 ] and sampling-based learning of PCs [ 26 , 27 ] . As a concrete example , consider PCs that are learned via approximate methods such as stochastic gradient descent [ 27 ] . In such a case , PCs are likely to converge to close but non-identical distributions . Given two such PCs , we would like to know whether they have converged to distributions close to each other . Thus , we raise the question : Does there exist an efficient algorithm to test the closeness of two PC distributions ? In this work , we design the first closeness test for PCs with respect to TV distance , called Teq . Assuming the tested PCs allow poly-time approximate weighted model counting and sampling , Teq runs in polynomial time . Formally , given two PC distributions P and Q , and three parameters ( ε , η , δ ) , for closeness ( ε ) , farness ( η ) , and tolerance ( δ ) , Teq returns Accept if dTV ( P , Q ) ≤ ε and Reject if dTV ( P , Q ) ≥ η with probability at least 1 − δ. Teq makes atmost O ( ( η − ε ) −2 log ( δ−1 ) ) calls to the sampler and exactly 2 calls to the counter . Teq builds on a general distance estimation technique of Canonne and Rubinfeld [ 4 ] that estimates the distance between two distributions with a small number of samples . In the context of PCs , the algorithm requires access to an exact sampler and an exact counter . Since not all PCs support exact sampling and counting , we modify the technique presented in [ 4 ] to allow for approximate samples and counts . Furthermore , we implement and test Teq on a dataset of publicly available PCs arising from applications in circuit testing . Our results show that closeness testing can be accurate and scalable in practice . For some NNF fragments , such as DNNF , no sampling algorithm is known , and for fragments such as PI , sampling is known to be NP-hard [ 32 ] . Since Teq requires access to approximate weighted counters and samplers to achieve tractability , the question of determining the closeness of the PCs mentioned above remains unanswered . Thus , we investigate further and characterize the complexity of closeness testing for a broad range of PCs . Our characterization reveals that PCs from the fragments d-DNNFs and SDNNFs can be tested for closeness in poly-time via Teq , owing to the algorithms of Darwiche [ 11 ] and Arenas et al . [ 1 ] . We show that the SDNNF approximate counting algorithm of Arenas et al . [ 1 ] can be extended to log-linear SDNNFs using chain formulas [ 8 ] . Then , using previously known results , we also find that there are no poly-time equivalence tests for PCs from PI and DNNF , conditional on widely believed complexity-theoretic conjectures . Our characterization also reveals some open questions regarding the complexity of closeness and equivalence testing of PCs . The rest of the paper is organized in the following way . We define the notation and discuss related work in Section 2 . We then present the main contribution of the paper , the closeness test Teq , and the associated proof of correctness in Section 3 . We present our experimental findings in Section 4 , and then discuss the complexity landscape of closeness testing in Section 5 . We conclude the paper and discuss some open problems in Section 6 . Due to space constraints , we defer some proofs to the supplementary Section A . 2 Background . Let ϕ : { 0 , 1 } n → { 0 , 1 } be a circuit over n Boolean variables . An assignment σ ∈ { 0 , 1 } n to the variables of ϕ is a satisfying assignment if ϕ ( σ ) = 1 . The set of all satisfying assignments of ϕ is Rϕ . If |Rϕ| > 0 , then ϕ is said to be satisfiable and if |Rϕ| = 2n , then ϕ is said to be valid . We use |ϕ| to denote the size of circuit ϕ , where the size is the total number of vertices and edges in the circuit DAG . The polynomial hierarchy ( PH ) contains the classes ΣP1 ( NP ) and Π P 1 ( co-NP ) along with generalizations of the form ΣPi and Π P i where Π P i+1 = co-NP ΠPi and ΣPi+1 = NP ΣPi [ 34 ] . The classes ΣPi and ΠPi are said to be at level i . If it is shown that two classes on the same or consecutive levels are equal , the hierarchy collapses to that level . Such a collapse is considered unlikely , and hence is used as the basic assumption for showing hardness results , including the ones we present in the paper . 2.1 Probability distributions . A weight function w : { 0 , 1 } n → Q+ assigns a positive rational weight to each assignment σ . We extend the definition of w to also allow circuits as input : w ( ϕ ) = ∑ σ∈Rϕ w ( σ ) . For weight function w and circuit ϕ , w ( ϕ ) is the weighted model count ( WMC ) of ϕ w.r.t . w. In this paper , we focus on log-linear weight functions as they capture a wide class of distributions , including those arising from graphical models , conditional random fields , and skip-gram models [ 24 ] . Log-linear models are represented as literal-weighted functions , defined as : Definition 1 . For a set X of n variables , a weight function w is called literal-weighted if there is a poly-time computable map w : X → Q ∩ ( 0 , 1 ) such that for any assignment σ ∈ { 0 , 1 } n : w ( σ ) = ∏ x∈σ { w ( x ) if x = 1 1− w ( x ) if x = 0 For all circuits ϕ , and log-linear weight functions w , w ( ϕ ) can be represented in size polynomial in the input . Probabilistic circuits : A probabilistic circuit is a satisfiable circuit ϕ along with a weight function w. ϕ and w together define a discrete probability distribution on the set { 0 , 1 } n that is supported over Rϕ . We denote the p.m.f . of this distribution as : P ( ϕ , w ) ( σ ) = { 0 ϕ ( σ ) = 0 w ( σ ) /w ( ϕ ) ϕ ( σ ) = 1 In this paper , we study circuits that are fragments of the Negation Normal Form ( NNF ) . A circuit ϕ in NNF is a rooted , directed acyclic graph ( DAG ) , where each leaf node is labeled with true , false , v or ¬v ; and each internal node is labeled with a ∧ or ∨ and can have arbitrarily many children . We focus on four fragments of NNF , namely , Decomposable NNF ( DNNF ) , deterministic-DNNF ( d-DNNF ) , Structured DNNF ( SDNNF ) , and Prime Implicates ( PI ) . For further information regarding circuits in NNF , refer to the survey [ 14 ] and the paper [ 28 ] . The TV distance of two probability distributions P ( ϕ1 , w1 ) and P ( ϕ2 , w2 ) over { 0 , 1 } n is defined as : dTV ( P ( ϕ1 , w1 ) , P ( ϕ2 , w2 ) ) = 1 2 ∑ σ∈ { 0,1 } n |P ( ϕ1 , w1 ) ( σ ) − P ( ϕ2 , w2 ) ( σ ) | . P ( ϕ1 , w1 ) and P ( ϕ2 , w2 ) are said to be ( 1 ) equivalent if dTV ( P ( ϕ1 , w1 ) , P ( ϕ2 , w2 ) ) = 0 , ( 2 ) ε-close if dTV ( P ( ϕ1 , w1 ) , P ( ϕ2 , w2 ) ) ≤ ε , and ( 3 ) η-far if dTV ( P ( ϕ1 , w1 ) , P ( ϕ2 , w2 ) ) ≥ η . Our closeness testing algorithm Teq , assumes access to an approximate weighted counter Awct ( α , β , ϕ , w ) , and an approximate weighted sampler Samp ( α , β , ϕ , w ) . We define their behavior as follows : Definition 2 . Awct ( α , β , ϕ , w ) takes a circuit ϕ , a weight function w , a tolerance parameter α > 0 and a confidence parameter β > 0 as input and returns the approximate weighted model count of ϕ w.r.t . w such that Pr [ w ( ϕ ) 1 + α ≤ Awct ( α , β , ϕ , w ) ≤ ( 1 + α ) w ( ϕ ) ] ≥ 1− β Tractable approximate counting algorithms for PCs are known as Fully Polynomial Randomised Approximation Schemes ( FPRAS ) . The running time of an FPRAS is given by T ( α , β , ϕ ) = poly ( α−1 , log ( β−1 ) , |ϕ| ) . Definition 3 . Samp ( α , β , ϕ , w ) takes a circuit ϕ , a weight function w , a tolerance parameter α > 0 and a confidence parameter β > 0 as input and returns either ( 1 ) a satisfying assignment σ sampled approximately w.r.t . weight function w with probability ≥ 1− β or ( 2 ) a symbol ⊥ indicating failure with probability < β . In other words , whenever Samp samples σ : P ( ϕ , w ) ( σ ) 1 + α ≤ Pr [ Samp ( α , β , ϕ , w ) = σ ] ≤ ( 1 + α ) P ( ϕ , w ) ( σ ) Tractable approximate sampling algorithms for PCs are known as Fully Polynomial Almost Uniform Samplers ( FPAUS ) . The running time of an FPAUS for a single sample is given by T ( α , β , ϕ ) = poly ( α−1 , log ( β−1 ) , |ϕ| ) . In the rest of the paper [ m ] denotes the set { 1 , 2 , . . .m } , 1 ( e ) represents the indicator variable for event e , and E ( v ) represents the expectation of random variable v .

Summary: The contribution of this paper is a method for testing the closeness of the probability distributions encoded by two different probabalistic circuits (PCs). The closeness tester (called Teq), assumes access to an approximate weighted counter and an approximate weighted sampler. Teq is described (in words and pseudo-code) and the bulk of the paper is devoted to proving (PAC-style) correctness of Teq (ie Theorem 1) and characterising its time complexity (Theorem 2). Empirical results are given supporting the correctness of Teq and the stated time complexity theoretical results on Teq.