Testing Probabilistic Circuits

1 Introduction . Probabilistic modeling is at the heart of modern computer science , with applications ranging from image recognition and image generation [ 29 , 30 ] to weather forecasting [ 3 ] . Probabilistic models have a multitude of representations , such as probabilistic circuits ( PCs ) [ 9 ] , graphical models [ 19 ] , generative networks [ 16 ] , and determinantal point processes [ 20 ] . Of particular interest to us are PCs , which are known to support guaranteed inference and thus have applications in safety-critical fields such as healthcare [ 2 , 25 ] . In this work , we will focus on PCs that are fragments of the Negation Normal Form ( NNF ) , specifically DNNFs , d-DNNFs , SDNNFs , and PIs [ 13 ] . We refer to the survey by Choi et al . [ 9 ] for more details regarding PCs . Given two distributions P and Q , a fundamental problem is to determine whether they are close . Closeness between distributions is frequently quantified using the total variation ( TV ) distance , dTV ( P , Q ) = ( 1/2 ) ‖P −Q‖1 , where ‖ · ‖ is the ` 1 norm [ 21 , 6 ] . Thus , stated formally , closeness testing is the problem of deciding whether dTV ( P , Q ) ≤ ε or dTV ( P , Q ) ≥ η for 0 ≤ ε < η ≤ 1 . Determining the closeness of models has applications in AI planning [ 13 ] , bioinformatics [ 31 , 33 , 35 ] and probabilistic program verification [ 15 , 23 ] . Equivalence testing is a special case of closeness testing , where one tests if dTV ( P , Q ) = 0 . Darwiche and Huang [ 13 ] initiated the study of equivalence testing of PCs by designing an equivalence test for d-DNNFs . An equivalence test is , however , of little use in contexts where the PCs under test ∗The accompanying tool , available open source , can be found at https : //github.com/meelgroup/teq . The Appendix is available in the accompanying supplementary material . †The authors decided to forgo the old convention of alphabetical ordering of authors in favor of a randomized ordering , denoted by r© . The publicly verifiable record of the randomization is available at https : //www.aeaweb.org/journals/policies/random-author-order/search with confirmation code : icoxrj0sNB2L . For citation of the work , authors request that the citation guidelines by AEA for random author ordering be followed . 35th Conference on Neural Information Processing Systems ( NeurIPS 2021 ) . encode non-identical distributions that are nonetheless close enough for practical purposes . Such situations may arise due to the use of approximate PC compilation [ 10 ] and sampling-based learning of PCs [ 26 , 27 ] . As a concrete example , consider PCs that are learned via approximate methods such as stochastic gradient descent [ 27 ] . In such a case , PCs are likely to converge to close but non-identical distributions . Given two such PCs , we would like to know whether they have converged to distributions close to each other . Thus , we raise the question : Does there exist an efficient algorithm to test the closeness of two PC distributions ? In this work , we design the first closeness test for PCs with respect to TV distance , called Teq . Assuming the tested PCs allow poly-time approximate weighted model counting and sampling , Teq runs in polynomial time . Formally , given two PC distributions P and Q , and three parameters ( ε , η , δ ) , for closeness ( ε ) , farness ( η ) , and tolerance ( δ ) , Teq returns Accept if dTV ( P , Q ) ≤ ε and Reject if dTV ( P , Q ) ≥ η with probability at least 1 − δ. Teq makes atmost O ( ( η − ε ) −2 log ( δ−1 ) ) calls to the sampler and exactly 2 calls to the counter . Teq builds on a general distance estimation technique of Canonne and Rubinfeld [ 4 ] that estimates the distance between two distributions with a small number of samples . In the context of PCs , the algorithm requires access to an exact sampler and an exact counter . Since not all PCs support exact sampling and counting , we modify the technique presented in [ 4 ] to allow for approximate samples and counts . Furthermore , we implement and test Teq on a dataset of publicly available PCs arising from applications in circuit testing . Our results show that closeness testing can be accurate and scalable in practice . For some NNF fragments , such as DNNF , no sampling algorithm is known , and for fragments such as PI , sampling is known to be NP-hard [ 32 ] . Since Teq requires access to approximate weighted counters and samplers to achieve tractability , the question of determining the closeness of the PCs mentioned above remains unanswered . Thus , we investigate further and characterize the complexity of closeness testing for a broad range of PCs . Our characterization reveals that PCs from the fragments d-DNNFs and SDNNFs can be tested for closeness in poly-time via Teq , owing to the algorithms of Darwiche [ 11 ] and Arenas et al . [ 1 ] . We show that the SDNNF approximate counting algorithm of Arenas et al . [ 1 ] can be extended to log-linear SDNNFs using chain formulas [ 8 ] . Then , using previously known results , we also find that there are no poly-time equivalence tests for PCs from PI and DNNF , conditional on widely believed complexity-theoretic conjectures . Our characterization also reveals some open questions regarding the complexity of closeness and equivalence testing of PCs . The rest of the paper is organized in the following way . We define the notation and discuss related work in Section 2 . We then present the main contribution of the paper , the closeness test Teq , and the associated proof of correctness in Section 3 . We present our experimental findings in Section 4 , and then discuss the complexity landscape of closeness testing in Section 5 . We conclude the paper and discuss some open problems in Section 6 . Due to space constraints , we defer some proofs to the supplementary Section A . 2 Background . Let ϕ : { 0 , 1 } n → { 0 , 1 } be a circuit over n Boolean variables . An assignment σ ∈ { 0 , 1 } n to the variables of ϕ is a satisfying assignment if ϕ ( σ ) = 1 . The set of all satisfying assignments of ϕ is Rϕ . If |Rϕ| > 0 , then ϕ is said to be satisfiable and if |Rϕ| = 2n , then ϕ is said to be valid . We use |ϕ| to denote the size of circuit ϕ , where the size is the total number of vertices and edges in the circuit DAG . The polynomial hierarchy ( PH ) contains the classes ΣP1 ( NP ) and Π P 1 ( co-NP ) along with generalizations of the form ΣPi and Π P i where Π P i+1 = co-NP ΠPi and ΣPi+1 = NP ΣPi [ 34 ] . The classes ΣPi and ΠPi are said to be at level i . If it is shown that two classes on the same or consecutive levels are equal , the hierarchy collapses to that level . Such a collapse is considered unlikely , and hence is used as the basic assumption for showing hardness results , including the ones we present in the paper . 2.1 Probability distributions . A weight function w : { 0 , 1 } n → Q+ assigns a positive rational weight to each assignment σ . We extend the definition of w to also allow circuits as input : w ( ϕ ) = ∑ σ∈Rϕ w ( σ ) . For weight function w and circuit ϕ , w ( ϕ ) is the weighted model count ( WMC ) of ϕ w.r.t . w. In this paper , we focus on log-linear weight functions as they capture a wide class of distributions , including those arising from graphical models , conditional random fields , and skip-gram models [ 24 ] . Log-linear models are represented as literal-weighted functions , defined as : Definition 1 . For a set X of n variables , a weight function w is called literal-weighted if there is a poly-time computable map w : X → Q ∩ ( 0 , 1 ) such that for any assignment σ ∈ { 0 , 1 } n : w ( σ ) = ∏ x∈σ { w ( x ) if x = 1 1− w ( x ) if x = 0 For all circuits ϕ , and log-linear weight functions w , w ( ϕ ) can be represented in size polynomial in the input . Probabilistic circuits : A probabilistic circuit is a satisfiable circuit ϕ along with a weight function w. ϕ and w together define a discrete probability distribution on the set { 0 , 1 } n that is supported over Rϕ . We denote the p.m.f . of this distribution as : P ( ϕ , w ) ( σ ) = { 0 ϕ ( σ ) = 0 w ( σ ) /w ( ϕ ) ϕ ( σ ) = 1 In this paper , we study circuits that are fragments of the Negation Normal Form ( NNF ) . A circuit ϕ in NNF is a rooted , directed acyclic graph ( DAG ) , where each leaf node is labeled with true , false , v or ¬v ; and each internal node is labeled with a ∧ or ∨ and can have arbitrarily many children . We focus on four fragments of NNF , namely , Decomposable NNF ( DNNF ) , deterministic-DNNF ( d-DNNF ) , Structured DNNF ( SDNNF ) , and Prime Implicates ( PI ) . For further information regarding circuits in NNF , refer to the survey [ 14 ] and the paper [ 28 ] . The TV distance of two probability distributions P ( ϕ1 , w1 ) and P ( ϕ2 , w2 ) over { 0 , 1 } n is defined as : dTV ( P ( ϕ1 , w1 ) , P ( ϕ2 , w2 ) ) = 1 2 ∑ σ∈ { 0,1 } n |P ( ϕ1 , w1 ) ( σ ) − P ( ϕ2 , w2 ) ( σ ) | . P ( ϕ1 , w1 ) and P ( ϕ2 , w2 ) are said to be ( 1 ) equivalent if dTV ( P ( ϕ1 , w1 ) , P ( ϕ2 , w2 ) ) = 0 , ( 2 ) ε-close if dTV ( P ( ϕ1 , w1 ) , P ( ϕ2 , w2 ) ) ≤ ε , and ( 3 ) η-far if dTV ( P ( ϕ1 , w1 ) , P ( ϕ2 , w2 ) ) ≥ η . Our closeness testing algorithm Teq , assumes access to an approximate weighted counter Awct ( α , β , ϕ , w ) , and an approximate weighted sampler Samp ( α , β , ϕ , w ) . We define their behavior as follows : Definition 2 . Awct ( α , β , ϕ , w ) takes a circuit ϕ , a weight function w , a tolerance parameter α > 0 and a confidence parameter β > 0 as input and returns the approximate weighted model count of ϕ w.r.t . w such that Pr [ w ( ϕ ) 1 + α ≤ Awct ( α , β , ϕ , w ) ≤ ( 1 + α ) w ( ϕ ) ] ≥ 1− β Tractable approximate counting algorithms for PCs are known as Fully Polynomial Randomised Approximation Schemes ( FPRAS ) . The running time of an FPRAS is given by T ( α , β , ϕ ) = poly ( α−1 , log ( β−1 ) , |ϕ| ) . Definition 3 . Samp ( α , β , ϕ , w ) takes a circuit ϕ , a weight function w , a tolerance parameter α > 0 and a confidence parameter β > 0 as input and returns either ( 1 ) a satisfying assignment σ sampled approximately w.r.t . weight function w with probability ≥ 1− β or ( 2 ) a symbol ⊥ indicating failure with probability < β . In other words , whenever Samp samples σ : P ( ϕ , w ) ( σ ) 1 + α ≤ Pr [ Samp ( α , β , ϕ , w ) = σ ] ≤ ( 1 + α ) P ( ϕ , w ) ( σ ) Tractable approximate sampling algorithms for PCs are known as Fully Polynomial Almost Uniform Samplers ( FPAUS ) . The running time of an FPAUS for a single sample is given by T ( α , β , ϕ ) = poly ( α−1 , log ( β−1 ) , |ϕ| ) . In the rest of the paper [ m ] denotes the set { 1 , 2 , . . .m } , 1 ( e ) represents the indicator variable for event e , and E ( v ) represents the expectation of random variable v .

Summary: This paper presents a statistical closeness test for evaluating whether the total variation distance between the probability distributions encoded by two circuits is below a lower threshold or above an upper threshold with a given confidence in polynomial time. The authors introduce the setting, present the algorithm and give a sample of the analysis by proving parts of the main result. The results are evaluated on public datasets.