Approximating the Permanent with Deep Rejection Sampling

1 Introduction . The permanent of an n× n matrix A = ( aij ) is defined as perA : = ∑ σ a ( σ ) , with a ( σ ) : = n∏ i=1 aiσ ( i ) , ( 1 ) where the sum is over all permutations σ on { 1 , 2 , . . . , n } . The permanent appears in numerous applications in various domains , including communication theory [ 31 ] , multi-target tracking [ 37 , 23 ] , permutation tests on truncated data [ 8 ] , and the dimer covering problem [ 2 ] . Bipartite graphs have the well-known property that the permanent of an adjancency matrix of such a graph is equal to the number of its perfect matchings . Finding the permanent is a notorious computational problem . The fastest known exact algorithms run in time O ( 2nn ) [ 29 , 15 ] , and presumably no polynomial-time algorithm exists , as the problem is # P-hard [ 38 ] . If negative entries are allowed , just deciding the sign of the permanent is equally hard [ 21 ] . For matrices with nonnegative entries , a fully polynomial-time randomized approximation scheme , FPRAS , was discovered two decades ago [ 21 ] , and later the time requirement was lowered to O ( n7 log4 n ) [ 3 ] . However , the high degree of the polynomial and a large constant factor render the scheme infeasible in practice [ 25 ] . Other , potentially more practical Godsil–Gutman [ 16 ] type estimators obtain high-confidence low-error approximations with O ( cn/2 ) evaluations of the determinant of an appropriate random n × n matrix over the reals [ 16 , 22 ] , complex numbers [ 22 ] , or quaternions [ 9 ] , with c equal to 3 , 2 , and 3/2 , respectively . These schemes might be feasible up to around n = 50 , but clearly not for n ≥ 100 . 35th Conference on Neural Information Processing Systems ( NeurIPS 2021 ) . From a practical viewpoint , however , ( asymptotic ) worst-case bounds are only of secondary interest : it would suffice that an algorithm outputs an estimate that , with high probability , is guaranteed to be within a small relative error of the exact value—no good upper bound for the running time is required a priori ; it is typically satisfactory that the algorithm runs fast on the instances one encounters in practice . The artificial intelligence research community , in particular , has found this paradigm attractive for problems in various domains , including probabilistic inference [ 1 , 7 ] , weighted model counting [ 12 , 11 ] , network reliability [ 26 ] , and counting linear extensions [ 35 ] . For approximating the permanent , this paradigm was recently followed by Kuck et al . [ 23 ] . Their AdaPart method is based on rejection sampling of permutations σ proportionally to the weight a ( σ ) , given in ( 1 ) . It repeatedly draws a uniformly distributed number between zero and an upper bound U ( A ) ≥ perA and checks whether the number maps to some permutation σ . The check is performed iteratively , by sampling one row–column pair at a time , rejecting the trial as soon as the drawn number is known to fall outside the set spanned by the permutations , whose measure is perA . The expected running time of the method is linear in the ratio U ( A ) /perA , motivating the use of a bound that is as tight as possible . Critically , the bound U is required to “ nest ” ( a technical monotonicity property we detail in Section 2 ) , constraining the design space . This basic strategy of AdaPart stems from earlier methods by Huber and Law [ 17 , 19 ] , which aimed at improved polynomial worst-case running time bounds for dense matrices . The key difference is that AdaPart dynamically chooses the next column to be paired with some row , whereas the method of Huber and Law proceeds in the static increasing order of columns . Thanks to this flexibility , AdaPart can take advantage of a tighter upper bound for the permanent that would not nest with respect to the static recursive partitioning . In this paper , we present a way to boost the said rejection samplers . Conceptually , the idea is simple : we replace the upper bound U ( A ) by a linear combination of the bounds for all submatrices that remain after removing the first d columns and the associated any d rows . Here the depth d is a parameter specified by the user ; the larger its value , the better the bound , but also the larger the time requirement of computing the bound . This can be viewed as an instantiation of a generic method we call deep rejection sampling or DeepAR ( deep acceptance–rejection ) . Our main observation is that for the permanent the computations can be carried out in time that scales , roughly , as 2d , whereas a straightforward approach would scale as nd , being infeasible for all but very small d. We demonstrate empirically that “ deep bounds ” , with d around 20 , are computationally feasible and can yield orders-of-magnitude savings in running time as compared to the basic depth-zero bounds . We also study analytically how the parameter d affects the ratio of the upper bound and the permanent . Following a series of previous works [ 22 , 13 , 14 ] , we consider random ( 0 , 1 ) -matrices where each entry takes value 1 with probability p , independently of the rest . We give a bound that holds with high probability and , when specialized to d = 0 and viewed as a function of n and p , closely resembles Huber ’ s [ 17 ] worst-case bound that holds whenever every row- and column-sum is exactly pn . We will compare the resulting time complexity bound of our approximation scheme to bounds previously proven for Godsil–Gutman type estimators [ 22 , 13 ] and some simpler Monte Carlo schemes [ 27 , 14 ] , and argue , in the spirit of Frieze and Jerrum [ 13 , Sec . 6 ] , that ours is currently the fastest practical and “ trustworthy ” scheme for random matrices . 2 Approximate weighted counting with rejection sampling . We begin with a generic method for exact sampling and approximate weighted counting called self-reducible acceptance–rejection . We also review the instantiations of the method to sampling weighted permutations due to Huber and Law [ 17 , 19 ] and Kuck et al . [ 23 ] . 2.1 Self-reducible rejection sampling . In general terms , we consider the following problem of approximate weighted counting . Given a set Ω , each element x ∈ Ω associated with a nonnegative weight w ( x ) , and numbers , δ > 0 , we wish to compute an ( , δ ) -approximation of w ( Ω ) : = ∑ x∈Ω w ( x ) , that is , a random variable that with probability at least 1− δ is within a factor of 1 + of the sum . The self-reducible acceptance–rejection method [ 17 ] solves the problem assuming access to ( i ) a partition tree of Ω , that is , a rooted tree where the root is Ω , each leaf is a singleton set , and the children of each node partition the node into two or more nonempty subsets ; ( ii ) an upper bound that nests over the partition tree , that is , a function u that associates each node S a number u ( S ) that equals w ( x ) when S = { x } , and is at least ∑n i=1 u ( Si ) when the children of S are S1 , S2 , . . . , Sn . The main idea is to estimate the ratio of w ( Ω ) to the upper bound u ( Ω ) by drawing uniform random numbers from the range [ 0 , u ( Ω ) ] and accept a draw if it lands in an interval of length w ( x ) spanned by some x ∈ Ω , and reject otherwise . The empirical acceptance rate is known [ 10 ] to yield an ( , δ ) -approximation of the ratio as soon as the number of accepted draws is at least ψ ( , δ ) : = 1 + 2.88 ( 1 + ) −2 ln ( 2/δ ) . The following function implements this scheme by calling a subroutine SAMPLE ( Ω , u ) , which makes an independent draw , returning 1 if accepted , and 0 otherwise . ( Huber ’ s [ 18 ] Gamma Bernoulli acceptance scheme , GBAS , reduces the required number of draws to around one third for practical values of and δ ; see Supplement A.1 . ) Function ESTIMATE ( Ω , u , , δ ) E1 k ← dψ ( , δ ) e , t← 0 , s← 0 E2 Repeat t← t+ 1 , s← s+ SAMPLE ( Ω , u ) until s = k E2 Return u ( Ω ) · k/t The partition tree and the nesting property are vital for implementing the sampling subroutine , enabling sequential random zooming from the ground set Ω to a single element of it . Function SAMPLE ( S , u ) S1 If |S| = 1 then return 1 ; else partition S into S1 , S2 , . . . , Sn S2 p ( i ) ← u ( Si ) /u ( S ) for i = 1 , 2 , . . . , n , p ( 0 ) ← 1− ∑n i=1 p ( i ) S3 Draw i ∼ p ( i ) S4 If i ≥ 1 then return SAMPLE ( Si , u ) ; else return 0 2.2 Application to approximating the permanent . Huber and Law [ 17 , 19 ] instantiate the method to approximating the permanent of an n× n matrix A with nonnegative entries by letting Ω be the set of all permutations on N : = { 1 , 2 , . . . , n } and letting the weight function w equal a . The recursive partitioning is obtained by simply branching on the row σ−1 ( j ) for each column j = 1 , 2 , . . . , n in increasing order . The bound u is derived from an appropriate upper bound U ( A ) ≥ perA as follows . Let S be the set of permutations that fix a bijection between rows I and columns J and denote the corresponding column for the row i by σ ( i ) . We get the upper bound at S by multiplying the upper bound for the permanent of the remaining matrix by the product of the already picked entries : u ( S ) : = ( ∏ i∈I aiσ ( i ) ) · U ( AĪJ̄ ) . ( 2 ) Here Ī and J̄ are the complement sets of I and J in relation to N , and indexing by subsets specifies a submatrix in an obvious manner . Note that u ( Ω ) = U ( A ) and u ( { σ } ) = a ( σ ) , provided that we let U ( A ) : = 1 when A vanishes , i.e. , has zero rows and columns . Various upper bound for the permanent are known . Let Ai denote the ith row vector of A , and let |Ai| denote the sum of its entries . For the permanent of a ( 0 , 1 ) -matrix , Minc conjectured [ 24 ] and Brègman proved [ 5 ] the Minc–Brègman bound UMB ( A ) : = n∏ i=1 γ ( |Ai| ) with γ ( 0 ) : = 0 and γ ( k ) : = ( k ! ) 1/k for k = 1 , 2 , . . . . An extension to arbitrary nonnegative weights is credited to Brouwer in Schrijver ’ s work [ 30 ] and published in corrected form by Soules [ 33 , cf . Footnote 4 ] . Letting a∗i1 ≥ a∗i2 ≥ · · · ≥ a∗in be the entries of Ai arranged into nonincreasing order , the Brouwer–Schrijver bound is given by UBS ( A ) : = n∏ i=1 n∑ k=1 a∗ik · [ γ ( k ) − γ ( k − 1 ) ] . One can verify that for a ( 0 , 1 ) -matrix the bound equals the Minc–Brègman bound . Since the Minc–Brègman bound does not yield , through ( 2 ) , a nesting upper bound u over the recursive column-wise partitioning , Huber and Law [ 19 ] introduced a somewhat looser upper bound , which has that desired property : UHL ( A ) : = n∏ i=1 h ( |Ai| ) e , h ( r ) : = { r + ( ln r ) /2 + e− 1 , if r ≥ 1 , 1 + ( e− 1 ) r , otherwise . ( 3 ) We will refer to this as the Huber–Law bound . In order to employ the tighter Brouwer–Schrijver bound , Kuck et al . [ 23 ] replaced the static columnwise partitioning by a dynamic partitioning , where the next column is selected so as to minimize the sum of the bounds of the resulting parts . More formally , if S is the set of permutations that fix a bijection between rows I and columns J , then S is partitioned according to a column j ∈ J̄ into sets Sij : = { σ ∈ S : σ−1 ( j ) = i } , i ∈ Ī , such that ∑ i∈Ī u ( Sij ) is minimized . Furthermore , if even the smallest sum exceeds the bound u ( S ) , then the partition is refined by replacing some set Sij by its minimizing partition ; this is repeated until the nesting condition is met . Kuck et al . report that the initial minimizing partition of S was always sufficient in their experiments ; this is vital for computational efficiency . Example 1 . It was left open whether the Brouwer–Schrijver bound is guaranteed to nest over the dynamic partitioning . The following matrix C is a counterexample , showing the answer is negative : C : = 1 1 1 10 0 1 11 1 0 0 1 1 1 1 . Indeed , we have UMB ( C ) = ( 4 ! ) 1/2 ( 2 ! ) = 4 √ 6 , but for any column j , the bounds of the three submatrices that remain after removing column j and row i with nonzero entry at ( i , j ) sum up to 2 ( 3 ! ) 1/3 ( 2 ! ) 1/2 ( 1 ! ) 1/1 + ( 3 ! ) 2/3 ( 2 ! ) 1/2 = ( 481/3 + 361/3 ) √ 2 > 2 √ 2 ( 48 · 36 ) 1/6 = 4 √ 6 . Here we used the fact that the inequality ( a+ b ) /2 ≥ √ a · b holds with equality only if a = b .

Summary: Computing the matrix permanent is known as a #P-complete problem and no exact polynomial-time algorithm is known. Several works developed a randomized algorithm for approximating the permanent of a non-negative matrix based on recursive acceptance-rejection (AR) sampling and various upper bounds of permanent. This paper adopts a deep AR sampling scheme that skips some recursive steps. Then, the authors additionally propose an efficient way for computing skipped steps based on dynamic programming of permanent. Empirical results demonstrate order-of-magnitude savings of computational time under random synthetic and real-world datasets.