Approximating the Permanent with Deep Rejection Sampling

1 Introduction . The permanent of an n× n matrix A = ( aij ) is defined as perA : = ∑ σ a ( σ ) , with a ( σ ) : = n∏ i=1 aiσ ( i ) , ( 1 ) where the sum is over all permutations σ on { 1 , 2 , . . . , n } . The permanent appears in numerous applications in various domains , including communication theory [ 31 ] , multi-target tracking [ 37 , 23 ] , permutation tests on truncated data [ 8 ] , and the dimer covering problem [ 2 ] . Bipartite graphs have the well-known property that the permanent of an adjancency matrix of such a graph is equal to the number of its perfect matchings . Finding the permanent is a notorious computational problem . The fastest known exact algorithms run in time O ( 2nn ) [ 29 , 15 ] , and presumably no polynomial-time algorithm exists , as the problem is # P-hard [ 38 ] . If negative entries are allowed , just deciding the sign of the permanent is equally hard [ 21 ] . For matrices with nonnegative entries , a fully polynomial-time randomized approximation scheme , FPRAS , was discovered two decades ago [ 21 ] , and later the time requirement was lowered to O ( n7 log4 n ) [ 3 ] . However , the high degree of the polynomial and a large constant factor render the scheme infeasible in practice [ 25 ] . Other , potentially more practical Godsil–Gutman [ 16 ] type estimators obtain high-confidence low-error approximations with O ( cn/2 ) evaluations of the determinant of an appropriate random n × n matrix over the reals [ 16 , 22 ] , complex numbers [ 22 ] , or quaternions [ 9 ] , with c equal to 3 , 2 , and 3/2 , respectively . These schemes might be feasible up to around n = 50 , but clearly not for n ≥ 100 . 35th Conference on Neural Information Processing Systems ( NeurIPS 2021 ) . From a practical viewpoint , however , ( asymptotic ) worst-case bounds are only of secondary interest : it would suffice that an algorithm outputs an estimate that , with high probability , is guaranteed to be within a small relative error of the exact value—no good upper bound for the running time is required a priori ; it is typically satisfactory that the algorithm runs fast on the instances one encounters in practice . The artificial intelligence research community , in particular , has found this paradigm attractive for problems in various domains , including probabilistic inference [ 1 , 7 ] , weighted model counting [ 12 , 11 ] , network reliability [ 26 ] , and counting linear extensions [ 35 ] . For approximating the permanent , this paradigm was recently followed by Kuck et al . [ 23 ] . Their AdaPart method is based on rejection sampling of permutations σ proportionally to the weight a ( σ ) , given in ( 1 ) . It repeatedly draws a uniformly distributed number between zero and an upper bound U ( A ) ≥ perA and checks whether the number maps to some permutation σ . The check is performed iteratively , by sampling one row–column pair at a time , rejecting the trial as soon as the drawn number is known to fall outside the set spanned by the permutations , whose measure is perA . The expected running time of the method is linear in the ratio U ( A ) /perA , motivating the use of a bound that is as tight as possible . Critically , the bound U is required to “ nest ” ( a technical monotonicity property we detail in Section 2 ) , constraining the design space . This basic strategy of AdaPart stems from earlier methods by Huber and Law [ 17 , 19 ] , which aimed at improved polynomial worst-case running time bounds for dense matrices . The key difference is that AdaPart dynamically chooses the next column to be paired with some row , whereas the method of Huber and Law proceeds in the static increasing order of columns . Thanks to this flexibility , AdaPart can take advantage of a tighter upper bound for the permanent that would not nest with respect to the static recursive partitioning . In this paper , we present a way to boost the said rejection samplers . Conceptually , the idea is simple : we replace the upper bound U ( A ) by a linear combination of the bounds for all submatrices that remain after removing the first d columns and the associated any d rows . Here the depth d is a parameter specified by the user ; the larger its value , the better the bound , but also the larger the time requirement of computing the bound . This can be viewed as an instantiation of a generic method we call deep rejection sampling or DeepAR ( deep acceptance–rejection ) . Our main observation is that for the permanent the computations can be carried out in time that scales , roughly , as 2d , whereas a straightforward approach would scale as nd , being infeasible for all but very small d. We demonstrate empirically that “ deep bounds ” , with d around 20 , are computationally feasible and can yield orders-of-magnitude savings in running time as compared to the basic depth-zero bounds . We also study analytically how the parameter d affects the ratio of the upper bound and the permanent . Following a series of previous works [ 22 , 13 , 14 ] , we consider random ( 0 , 1 ) -matrices where each entry takes value 1 with probability p , independently of the rest . We give a bound that holds with high probability and , when specialized to d = 0 and viewed as a function of n and p , closely resembles Huber ’ s [ 17 ] worst-case bound that holds whenever every row- and column-sum is exactly pn . We will compare the resulting time complexity bound of our approximation scheme to bounds previously proven for Godsil–Gutman type estimators [ 22 , 13 ] and some simpler Monte Carlo schemes [ 27 , 14 ] , and argue , in the spirit of Frieze and Jerrum [ 13 , Sec . 6 ] , that ours is currently the fastest practical and “ trustworthy ” scheme for random matrices . 2 Approximate weighted counting with rejection sampling . We begin with a generic method for exact sampling and approximate weighted counting called self-reducible acceptance–rejection . We also review the instantiations of the method to sampling weighted permutations due to Huber and Law [ 17 , 19 ] and Kuck et al . [ 23 ] . 2.1 Self-reducible rejection sampling . In general terms , we consider the following problem of approximate weighted counting . Given a set Ω , each element x ∈ Ω associated with a nonnegative weight w ( x ) , and numbers , δ > 0 , we wish to compute an ( , δ ) -approximation of w ( Ω ) : = ∑ x∈Ω w ( x ) , that is , a random variable that with probability at least 1− δ is within a factor of 1 + of the sum . The self-reducible acceptance–rejection method [ 17 ] solves the problem assuming access to ( i ) a partition tree of Ω , that is , a rooted tree where the root is Ω , each leaf is a singleton set , and the children of each node partition the node into two or more nonempty subsets ; ( ii ) an upper bound that nests over the partition tree , that is , a function u that associates each node S a number u ( S ) that equals w ( x ) when S = { x } , and is at least ∑n i=1 u ( Si ) when the children of S are S1 , S2 , . . . , Sn . The main idea is to estimate the ratio of w ( Ω ) to the upper bound u ( Ω ) by drawing uniform random numbers from the range [ 0 , u ( Ω ) ] and accept a draw if it lands in an interval of length w ( x ) spanned by some x ∈ Ω , and reject otherwise . The empirical acceptance rate is known [ 10 ] to yield an ( , δ ) -approximation of the ratio as soon as the number of accepted draws is at least ψ ( , δ ) : = 1 + 2.88 ( 1 + ) −2 ln ( 2/δ ) . The following function implements this scheme by calling a subroutine SAMPLE ( Ω , u ) , which makes an independent draw , returning 1 if accepted , and 0 otherwise . ( Huber ’ s [ 18 ] Gamma Bernoulli acceptance scheme , GBAS , reduces the required number of draws to around one third for practical values of and δ ; see Supplement A.1 . ) Function ESTIMATE ( Ω , u , , δ ) E1 k ← dψ ( , δ ) e , t← 0 , s← 0 E2 Repeat t← t+ 1 , s← s+ SAMPLE ( Ω , u ) until s = k E2 Return u ( Ω ) · k/t The partition tree and the nesting property are vital for implementing the sampling subroutine , enabling sequential random zooming from the ground set Ω to a single element of it . Function SAMPLE ( S , u ) S1 If |S| = 1 then return 1 ; else partition S into S1 , S2 , . . . , Sn S2 p ( i ) ← u ( Si ) /u ( S ) for i = 1 , 2 , . . . , n , p ( 0 ) ← 1− ∑n i=1 p ( i ) S3 Draw i ∼ p ( i ) S4 If i ≥ 1 then return SAMPLE ( Si , u ) ; else return 0 2.2 Application to approximating the permanent . Huber and Law [ 17 , 19 ] instantiate the method to approximating the permanent of an n× n matrix A with nonnegative entries by letting Ω be the set of all permutations on N : = { 1 , 2 , . . . , n } and letting the weight function w equal a . The recursive partitioning is obtained by simply branching on the row σ−1 ( j ) for each column j = 1 , 2 , . . . , n in increasing order . The bound u is derived from an appropriate upper bound U ( A ) ≥ perA as follows . Let S be the set of permutations that fix a bijection between rows I and columns J and denote the corresponding column for the row i by σ ( i ) . We get the upper bound at S by multiplying the upper bound for the permanent of the remaining matrix by the product of the already picked entries : u ( S ) : = ( ∏ i∈I aiσ ( i ) ) · U ( AĪJ̄ ) . ( 2 ) Here Ī and J̄ are the complement sets of I and J in relation to N , and indexing by subsets specifies a submatrix in an obvious manner . Note that u ( Ω ) = U ( A ) and u ( { σ } ) = a ( σ ) , provided that we let U ( A ) : = 1 when A vanishes , i.e. , has zero rows and columns . Various upper bound for the permanent are known . Let Ai denote the ith row vector of A , and let |Ai| denote the sum of its entries . For the permanent of a ( 0 , 1 ) -matrix , Minc conjectured [ 24 ] and Brègman proved [ 5 ] the Minc–Brègman bound UMB ( A ) : = n∏ i=1 γ ( |Ai| ) with γ ( 0 ) : = 0 and γ ( k ) : = ( k ! ) 1/k for k = 1 , 2 , . . . . An extension to arbitrary nonnegative weights is credited to Brouwer in Schrijver ’ s work [ 30 ] and published in corrected form by Soules [ 33 , cf . Footnote 4 ] . Letting a∗i1 ≥ a∗i2 ≥ · · · ≥ a∗in be the entries of Ai arranged into nonincreasing order , the Brouwer–Schrijver bound is given by UBS ( A ) : = n∏ i=1 n∑ k=1 a∗ik · [ γ ( k ) − γ ( k − 1 ) ] . One can verify that for a ( 0 , 1 ) -matrix the bound equals the Minc–Brègman bound . Since the Minc–Brègman bound does not yield , through ( 2 ) , a nesting upper bound u over the recursive column-wise partitioning , Huber and Law [ 19 ] introduced a somewhat looser upper bound , which has that desired property : UHL ( A ) : = n∏ i=1 h ( |Ai| ) e , h ( r ) : = { r + ( ln r ) /2 + e− 1 , if r ≥ 1 , 1 + ( e− 1 ) r , otherwise . ( 3 ) We will refer to this as the Huber–Law bound . In order to employ the tighter Brouwer–Schrijver bound , Kuck et al . [ 23 ] replaced the static columnwise partitioning by a dynamic partitioning , where the next column is selected so as to minimize the sum of the bounds of the resulting parts . More formally , if S is the set of permutations that fix a bijection between rows I and columns J , then S is partitioned according to a column j ∈ J̄ into sets Sij : = { σ ∈ S : σ−1 ( j ) = i } , i ∈ Ī , such that ∑ i∈Ī u ( Sij ) is minimized . Furthermore , if even the smallest sum exceeds the bound u ( S ) , then the partition is refined by replacing some set Sij by its minimizing partition ; this is repeated until the nesting condition is met . Kuck et al . report that the initial minimizing partition of S was always sufficient in their experiments ; this is vital for computational efficiency . Example 1 . It was left open whether the Brouwer–Schrijver bound is guaranteed to nest over the dynamic partitioning . The following matrix C is a counterexample , showing the answer is negative : C : = 1 1 1 10 0 1 11 1 0 0 1 1 1 1 . Indeed , we have UMB ( C ) = ( 4 ! ) 1/2 ( 2 ! ) = 4 √ 6 , but for any column j , the bounds of the three submatrices that remain after removing column j and row i with nonzero entry at ( i , j ) sum up to 2 ( 3 ! ) 1/3 ( 2 ! ) 1/2 ( 1 ! ) 1/1 + ( 3 ! ) 2/3 ( 2 ! ) 1/2 = ( 481/3 + 361/3 ) √ 2 > 2 √ 2 ( 48 · 36 ) 1/6 = 4 √ 6 . Here we used the fact that the inequality ( a+ b ) /2 ≥ √ a · b holds with equality only if a = b .

Summary: The paper gives a new algorithm to approximate permanents of non-negative matrices. The method builds upon the well-known rejection sampling method by Huber-Law (SODA'08). The algorithm is experimentally shown to perform well, and in particular better than the related result by Kuck et al. (NeurIPS'19). Finally, a mathematical analysis of the performance of the algorithm on a family of random matrices is given, where it is shown to run in polynomial time to yield arbitrarily good precision.