Perfect Sampling from Pairwise Comparisons
Abstract
In this work, we study how to efﬁciently obtain perfect samples from a discrete distribution D given access only to pairwise comparisons of elements of its support.
Speciﬁcally, we assume access to samples ( x, S ) , where S is drawn from a distribution over sets Q (indicating the elements being compared), and x is drawn from the conditional distribution D S (indicating the winner of the comparison) and aim to output a clean sample y distributed according to D .
We mainly focus on the case of pairwise comparisons where all sets S have size 2.
We design a Markov chain whose stationary distribution coincides with D and give an algorithm to obtain exact samples using the technique of Coupling from the Past.
However, the sample complexity of this algorithm depends on the structure of the distribution D and can be even exponential in the support of D in many natural scenarios.
Our main contribution is to provide an efﬁcient exact sampling algorithm whose complexity does not depend on the structure of D .
To this end, we give a parametric Markov chain that mixes signiﬁcantly faster given a good approximation to the stationary distribution.
We can obtain such an approximation using an efﬁcient learning from pairwise comparisons algorithm (Shah et al., JMLR 17, 2016).
Our technique for speeding up sampling from a Markov chain whose stationary distribution is approximately known is simple, general and possibly of independent interest.
Introduction
Machine Learning questions dealing with pairwise comparisons (PCs) constitute a fundamental and broad research area with multiple real-world applications [SBB + 16, Kaz11, LOAF12, SBGW16,
FH10, WJJ13].
For instance, the preference of a consumer to choose one product over another constitutes a pairwise comparison between the two products.
The two bedrocks of modern ML applications are inference/learning and sampling.
The area of learning from pairwise comparisons has a vast amount of work [RA14, SBGW16, DKNS01, NOS12, Hun04, JLYY11, APA18, LSR21,
HOX14, SBB + 16, VY16, NOS17, VYZ20].
In this problem, there exist n alternatives and an unknown weight vector D ∈ R n where D ( i ) is the weight of the i -th element with D ( i ) ≥and (cid:80) i ∈ [ n ] D ( i ) = 1 , i.e., D is a probability distribution over [ n ] .
Roughly speaking, the learner observes pairwise comparisons (or more generally k -wise comparisons) of the form ( w, { u, v } ) ∈ [ n ] × [ n ] 2 that compare the alternatives u (cid:54) = v and w equals either u or v indicating the winner of the comparison.
The most fundamental distribution over pairwise comparisons is the BTL model [BT52, Luc59] where the probability that the learner observes w = u when the items u, v are compared is D ( u ) /
( D ( u ) + D ( v )) .
The learner’s goal is mainly to estimate the underlying distribution D using a small number of pairwise comparisons.
In this work, we focus on questions concerning sampling, the second bedrock of modern ML.
In fact, we will deal with a fundamental area of the sampling literature, namely perfect sampling .
In contrast to approximate sampling, perfect (or exact) sampling from a probabilistic model requires an exact sample from this model.
Problems in this vein of research lie in the intersection of computer science and probability theory and have been extensively studied [PW96, PW98, Hub04, Hub16, FGY19,
JSS21, AJ21, GJL19, BC20, HSW21].
Our work poses questions concerning perfect sampling in the area of pairwise comparisons.
We ask the following simple yet non-trivial question: Is it possible to obtain a perfect sample x distributed as in D by observing only pairwise comparisons from D ?
Apart from the algorithmic and mathematical challenge of generating exact samples, perfect simulation has also practical motivation, noticed in various previous works [Hub16, JSS21, AJ21].
Crucially, its value does not stem from the fact that the output of the sampling distribution is perfect; usually, one can come up with a Markov chain whose evolution rapidly converges to the stationary measure.
Hence, the distance between the (approximate) sampling distribution from the desired one can become small efﬁciently.
This approximate sampling approach has a clear drawback: there is no termination criterion, i.e., the Markov chain has to be run for sufﬁciently long time and this requires an a priori known bound on the chain’s mixing time, which may be much larger than necessary.
On the contrary, perfect sampling algorithms come with a stopping rule and this termination condition can be attained well ahead of the worst-case analysis time bounds in practice.
Second, the question that we pose appears to have links with the important literature of truncated statistics [Gal97, DGTZ18].
Truncation is a common biasing mechanism under which samples from an underlying distribution D over X are only observed if they fall in a given set S ⊆ X , i.e., one only sees samples from the conditional distribution D S .
There exists a vast amount of applications that involve data censoring (that go back to at least [Gal97]) where one does not have direct sample access to D .
Recent work in this literature [FKT20] asked when it is possible to obtain perfect samples from some speciﬁc truncated discrete distributions (namely, truncated Boolean product distributions).
The area of pairwise comparisons can be identiﬁed as an extension of the truncated setting where the truncation set can change (i.e., each subset of alternatives corresponds to some truncation set); while perfect sampling has been a subject of interest in truncated statistics, the neighboring area of pairwise
Problem Formulation.
Let us deﬁne our setting formally (which we call Local Sampling Scheme).
This setting is standard (without this terminology) and can be found e.g., at [RA14].
Deﬁnition 1 ([RA14]) .
Let Z be a ﬁnite discrete domain.
Consider a target distribution D supported on Z and a distribution Q supported on subsets of Z .
The sample oracle Samp( Q ; D ) is called
Local Sampling Scheme (LSS) and each time it is invoked, it returns an example ( x, S )
such that: (i) the set S ⊆
Z is drawn from Q and (ii) x is drawn from the conditional distribution D S , where
D S ( x ) = D ( x ) 1 { x ∈ S } / D ( S ) and D ( S ) = (cid:80) x ∈ S D ( x ) .
For simplicity, we mostly focus on the case where Q is supported on a subset of Z ×Z , and often refer to Q as the pair distribution .
Then, Q naturally induces an (edge weighted) undirected graph G Q on Z , where { u, v } ∈ E ( G Q ) , if { u, v } is supported on Q , and has weight equal to the probability
Q ( u, v ) that { u, v } is drawn from Q .
When we deal with a pair distribution Q , the above generative model lies in the heart of the pairwise comparisons [FV86, Mar96, WJJ13].
The comparisons are generated according to the Bradley-Terry-Luce (BTL) model (where the comparisons have size 2 , [BT52, Luc59]): there is an unknown weight vector D ∈ R n + ( D ( u ) indicates the quality of item u ) and, for two items u, v , the algorithm observes the sample ( u, { u, v } ) , i.e., that u beats v with probability
D ( u ) D ( u )+ D ( v ) .
Motivated by the importance of perfect simulation, we ask the following question:
Question 1.
Is there an efﬁcient algorithm that draws i.i.d.
samples ( x, S ) from Samp( Q ; D ) and generates a single perfect sample y ∼ D ?
To put our contribution into context, let us ask another question whose answer is clear from previous work.
What is the sample complexity of learning D from a Local Sampling Scheme Samp( Q ; D )?
Here, the goal is to estimate D in some L p norm given such comparisons.
Some works focus on learning the re-parameterization z ∈ R n with z x = log ( D ( x )) .
Depending on the context, either the normalization condition (cid:104) 1 , D(cid:105)
= 1 or (cid:104) 1 , z (cid:105) = 0 is used.
[SBB + 16] deal with the problem of learning the re-parameterization z in the L 2 norm when | S | = 2 (pairwise comparisons) or | S | =
k ( k -wise comparisons with k = O (1) ).
The sample complexity for learning z in L 2 norm from pairwise comparisons is n/ ( λ ( Q ) (cid:15) 2 ) , where λ ( Q ) is the second smallest eigenvalue of a Laplacian matrix Q , induced by the pair distribution Q (the learning result is tight in a minimax sense for some appropriate semi-norm [SBB + 16]).
In particular, the matrix Q is deﬁned as Q xy =
−Q ( x, y ) and Q xx =
(cid:80) y
(cid:54) =
x Q ( x, y ) .
Hence, in the pairwise comparisons setting, the sample complexity of learning incurs an overhead associated with λ ( Q ) (Table 1).
The algorithm of [SBB + 16] is a pivotal component for our main result, as we will see later.
[APA18] provide a similar result for learning D in TV distance using a random-walk approach.
We remark that learning from pairwise comparisons requires some mild conditions concerning the sampling process (LSS in our case).
We review them shortly below.
[Table]
[Table]
Pairwise
[Table]
Conditions for Local Sampling Schemes.
We provide a standard pair of conditions for Local
Sampling Schemes (these or similar conditions are also needed in the learning problem).
We are interested in LSSs that satisfy both of these conditions.
However, some of our results still hold when only the ﬁrst condition is true; this will be more clear later.
Assumption 1 is a necessary information-theoretic condition for the pair distribution Q .
Assumption 1 (Identiﬁability) .
The support of the pair distribution Q of the Local Sampling Scheme contains a spanning tree, i.e., the induced graph G Q is connected.
We deﬁne E := E ( G Q ) .
Intuitively, the pair distribution Q needs to be supported on a spanning tree.
Equivalently, the associated Laplacian matrix Q should have positive Fiedler eigenvalue λ ( Q ) > 0 .
Assumption 2 is a condition about the support of Q and the target distribution D ; it allows efﬁcient learning of D from pairwise comparisons and is related to the difﬁculty of estimating small D -probabilities.
Assumption 2 (Efﬁciently Learnable) .
There exists a constant φ > 1 such that target distribution D satisﬁes 1 φ ≤ max ( x,y ) ∈E D ( x ) D ( y ) ≤ φ , where E is the support of the distribution Q .
From a dual perspective, Assumption 2 says that Q is supported only on edges where the two corresponding D -ratios are well controlled and implies that any conditional distribution D { x,y } has bounded variance, i.e., (1 /φ ) / (1 + φ ) 2 ≤
D ( x ) D ( y ) ( D ( x )+ D ( y ))
2 ≤ φ/ (1 + 1 /φ ) 2 .
A quite similar property has been proposed in the problem of learning from pairwise comparisons [SBB + 16].
Contributions and Techniques.
We provide an algorithmic answer to Question 1 regarding perfect sampling (under the above mild conditions) by showing the following:
There exists an efﬁcient algorithm that draws (cid:101) O ( n 2 /λ ( Q ))
samples from Samp( Q ; D ) and outputs a perfect sample x ∼ D .
The tool that makes this possible (and algorithmic) is a novel variant of the elegant Coupling from the Past (CFTP) algorithm of [PW96].
CFTP is a technique developed by Propp and Wilson ([PW96,
PW98]), that provides an exact random sample from a Markov chain, that has law the (unique) stationary distribution (we refer to Appendix A.2.3 for an introduction to CFTP).
To make our contribution clear, we provide two theorems; Informal Theorem 1 that directly and “naively” applies
CFTP (indicating the challenges behind efﬁcient sample correction) and a more sophisticated Informal
Theorem 2 that combines CFTP and learning from pairwise comparisons in order to get the result of Table 1.
We show how to efﬁciently generate “global” samples from the true distribution D from few samples from a Local Sampling Scheme Samp( Q ; D ) .
To this end, our novel algorithm combines the CFTP method [PW96, PW98], a remarkable technique that generates exact samples from the stationary distribution of a given Markov chain, with (i) an efﬁcient algorithm that estimates the parameters of Bradley-Terry-Luce ranking models from pairwise (and, in general, k -wise) comparisons [SBB + 16] and (ii) procedures based on Bernoulli factories [DHKN17, MP05, NP05] and rejection sampling.
At a technical level, we observe (Section 2) that a Local Sampling Scheme Samp( Q ; D ) naturally induces a canonical ergodic Markov chain on [ n ] with transition matrix M .
The probability of a transition from a state x to a state y is equal to the probability Q ( x, y ) , that the pair { x, y } is drawn from Q , times the probability D ( y ) /
( D ( x ) + D ( y )) , that y is drawn from the conditional distribution
D { x,y } .
Then, the unique stationary distribution of the resulting Markov chain coincides with the true distribution D .
Applying (a non-adaptive version of) the Coupling from the Past algorithm, we obtain the following result under Assumption 1 and Assumption 2 for a target distribution D over [ n ] .
For the matrix M , we let Γ( M ) be its absolute spectral gap (see Appendix A.1 for extensive notation).
Informal Theorem 1 (Direct Exact Sampling from LSS) .
With an expected number of (cid:101) O (cid:16) n 2 Γ( M ) (cid:17) samples from a Local Sampling Scheme Samp( Q ; D ) , there exists an algorithm (Algorithm 1) that generates a sample distributed as in D .
The sample complexity of Informal Theorem 1 is, for instance, attained by the instance of Figure 1, which we will discuss shortly.
1 We remark that Informal Theorem 1 holds more generally for Local
Sampling Schemes that only satisfy Assumption 1 and not Assumption 2; for a formal version of the above result, we refer to Theorem 2 and for a discussion on the resulting sample complexity, we refer also to Remark 1.
Let us now see why Informal Theorem 1 is quite unsatisfying.
In Informal Theorem 1, the matrix M contains the transitions induced by Samp( Q ; D ) and the transition u → v is performed with probability Q ( u, v ) D ( v ) /
( D ( u ) + D ( v )) and Γ( M ) is the absolute spectral gap of the Markov chain’s transition matrix M on which CFTP is performed.
Note that both M and Γ( M ) depend on Q and the target distribution D .
In many natural cases, e.g., if
D is multimodal or includes many low probability points, the CFTP-based algorithm of Informal
Theorem 1 can be quite inefﬁcient!
For instance, Figure 1 is a bad instance for the standard CFTP algorithm.
It corresponds to an instance where the support of Q is the path graph over [ n ] and the target distribution D is bimodal, satisfying Assumption 1 and Assumption 2 with φ = 2 .
In this case, M captures the transitions of a negatively biased nearest-neighbor random walk on the path, i.e., the probability of moving to the boundary is larger than moving to the interior.
The coalescence probability of the two extreme points is of order α n for some 0 < α < 1 .
This probability is connected to the maximum hitting time and, consequently, we get that the mixing time is Ω( α − n ) [LP17] and so the coalescence time is exponentially large.
Informal Theorem 1 depends on Γ( M ) and hence the exponential dependence on n is reﬂected in the conductance of the chain, which is exponentially small.
So, can we eliminate the bias due to Deﬁnition 1 efﬁciently?
As a technical contribution, we provide a novel exact sampling algorithm whose convergence does not depend on D .
We show how to improve the efﬁciency of CFTP by removing the dependence of its sample (and computational) complexity on the structure of the true distribution D .
In particular, in
Section 3, we show the following under Assumption 1 and Assumption 2:
Informal Theorem 2 (Exact Sampling from LSS using Learning) .
There exists an algorithm (Algorithm 2 & Algorithm 3) that draws an expected number of (cid:101) O (cid:16) n 2 λ ( Q ) (cid:17) samples from a Local Sampling
Scheme Samp( Q ; D ) , and generates a sample distributed as in D .
In the above, λ ( Q ) is the second smallest eigenvalue (a.k.a., the Fiedler eigenvalue) of Q , the weighted Laplacian matrix of the graph G Q induced by Q ; this is the same matrix that appears in the learning result of Table 1.
Crucially, the sample complexity in the above result is polynomial in n , provided that the pair distribution Q is reasonable.
For the path graph of Figure 1 with Q the uniform distribution, Informal Theorem 1 yields an exponential sample complexity but the spectrum of the matrix of Informal Theorem 2 is spec( Q ) = { 1 n Θ(1 − cos( πi/n )) } n − 1 i =0 .
Hence, since 1 − cos( x ) = sin 2 ( x/ 2) ≈ x 2 , we get that the sample complexity is poly( n ) .
Moreover, the runtime of the algorithm is sample polynomial.
Technical Overview.
A ﬁrst key idea towards establishing Informal Theorem 2 is to exploit learning in order to accelerate the convergence of CFTP.
We ﬁrst apply (a modiﬁcation of) the gradient-based algorithm of [SBB + 16] to the empirical log-likelihood objective, which estimates the parameters of BTL ranking models from PCs, with samples from Samp( Q ; D ) .
As a result of this learning algorithm (whose efﬁciency requires Assumption 2, as in [SBB + 16]), we get a probability distribution (cid:101) D , which approximates D with small relative error, in the sense that D ( x ) ∈ (1 ± (cid:15) ) · (cid:101) D ( x ) , for any x in D ’s support.
As we saw in Table 1, the sample complexity will be roughly of order
O ( n/ ( λ ( Q ) (cid:15) 2 ))) .
To be precise, this learning phase gives us a sample complexity term of order
O ( n 2 /λ ( Q )) and corresponds to a single execution of the above learning algorithm for the target D in relative error with accuracy (cid:15) =
Θ(1 / √ n ) (the accuracy’s choice will be more clear later).
The new idea is that we can use the output of the learning algorithm (cid:101) D and a Bernoulli factory mechanism to transform the Markov chain induced by Samp( Q ; D ) into an ergodic chain with almost uniform stationary probability distribution, whose transition probabilities (and mixing time) essentially do not depend on D !
To do so, for each pair of states x, y , with ( x, y ) ∈ E and (cid:101) D ( x ) > (cid:101) D ( y ) , we can use Bernoulli downscaling (which constitutes an essential building block for general Bernoulli factories, see e.g., [DHKN17, MP05, NP05]) to transform the (unknown) probability of a transition from y to x to D ( x ) /
( D ( x ) + D ( y )) · (cid:101) D ( y ) / (cid:101) D
( x ) (conditional that the edge ( x, y ) is drawn).
Since (cid:101) D ( x ) ≈ D ( x ) , for all states x , this makes the probability of a transition from x to y almost equal to the probability of a transition from y to x , which, in turn, implies that the stationary distribution of the modiﬁed Markov chain is almost uniform .
Thus, we speed up the CFTP algorithm by removing the modes due to the landscape of the target D and can get an exact sample from the stationary distribution of the modiﬁed Markov chain with O ( n log( n ) /λ ( Q )) samples.
Namely, the sample complexity of this parameterized variant of the CFTP algorithm does not depend on the structure of D anymore (of course, it does depend on the size n of D ’s support).
The above constitute our second key idea and the intuition behind this re-weighting comes from the method of simulated tempering [MP92, GLR18b, GLR18a].
Our third idea deals with the output of the algorithm.
Since we want an exact/clean sample from D , we cannot simply print the output of the above process.
As a last step, we use rejection sampling, guided by the distribution (cid:101) D , and apply the “reverse” modiﬁcation, bringing the distribution of the resulting sample back to D (see Algorithm 2).
With high probability, this last rejection sampling will succeed after Θ( n ) executions of the parameterized CFTP algorithm.
However, note that we do not have to execute the learning algorithm again.
Hence, the total sample complexity is equal to the cost of the learning phase (that is performed only once and its output is given as input to the parameterized
CFTP algorithm; see Algorithm 2) and the cost of the modiﬁed CFTP multiplied by Θ( n ) (due to rejection sampling), which yields a total of (cid:101) O ( n 2 /λ ( Q )) samples.
The above modiﬁcation of the
Markov chain induced by Samp( Q ; D ) , which improves possibly signiﬁcantly its mixing time, is simple and general and the parameterized variation of the standard CFTP algorithm (Algorithm 2) might be of independent interest.
In Appendix G, we discuss how to extend the analysis of the main
Algorithm 2 to sets of size larger than 2 .
Related Work.
Our results are related to and draw ideas from several areas of theoretical computer science, which we review below.
Exact Sampling.
Exact sampling comprises a well-studied ﬁeld [Ken05, Hub16] with numerous applications in theoretical computer science (e.g., to approximate counting [Hub98]).
One of our main tools, the Coupling From the Past algorithm, is the breakthrough result of [PW96, PW98], which established the applicability of perfect simulation, and made efﬁcient simulation of challenging problems (e.g., Ising model) feasible.
After this fundamental result, the literature of perfect simulation ﬂourished and various exact sampling procedures have been developed, see e.g., [PW96, GM99, Wil00, HS00, FH00, Hub04]), and the references therein.
Learning and Ranking from Pairwise Comparisons.
There is a vast amount of literature on ranking and parameter estimation from pairwise comparisons (see e.g., [FV86, Mar96, Cat12] and the references therein).
A signiﬁcant challenge in that ﬁeld is to come up with models that accurately capture uncertainty in pairwise comparisons.
The standard ones are the Bradley-Terry-Luce (BTL) model [BT52, Luc59] and the Thurstone model [Thu27].
Our work is closely related to previous work on the BTL model [RA14, SBGW16, DKNS01, NOS12, Hun04, JLYY11, APA18, LSR21] and our sample complexity rates depend on the Fiedler eigenvalue of a pairwise comparisons’ matrix, as in [HOX14, SBB + 16, VY16, NOS17, VYZ20].
Truncated Statistics.
Our work falls in the research agenda of efﬁcient learning and exact sampling from truncated samples [Bre96, Coh16].
Recent work has obtained efﬁcient algorithms for efﬁcient learning from truncated [DGTZ18, KTZ19, DGTZ19, IZD20, DRZ20, BDNP21, NP19, FKT20,
DKTZ21] and censored [MMS21, FKKT21, Ple21] samples.
Closer to our work, [FKT20] also deal with the question of exact sampling from truncated samples.
Correction Sampling.
Sample correction is a ﬁeld closely related to our work.
This area of research deals with the case where input data are drawn from a noisy or imperfect source and the goal is to design sampling correctors [CGR18] that operate as ﬁlters between the noisy data and the end user.
These algorithms exploit the structure of the distribution and make “on-the-ﬂy” corrections to the drawn samples.
Moreover, the problem of local correction of data has received much attention in the contexts of self-correcting programs, locally correctable codes, and local ﬁlters for graphs and functions (see e.g., [BLR93, SS10, JR13, ACCL04, BGJ + 10, GLT20]).
Conditional Sampling.
Conditional sampling deals with the adaptive learning analog of LSS, where the goal is again to learn an underlying discrete distribution from conditional/truncated samples, but the learner can change the truncation set on demand and is extensively studied [Can15, FJO + 15, ACK15b, BC18, ACK15a, GTZ17, KT19, CCK + 19].
An Exact Sampling Algorithm using CFTP
Consider a discrete target distribution D .
For pair distributions Q , the Local Sampling Scheme
Samp( Q ; D ) can be regarded as a graph G =
( V, E ) with V
= supp
( D ) and E
= supp( Q ) .
We let
V =
[ n ] .
Let M denote the canonical transition matrix of the Markov chain, associated with the oracle Samp( Q ; D ) .
The entries of M =
[ M xy ] x,y ∈ [ n ] are deﬁned as:
[Equation]
Observe that the transition from x to y is performed when both the edge { x, y } is drawn from Q , with probability proportional to Q ( x, y ) and y is drawn from the conditional distribution D { x,y } .
The Markov chain, whose transitions are described by M , has some notable properties: it has D as stationary distribution and is ergodic.
We can invoke the fundamental CFTP algorithm over the
Markov chain of [PW96, PW98] to obtain a perfect sample from D .
Algorithm
) (cid:46) Sample access to the LSS oracle Samp( Q ; D ) . 2: t ←3: F 0 ( x ) ← x , for any x ∈ [ n ] 4: while F t has not coalesced do (cid:46) While no coalescence has occured. 5: t ← t −6:
Draw sample ( i, j, w ) with ( i, j ) ∈ E and w ∈ { i, j } 7: for x = 1 . . .
n do (cid:46)
In order to update state x .
8: if x / ∈ { i, j } or x = w then F t ( x ) ← F t +1 ( x ) (cid:46)
Append F t in the past.
9: else F t ( x ) ← F t +1 ( w ) (cid:46) New transition with probability Q ( x, w )
D ( w ) D ( x )+ D ( w ) .
10: end 11: end 12:
Output F t (1) (cid:46) Output the perfect sample.
13: end procedure
We extensively discuss CFTP in Appendix A.2.3 for the interested reader.
CFTP yields the following (unsatisfying) result, whose proof can be found at the Appendix B.
Theorem 2 (Direct Exact Sampling from LSS) .
Let D min be the minimum value of the target distribution D .
Under Assumption 1 and for any δ > 0 , Algorithm 1 draws, with probability at least
[Equation]
[Equation]
log( 1 δ ) (cid:17) samples from a Local Sampling Scheme Samp( Q ; D ) over [ n ] , runs in time polynomial in N and outputs a sample distributed as in D .
Note that in the above result (Theorem 2), the conﬁdence parameter δ corresponds to the number of samples required and not the quality of the output sample (the sample is perfect with probability
We comment that the proof of the above Theorem is nearly straightforward and follows from the analysis of the CFTP algorithm; the purpose of Theorem 2 is to indicate that a direct application of CFTP would potentially lead to an exponential number of samples due to the shape of the target distribution.
Our main technical contribution begins in the next section: We give an exact sampler that surpasses the dependence on the target’s structure using learning and rejection sampling.
Improving the Exact Sampling Algorithm using Learning
The main drawback of Theorem’s 2 algorithm is that its sample complexity depends on both Q and
D and consequently in many natural scenaria the sample complexity may be exponentially large in n .
Next, we present Algorithm 2, which will allow us to remove the dependence on D for the Markov chain and reduce the sample complexity to polynomial in the domain size.
The crucial differences compared to Algorithm 1 are the orange lines.
Algorithm 2 is a parameterized extension of CFTP and its input is a vector p ∈ [0 , 1] n .
We remark that when p is the all-ones vector 1 , we obtain the previous Algorithm 1.
( x ) , otherwise 10: end 11: end 12: Draw C ∼ Be( p ( F t (1)))
(cid:46)
Be( p ) is a p -biased Bernoulli coin .
13: if C = 1 then Output F t (1) else Output ⊥ (cid:46) Output the perfect sample or Fail.
14: end procedure
We perform Algorithm 2 using as input parameter p an estimate (cid:101) D of the target D with (relative) error of order (cid:15)
= 1 / √ n (this estimate is obtained via Theorem 4 & Algorithm 5 using O ( n 2 /λ ( Q ))
samples from Samp( Q ; D ) , as we will see later).
Under the perspective of LSSs as random walks on an irreducible Markov chain, Algorithm 2 proceeds by executing CFTP.
Recall that in each draw from
Samp( Q ; D ) , a transition can be realized by the matrix M .
The fact that this transition depends on the target D is pathological, as we observed previously.
The algorithm, instead of performing this transition (as the non-adaptive CFTP algorithm of Theorem 2 does), performs a Bernoulli factory mechanism (downscaling), using the estimates (cid:101) D in order to make each transition almost uniform (see
Line 9 of Algorithm 2).
This change introduces some (known) bias to the random walk and changes the stationary distribution from D to an almost uniform one.
By performing the CFTP method, the algorithm iteratively reconstructs the past of the inﬁnite simulation, until all the simulations have coalesced at time t = 0 .
When coalescence occurs, the algorithm has an exact sample from the biased (almost uniform) stationary distribution.
Since the introduced bias is known (it corresponds to the inverse of the estimates (cid:101) D ), the algorithm accepts the sample using rejection sampling, guided by (cid:101) D , so that the ﬁnal sample is distributed according to the target distribution D (see Line 13 of Algorithm 2).
Speciﬁcally, we show (for the proof, see Appendix C) that:
Theorem 3 (Exact Sampling from LSS using Learning) .
Under Assumption 1 and Assumption 2, for any δ > 0 , there exists an algorithm (Algorithm 3) that draws, with probability at least 1 − δ ,
N = O (cid:0) n 2 log 2 ( n ) /λ
( Q ) ·
log(1 /δ ) (cid:1) samples from a Local Sampling Scheme Samp( Q ; D ) , runs in time polynomial in N , and outputs a sample distributed as in D .
We proceed with the analysis of Algorithm 3, which can be decomposed into two parts: the Learning
Step (Line 2) and several iterations of Algorithm 2 (Line 5).
1: procedure E XACT S AMPLER W ITH L EARNING ( δ ) (cid:46)
The algorithm of Theorem 3 .
2: (cid:101) D ← L EARN ( (cid:15) :
= 1 / √ n, δ ) (cid:46)
Learn D in relative error with Algorithm 5 (Theorem 4).
3: x ←⊥ 4: while x = ⊥ repeat 5: x ← E XACT S AMPLER ( (cid:101) D ) (cid:46) Call Algorithm 2.
Output x 7: end procedure
Learning Step.
For two distributions D , (cid:101) D with ground set [ n ] , we introduce the sequence/list (of length n ) 1 −D / (cid:101) D :=
(1 −D ( x ) / (cid:101) D ( x )) x ∈ [ n ] .
Observe that the pair of sequences (1 −D / (cid:101) D , 1 − (cid:101) D / D ) captures the relative error between the two distributions.
The sample complexity of the task of learning
D in (cid:15) -relative error is summarized by the following:
Theorem 4 (Learning Phase) .
For any (cid:15), δ > 0 , there exists an algorithm (Algorithm 5) that draws
N = O (cid:16) n λ ( Q ) (cid:15) 2 log( 1 δ ) (cid:17) samples from Samp( Q ; D ) satisfying Assumptions 1 and 2, runs in time polynomial in N , and, with probability at least 1 − δ , computes an estimate (cid:101) D of the target distribution
D , that satisﬁes the relative error guarantee max (cid:8) (cid:107) 1 − D / (cid:101) D(cid:107) ∞ , (cid:107) 1 − (cid:101) D / D(cid:107) ∞ (cid:9) ≤ (cid:15) .
We execute once the learning algorithm
(Algorithm 5) that estimates D in relative error
and we choose accuracy (cid:15) = 1 / √ n .
We would like to shortly comment on the choice of the accuracy value: An accuracy of constant order would still potentially yield a biased random walk.
After the re-weighting step, our goal is to show that the target distribution becomes almost uniform and that the absolute spectral gap of the reformed matrix is of order λ ( Q ) (see the proof of Lemma 15).
In order to ensure this, any accuracy that vanishes with n is sufﬁcient.
We chose to set (cid:15)
= 1 / √ n so that the complexity of learning ( n λ ( Q ) (cid:15) 2 ) matches that of the exact sampling procedure ( n 2 λ ( Q ) ) .
This learning step costs n 2 /λ ( Q ) samples and the estimate (cid:101) D is used as input to Algorithm 2.
For the proof of Theorem 4, we refer to the Appendix E. We can now move to Line 5.
Executing Algorithm 2.
We next focus on a single execution of the parameterized CFTP algorithm.
Our goal is to transform the Markov chain of the Local Sampling Scheme deﬁned in Equation (1) to a modiﬁed Markov chain with an almost uniform stationary distribution.
We are going to provide some intuition.
Applying Theorem 4, we can consider that, for any x ∈ [ n ] , there exists a coefﬁcient (cid:101) D ( x ) ≈ D ( x ) .
The idea is to use (cid:101) D ( x ) and make the stationary distribution of the modiﬁed chain close to uniform.
Intuitively, this transformation should speedup the convergence of the CFTP algorithm.
The downscaling method of Bernoulli factories gives us a tool to do this modiﬁcation.
We can implement the modiﬁed Markov chain via downscaling as follows: Consider an edge { x, y } with transition probability pair ( p xy , p yx ) .
Without loss of generality, we assume that (cid:101) D ( y ) >
(cid:101) D ( x ) (which intuitively means that we should expect that p xy > p yx ).
Then, the downscaler leaves p yx unchanged and reduces the mass of p xy to make the two transitions almost balanced.
Consider the LSS transition matrix M with M xy =
Q
( x, y )
D [Equation] ) D ( x )+ D ( y ) and M xx = 1 − (cid:80) y
(cid:54) =
x M xy .
Also, let (cid:101) D be an estimate for the distribution D (Theorem 4).
For the pair ( x, y ) , we modify the transition probability p xy
=
M xy , only if p xy > p yx , to be equal to the following: (cid:101) p xy = Q
( x, y )
D ( y ) D ( x )+ D ( y ) (cid:101) D ( x ) (cid:101) D ( y )
≈
Q ( x, y )
D ( x ) D ( x )+ D ( y ) =
p yx , where we use that (cid:101) D ( x ) ≈ D ( x ) and (cid:101) D
( y )
≈ D
( y ) .
The transition probability from x to y corresponds to a Bernoulli variable Be( p xy ) , which is downscaled by (cid:101) D ( x ) / (cid:101) D
( y ) <
The modiﬁed transition probability (cid:101) p xy can be implemented by drawing a Λ ∼
Be( (cid:101) D ( x ) / (cid:101) D
( y ))
and then drawing a P ∼ Be( p xy ) (from Samp( Q ; D ) ) and, ﬁnally, realizing the transition from x to y only if Λ P = 1 .
This implementation is valid since the two sources of randomness are independent.
So, we perform a downscaled random walk based on the transition matrix (cid:102) M with (cid:102) M xy = (cid:101) p
xy (the “small” probability p yx remained intact).
We call (cid:102)
M the (cid:101) D -scaling of M .
Lemma 5 summarizes the key properties of (cid:102) M .
The deﬁnition of the mixing time, used below, can be found at the notation Appendix A.1 and the full proof can be found at the Appendix D.
Lemma 5 (Properties of Rescaled Random Walk) .
Let D be a distribution on [ n ] and consider an (cid:15) -relative approximation (cid:101) D of D , as in Theorem 4 with (cid:15) = 1 / √ n .
Consider the transition matrix
M of the Local Sampling Scheme Samp( Q ; D ) (see Equation (1) ), and let (cid:102)
M be the (cid:101) D -scaling of
M .
Then, the transition matrix (cid:102) M (i) has stationary distribution (cid:101) π 0 with (cid:101)
π 0 ( x ) = Θ(1 /n ) for all x ∈ [ n ] and (ii) the mixing time of the transition matrix (cid:102)
M is T mix ( (cid:102) M ) =
O (cid:16) log( n ) λ ( Q ) (cid:17) .
Proof Sketch.
Set J xy
=
D
( x ) + D ( y ) .
For a pair ( x, y ) ∈ E , before the downscaling, the pairwise ( x, y )
-comparison corresponds to the random coin ( D ( y ) /J
xy , D ( x ) /J
xy ) and let D ( y ) > D ( x ) .
After the downscaling, this coin becomes (locally) almost fair, i.e., ( D ( x ) /J
xy + ξ xy , D ( x ) /J
xy ) .
Hence, the modiﬁed transition matrix (cid:102) M can be written as (cid:102) M = (cid:101) D ⊗ Q + Q ◦ [ ξ xy ] , where (cid:101) D is a symmetric matrix with (cid:101)
D xy = min {D ( x ) , D ( y ) } /
( D ( x ) + D ( y ))
= (cid:101) D yx , Q is the Laplacian matrix of the graph G Q and [ ξ xy ] is a matrix with entries ξ xy where ξ xy is only non-zero in the modiﬁed transitions x → y , i.e., if p xy > p yx then (cid:102) M xy =
Q ( x, y ) ·
( D ( x ) /J xy + ξ xy ) ; for this pair, we set ξ yx = 0 .
Finally, Q ◦ [ ξ xy ] denotes the modiﬁed Hadamard product 2 between Q and [ ξ xy ] .
Due to the accuracy (cid:15) of the learning algorithm, we have that | ξ xy | =
O (1 / √ n ) .
We analyze (i) the stationary distribution and (ii) the mixing time.
For Part (i) , the chain (cid:102) M remains irreducible and the detailed balance equations of (cid:102)
M satisfy (cid:101) p
xy / (cid:101) p yx = D ( y ) · (cid:101) D ( x ) /
( (cid:101) D ( y ) · D ( x )) .
Since (cid:101) D is an (cid:15) -relative approximation of D with (cid:15) =
O (1 / √ n ) , it holds that for any x ∈ [ n ] :
D ( x ) / (cid:101) D ( x ) ∈ [1 − (cid:15), 1 + (cid:15) ] .
Hence, the unique stationary distribution (cid:101) π 0 satisﬁes: (cid:101) π 0 ( x ) / (cid:101) π 0 ( y ) [-∈[
-]∈
[1 − Θ(1 / √ n ) , 1 + Θ(1 / √ n )] for any x, y ∈ [ n ] .
So, it must be the case that (cid:101) π
( x ) = Θ(1 /n ) .
For Part (ii) , in order to show that the mixing time is of order O (log( n ) /λ ( Q )) , it sufﬁces to show (by Lemma 9) that (i) the minimum value of the stationary distribution is O (1 /n ) and (ii) the absolute spectral gap Γ( (cid:102) M ) of (cid:102) M satisﬁes Γ( (cid:102) M ) =
Ω( λ ( Q )) , where λ ( Q ) is the minimum non-zero eigenvalue of the Laplacian matrix Q .
The ﬁrst property follows directly from the closeness of the stationary distribution of (cid:102) M to the uniform one.
We now focus on the latter.
Our goal is to control the absolute spectral gap Γ( (cid:102) M ) .
In fact, this matrix can be seen as a perturbed version of the matrix (cid:101) D ⊗ Q as we discussed in the beginning of the proof sketch.
The perturbation noise matrix E = Q ◦ [ ξ xy ] is a matrix whose entries contain (among others) the approximation errors [ ξ xy ] .
Hence, we can use Weyl’s inequality [Tao12] in order to control the absolute spectral gap of (cid:102) M .
This inequality gives a crucial perturbation bound for the singular values for general matrices.
Since the matrix (cid:102)
M can be seen as a perturbation (as we discussed previously), we can (informally speaking) use Weyl’s inequality to control the singular values After some algebraic manipulation, we derive that Γ( (cid:102) M )
=
Ω( λ ( Q )) .
So, a single iteration of the parameterized CFTP algorithm with transition matrix (cid:102)
M (see Line 5 of
Algorithm 3) guarantees that with an expected number of N = (cid:101) O
( nT mix ( (cid:102) M ))
= (cid:101) O
( n log( n ) /λ ( Q ))
samples, it reaches Line 12 with a sample y =
F t (1) that is generated by the parameterized CFTP algorithm (the iterations in Line 4).
We then have to study the quality of the output sample y .
Crucially, by the utility of the standard CFTP algorithm, the law of y is the normalized measure that is proportional to D / (cid:101) D , i.e., (cid:101) π 0 ( y ) =
( D ( y ) / (cid:101) D ( y )) / (cid:80) x ∈ [ n ] D ( x ) / (cid:101) D
( x ) (this is the stationary distribution of the modiﬁed random walk).
Hence, we cannot simply output y. For this reason, we perform rejection sampling.
Speciﬁcally, as we will see, a single execution of Algorithm 2 either produces a single sample distributed as in D with some known probability p or fails to output any sample with probability 1 − p (this is because we do not sample from D but from an almost uniform distribution).
If the rejection sampling process is successful, then the while loop (Line 4) of Algorithm 3 will break and the clean sample will be given as output.
One can show that Θ( n ) iterations of the parameterized CFTP algorithm sufﬁce to generate a perfect sample.
Hence, the total sample complexity is of order O ( n 2 log( n ) log( n/δ ) /λ
( Q ))
(at each step δ (cid:48)
=
Θ( δ/n )) .
We continue with a sketch behind the rejection sampling process of Line 13 of Algorithm 2.
Claim 1 (Sample Quality and Rejection Sampling) .
Algorithm 2, with input the output of the Learning
Phase, outputs a state x ∈ [ n ] with probability proportional to D ( x ) or outputs ⊥ .
Moreover,
Algorithm 3 outputs a perfect sample from D with probability 1.
To see this, set A = (cid:80) y ∈ [ n ] D ( y ) / (cid:101) D ( y ) .
At the end of Line 11, Algorithm 2 outputs x ∝
D ( x ) / (cid:101) D ( x ) , since the unique stationary distribution is (cid:101) π ( x ) =
( D ( x ) / (cid:101) D ( x )) /A and the utility of the CFTP guarantees that the outputs state follows the stationary distribution.
For the sample x , we perform a rejection sampling process, with acceptance probability (cid:101) D ( x ) .
Algorithm 2 has n +potential outputs: It either prints x ∈ [ n ] or ⊥ indicating failure.
The arbitrary sample x ∈ [ n ] is observed with probability (cid:101) π ( x ) · (cid:101) D ( x ) =
D ( x ) /A .
The remaining probability mass is assigned to ⊥ .
We claim that the output of Line 6 of Algorithm 3 has law D .
Observe that the whole stochastic process of Algorithm 3 outputs x ∈ [ n ] with probability (cid:80) ∞
i
=
Pr[Reject] i
Pr[ x is Accepted ] =
D ( x ) /A · (cid:80) ∞ i =0 Pr[Reject] i , where Pr[Reject] =
A (cid:80) y ∈ [ n ] D ( y ) (cid:101) D ( y ) (1 − (cid:101) D
( y ))
= 1 − 1 A ∈ (0 , 1) .
As a result, we get that the probability that x is the output of the above stochastic process is D ( x ) .
Conclusion
We close with an open question: Is the bound (cid:101)
O ( n 2 /λ ( Q )) tight?
The main difﬁculty is that it is not clear how to obtain a lower bound against any possible exact sampling algorithm.
We would like to mention that the dependence on 1 /λ ( Q ) is expected and intuitively unavoidable. This term is connected to the mixing time. Let us become more speciﬁc. The (sample) complexity of our proposed method is (cid:101) O ( n 2 /λ ( Q )) .
The term 1 /λ ( Q ) appears due to the mixing time of the (transformed)
Markov chain.
The complexity of CFTP is (cid:101) Θ( nT mix ) .
Hence, any CFTP-based exact sampler (or more broadly random walk based algorithm) will have that kind of spectral dependence.
Similarly, a dependence on the support’s size n is necessary in general.
Note that the structure of the target distribution D is crucial: if the target is uniform (or generally “ﬂat”), then a bound of order (cid:101) O ( n/λ ( Q )) is obtained by the naive CFTP method; for distributions with modes, our algorithm achieves quadratic dependence on n .
The sample complexity of a potential (not necessarily randomwalk based) exact sampling algorithm will depend on Q , but might achieve a sample complexity not directly dependent on λ ( Q ) .
The unclear part is our algorithm’s dependence on n .
While our algorithm attains a bound of (cid:101) O ( n 2 ) , it is not obvious whether an algorithm with sample complexity o ( n 2 ) exists.
Similarly, it is interesting to ask whether there are (efﬁcient) ways to transform the chain in order to get a faster sample corrector.
Our learning method ﬁnds a vector p (input of Algorithm 2) that makes the re-weighted distribution close to uniform.
Is it possible to ﬁnd another p (without our learning algorithm)
such that (i) when transforming the chain with p , we obtain a fast mixing chain; and (ii) the success probability in the rejection sampling step will be sufﬁciently high?
This work provided a theoretical understanding to perfect sampling from pairwise comparisons.
We believe it is an interesting direction to experimentally evaluate our proposed methodology.
We mention that this work is purely theoretical and has no negative societal impact.
Acknowledgments and Disclosure of Funding
We thank the anonymous reviewers for useful remarks and comments on the presentation of our manuscript.
This work was supported by the Hellenic Foundation for Research and Innovation (H.F.R.I.) under the “First Call for H.F.R.I. Research Projects to support Faculty members and
Researchers and the procurement of high-cost research equipment grant”, project BALSAM, HFRIFM17-1424.
Checklist
For all authors... (a) Do the main claims made in the abstract and introduction accurately reﬂect the paper’s contributions and scope?
[Yes] (b) Did you describe the limitations of your work?
[Yes] We have presented the assumptions of our models in Section 1.
We have also explained future research directions that will further improve this work in the Conclusion paragraph at the end of the main body.
(c) Did you discuss any potential negative societal impacts of your work?
[Yes] We have discussed that in the Conclusion paragraph at the end of the main body.
(d) Have you read the ethics review guidelines and ensured that your paper conforms to them?
[Yes] 2.
If you are including theoretical results... (a) Did you state the full set of assumptions of all theoretical results?
[Yes] We have speciﬁed the theoretical models we are working on, see the Problem Formulation in Section 1.
(b) Did you include complete proofs of all theoretical results?
[Yes] Due to space constraints we have included the full proofs of the results in the Appendix. 3.
If you ran experiments... (a) Did you include the code, data, and instructions needed to reproduce the main experimental results (either in the supplemental material or as a URL)?
[N/A] (b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)?
[N/A] (c) Did you report error bars (e.g., with respect to the random seed after running experiments multiple times)?
[N/A] (d) Did you include the total amount of compute and the type of resources used (e.g., type of GPUs, internal cluster, or cloud provider)?
[N/A] 4.
If you are using existing assets (e.g., code, data, models) or curating/releasing new assets... (a) If your work uses existing assets, did you cite the creators?
[N/A] (b) Did you mention the license of the assets?
[N/A] (c) Did you include any new assets either in the supplemental material or as a URL?
[N/A] (d) Did you discuss whether and how consent was obtained from people whose data you’re using/curating?
[N/A] (e) Did you discuss whether the data you are using/curating contains personally identiﬁable information or offensive content?
[N/A] 5.
If you used crowdsourcing or conducted research with human subjects... (a) Did you include the full text of instructions given to participants and screenshots, if applicable?
[N/A] (b) Did you describe any potential participant risks, with links to Institutional Review Board (IRB) approvals, if applicable?
[N/A] (c) Did you include the estimated hourly wage paid to participants and the total amount spent on participant compensation?
[N/A]
A Additional Notation and Preliminaries A.1 Notation
In this section, we provide the basic notation we are going to use in the technical part.
General Notation.
We deﬁne [ n ] := { 1 , . . . , n } .
We denote vectors with small bold letters x and matrices with bold letters P =
[ P xy ] .
For a vector x , we let x i
=
x
( i ) be its i -th entry.
For p ∈ { 1 , 2 , ∞} , we denote the L p norm by (cid:107) .
(cid:107) p .
For a matrix A , let (cid:107) A (cid:107) 2 denote its spectral norm (largest singular value).
We consider graphs G =
( V, E ) , with | V | = n vertices, whose associated
Laplacian matrix is denoted Λ ( G ) .
The edge set of the graph G is the set E ( G ) .
In Appendix A.2.2, we have included some basic deﬁnitions and facts about random walks and Markov chains.
We deﬁne the weighted Laplacian matrix of G Q for a distribution Q as
[Equation]
Distributions and Distances.
The probability simplex is denoted by ∆ n .
We let supp( D ) denote the support of a distribution D .
The total variation distance between two discrete distributions D 1 , Dover [ n ] is equal to TV( D 1 , D 2 ) = max S ⊆
[ n ] ( D 1 ( S ) − D 2 ( S ))
=
12 (cid:107)D 1 − D 2 (cid:107) 1 .
We let D ( S ) =
(cid:80) x ∈ S D ( x ) and D S is the conditional distribution on the set S , i.e., D S ( x )
= D ( x ) D ( S ) 1 { x ∈ S } .
The vector z ∈ R n with z i = log( D ( i )) is called the natural parameter vector of D .
Random Walks.
For a reversible transition matrix P , let its (real) spectrum be 1 = λ 1 ≥ λ 2 ≥ [Equation]
We deﬁne the absolute spectral gap of P to be the difference Γ( P ) = 1 − max { λ 2 ( P ) , | λ n ( P ) |} .
If P is aperiodic and irreducible, then Γ( P ) > 0 .
One could also deﬁne the spectral gap 1 − λ 2 ( P ) .
The two gaps are equal when the chain is lazy, i.e., when ∀ i ∈ [ n ] ,
P ii ≥ 1 / 2 .
We let λ ( L ) be the second smallest eigenvalue of a Laplacian matrix L , a.k.a., its Fiedler eigenvalue.
The Laplacian matrix L ∈ R n × n is positive semi-deﬁnite and induces a semi-norm on
R n with (cid:107) v (cid:107) L := √ v T Lv for v ∈ R n .
Recall that a semi-norm differs from a norm in that the semi-norm of a non-zero element is allowed to be zero.
In Algorithm 1, Algorithm 2 and Algorithm 7, we use the notation F t ( x ) ← F t +1 ( w ) for some t < 0 .
This means that we append the path of x at time t by adding the transition x → w and then the path of w from time t + 1 up to 0 .
Matrix Operators.
We denote by A (cid:12) B =
( A ij B ij ) 1 ≤ i ≤ j ≤ n the standard Hadamard matrix product and by A ◦ B a variation of the Hadamard matrix product, where the off-diagonal entries are equal to those of the standard Hadamard product A (cid:12) B , but the diagonal entries correspond to the diagonal matrix with entries ( − (cid:80)
j (cid:54) =
i A ij B ij ) i ∈ [ n ] .
Finally, we let A ⊗ B
=
I − A ◦ B .
A.2 Preliminaries
In this section, we provide some preliminaries and some useful tools about (i) concentration of random matrices, (ii) random walks and (iii) CFTP.
The reader may skip this section.
A.2.1 Random Matrices
We continue with some deﬁnitions for random matrices, needed for the proof of Lemma 17.
The following can be found at [Tro15].
Deﬁnition 6.
Let (Ω , F , µ ) be a probability space.
A random matrix X is a measurable map
X : Ω → M n 1 × n 2 .
The entries of X may be considered complex random variables that may or may not be correlated with each other.
A ﬁnite sequence { X i } of random matrices is independent when
[Equation]
for each k )
= (cid:89)
[Equation]
for any collection { F k } of Borel subsets of M n 1 × n 2 .
Proposition 7 (Hermitian Matrix Chernoff Bounds (see [Tro15])) .
Consider a ﬁnite sequence { X i } of m independent, random, Hermitian square matrices with common dimension n .
Assume that 0 ≤ λ min ( X i ) ≤ λ max ( X i ) ≤ M for any i ∈ [ m ] .
Let Y =
(cid:80) i X i .
Then, for (cid:15) ∈ [0 , 1) :
Pr (cid:104) λ min ( Y ) ≤ (1 − (cid:15) )
λ min ( E Y ) (cid:105) ≤ n ( e − (cid:15) / (1 − (cid:15) ) 1 − (cid:15) )
λ min ( E Y ) /M
≤ n exp (cid:16) − (cid:15) 2 λ min ( E Y ) / (2 M ) (cid:17) .
A.2.2 Random Walks and Markov Chains
This section is mostly based on [LP17] and we refer the reader to [LP17] for a thorough exposition.
Markov Chains.
Let Ω be a ﬁnite state space.
A Markov chain is a sequence of random variables
X 0 , X 1 , . . .
that satisfy the Markov property, i.e.,
[Equation]
A Markov chain is called time-homogeneous, if the RHS of the above equation does not depend of t .
Such a chain is associated with a transition matrix P =
{ P ( x, y ) } , where ( x, y ) ∈
Ω × Ω .
It holds that
[Equation]
A Markov chain is ergodic if there exists a time t ∈ N such that P ( t ) ( x, y ) > 0 for any x, y ∈ Ω , i.e., there exists a ﬁnite time t so that the probability of going from any vertex to any other in t steps is positive.
For ﬁnite state Markov chains, ergodicity is equivalent to irreducibility and aperiodicity.
A Markov chain is irreducible if for any two states x, y ∈ Ω , there exists a time step t such that
P ( t ) ( x, y ) > 0 and is aperiodic if, for any state x , it holds that gcd { t | P ( t ) ( x, x ) > 0 } = 1 .
Observe that a Markov chain with self-loops is aperiodic.
Stationary Distribution.
A stationary distribution π ∈ ∆ n , i.e., the ( n − 1) -dimensional simplex whose vertices are the n standard unit vectors, is deﬁned as the ﬁxed point of the transition matrix
P , that is π T P = π T .
An ergodic Markov chain has a unique stationary distribution π and, as t increases, it converges to it.
An interesting property of Markov chains is the reversibility.
A Markov chain is reversible if there exists a distribution π that satisﬁes the detailed balanced equations:
[Equation]
In this case, we can verify that π is a stationary distribution.
We can simply write π for the stationary distribution with associated probability vector π .
Mixing Time.
It is important to understand the convergence time of a Markov chain to its stationary distribution π .
A crucial random variable for convergence time is the mixing time .
Deﬁnition 8 (Mixing Time) .
Let 0 < (cid:15) < 1 / 2 .
Let M be an ergodic Markov chain on a ﬁnite state space Ω with stationary distribution π .
Then, the mixing time with accuracy (cid:15) of M equals:
[Equation]
where P x ( X t ) is the distribution of the state X t at time t for starting state x ∈ Ω .
For a reversible transition matrix P , let its spectrum be:
[Equation]
Note that λ 2 < 1 if and only if the chain is irreducible (exactly one connected component) and λ n > − 1 if
and only if the chain is aperiodic (e.g., not a bipartite graph).
We deﬁne the absolute spectral gap of P to be the difference: Γ( P ) = 1 − max {| λ 2 ( P ) | , | λ n ( P ) |} .
It holds that if P is aperiodic and irreducible, then Γ( P ) is strictly positive.
One could also deﬁne the spectral gap λ ( P ) = 1 − λ 2 ( P ) .
The two gaps are equal when the chain is lazy, i.e., when for any state x ∈ Ω , it holds that P ( x, x ) ≥ 1 / 2 .
Lemma 9 (Bounding T mix , see [LP17]) .
Let 0 < (cid:15) < 1 .
Assume that the transition matrix P is aperiodic, irreducible and reversible with respect to π .
Then, it holds that:
[Equation]
where t rel = 1 / Γ( P ) is the relaxation time of the Markov chain, i.e., the inverse absolute spectral gap.
Coalescencing Random Walks.
Let Ω =
[ n ] .
In a coalescing random walk, a set of n particles perform independent discrete time random walks on an undirected connected graph G =
( V, E ) with | V | = n , with each particle initially placed at a single (distinct) vertex x ∈ V .
In each time step, all particles move simultaneously.
Whenever two or more particles meet at a vertex, they unite to form a single particle which then continues to make a random walk through the graph.
The coalescence time
T coal is a random variable and is the ﬁrst time when all particles coalesce.
More formally, we can deﬁne coalescence as follows.
Deﬁnition 10 (Coalescence of Stochastic processes (see [Hub16])) .
Let A be a collection of stochastic processes, deﬁned over a common index set I and common state space Ω .
If there is an index i ∈ I and state x ∈ Ω such that, for all stochastic processes X ∈ A , it holds that X i
= x , then we say that the stochastic processes have coalesced.
Coalescence Time of Random Walks.
Consider the random variable X ( i ) t , that indicates the position of the i -th particle at time t .
The coalescence time is equal to:
[Equation]
for
[Equation]
We conclude this section with a folklore result, that deals with the possible locations in the complex plane of the eigenvalues of a square matrix A ∈ M n .
Lemma 11 (Geršgorin’s Theorem (see [HJ12])) .
Let A =
[ A ij ] ∈ M n .
For any i ∈ [ n ] , let
[Equation]
and consider the n Geršgorin disks
[Equation]
Then, the eigenvalues of A are in the union of the Geršgorin disks, i.e., spec( A ) ⊆
n (cid:91) i =
B i .
A.2.3 Exact Sampling and CFTP Preliminaries
Markov chain Monte Carlo (MCMC) methods constitute a class of algorithms for sampling from probability measures and arise naturally in various ﬁelds of science such as theoretical computer science (e.g., approximation algorithms for # P -complete problems (see [Jer03, JSV04])), statistical physics (e.g., in order to understand phase transition phenomena for Ising models) and statistics.
However, the theory of MCMC, in terms of exact sampling, can be seen as an asymptotic analysis (i.e., in the limit, the total variation distance vanishes).
Perfect simulation is analogous to MCMC and deals with techniques for designing algorithms that return exact draws from the target distribution, instead of long-time approximations.
Exact sampling comprises a well-studied ﬁeld (see [Ken05, Hub16]) and has numerous applications in computer science (e.g., in approximate counting [Hub98]).
The importance of exact simulation gave rise to various procedures in order to generate perfect samples.
A small sample of this line of research ([PW96, GM99, Wil00, HS00, FH00, Hub04]).
Coupling from the past (CFTP) is a technique developed by Propp and Wilson ([PW96, PW98]), that provides an exact random sample from a Markov chain, that has law the (unique) stationary distribution.
The algorithm assumes that a particle has been running on the Markov chain since time −∞ (arbitrarily long in the past) and we are concerned with the location of the particle at (the ﬁxed) time t = 0 .
The fact that the stopping time is deterministic and not a random variable (as in MCMC) is crucial.
Since the particle performs a random walk in the Markov chain inﬁnitely long, one would intuitively believe that the particle at time t = 0 is distributed according to the stationary distribution.
We deﬁne F ( t, 0] as the t -step evolution of the Markov chain from time t ∈ Z ≤ 0 to the ﬁxed time
T =
(evolution from the past ).
Note that we can decompose the evolution of the chain into t independent applications of the random function f , which encodes the information of the random walk, i.e., Pr[ f ( x ) =
y ]
= P xy for any pair of states x, y ∈ Ω .
Hence, we have that
[Equation]
end 9:
We can think of the data structure F ( t, 0] as a list of n values, each one capturing the evolution of the
Markov chain, initiated in a different state at time t and stopped at time 0 .
Similarly, the oracle call
Markov() corresponds to n adaptive calls, each one giving a transition from a ﬁxed state x ∈ Ω , where the next state is distributed according to P ( x, · ) .
Hence, f t is an n -dimensional vector with values in Ω .
We refer the reader to Figure 2 for an execution of the CFTP algorithm
The following fact is the main result behind CFTP.
Fact 1 ([PW96]) .
With probability 1 , the Coupling from the Past algorithm returns a value and this value is distributed according to the stationary distribution of the Markov chain.
The following proposition provides an upper bound on the convergence of the CFTP algorithm.
Proposition 12 ([ZCKM20]) .
Let π be the stationary distribution of an ergodic Markov chain P with n states.
We run n simulations of the chain each starting at a different state.
When two or more simulations coalesce, we merge them into a single simulation.
With probability at least 1 − δ , all n chains have merged after at most O ( nT mix ( P ; 1 / 4) log(1 /δ ))
iterations.
B The Proof of Theorem 2 (Direct Exact Sampling from LSS)
In this section, we provide the proof of Theorem 2.
Let us begin with the algorithm.
We remark that in the proof we will use the more technical notation F ( a,b ] to denote the evolution of the chain from a to b .
In Algorithm 1, this is abbreviated as F t to denote F ( t, 0] .
Proof.
At each iteration, the algorithm draws a single sample from Samp( Q ; D )
Then, the algorithm performs the following procedure for each state v ∈ [ n ] : let the drawn sample be ( i, j, w ) , where ( i, j ) ∈ E and w ∈ { 0 , 1 } indicating the winning node between i and j , i.e., the node i stays in i (if w = 1 ) or node i moves to j (if w = 0 ).
For any v ∈ [ n ] , the algorithm checks whether v ∈ { i, j } .
If not, the state v remains at v ; otherwise, assume that i = v .
Then, the transition v → j is implemented only if w = 0 , otherwise it remains at v .
We claim that this process simulates the transitions of the matrix M and has unique stationary distribution D .
The probability that the transition v → j occurs is
Pr[ [Equation] → [Equation]
= [Equation] [Equation] , j, w ) is drawn and w = 0] =
Pr[( v, j ) is drawn ] · Pr[ w = 0 | ( v, j ) is drawn ]
[Equation]
Hence, for v ∈ [ n ] and by the above process, the transition probabilities are given by the matrix M , that is a transition matrix with unique stationary distribution D .
We will see that this random process gives a perfect sample.
Consider the state space Ω
= [ n ] and let T be a positive integer.
Let ( X t ) t ∈
Z be a random walk on [ n ] with transition probabilities as in M (see Equation (1)).
Let F ( a,b ] : [ n ] → [ n ] be such that
F ( − t 0 ,T ] ( x 0 ) denotes the state of the walk at time t
=
T , i.e., X T ∈ [ n ] , when the random walk begun at time t = − t 0 (i.e., in the past ) with initial state X − t
= x 0 .
Speciﬁcally, for each t 0 , T , we have that F ( − t 0 ,T ] : [ n ] → [ n ] is a random map and F ( − t 0 ,T ] ( x 0 ) is a realization of the Markov chain, with initial state x 0 , executed from time − t 0 to time T .
We call F ( − t 0 ,T ] a stochastic ﬂow .
The CFTP method is initialized at time t = − t 0 into the past and places n particles (i.e., simulates n concurrent random walks), one at each state x ∈ [ n ] .
CFTP terminates when the stochastic ﬂow
F ( − t 0 , 0] is constant (it has a single point random image, i.e., F ( − t 0 , 0] ( x ) =
F ( − t 0 , 0] ( y ) , for all x, y ).
Namely, CFTP terminates when all the random walks that start at time t = − t 0 have coalesced at time t = 0 at a single z ∈ [ n ] .
Then, the algorithm outputs the (common) state z = F ( − t 0 , 0] ( x ) , for any x ∈ [ n ] .
Since the transitions of each random walk are performed with respect to M , the resulting sample z is distributed according to the stationary distribution D .
This can be shown as follows (see also [PW96, Hub16]): Let L ( X ) denote the distribution of the random variable X .
Since the chain is aperiodic and irreducible, coalescence occurs almost surely.
Speciﬁcally, assume that coalescence occurs when the simulation starts at a ﬁnite time t = − t 0 .
First, observe that if coalescence occurs at time − t 0 , then if CFTP starts at any time − t < − t 0 , we end up in the same state (because the randomness in [ − t 0 , 0] remains the same), i.e., F ( − t, 0] ( x ) =
F ( − t 0 , 0] ( x ) for any − t < − t 0 and any state x ∈ [ n ] .
Since, coalescence has occurred, we can work with a ﬁxed starting state x 0 and let
F ( − t, 0] =
F ( − t, 0] ( x 0 ) .
Hence, it holds that L ( F ( − t, 0] ) =
L ( F ( − t 0 , 0] ) and so
[Equation]
We note that the distribution of the time-backward coalescing random walk is equal to the distribution of the time-forward random walk, i.e., L ( F ( − t, 0] )
=
L ( F (0 ,t ] ) .
Thus, TV( L ( F ( − t 0 , 0] ) , D ) =
lim t →∞ TV( L ( F (0 ,t ] ) , D )
= 0 .
So, the output sample of the CFTP algorithm is distributed according to D .
Using Proposition 12, we get that, with high probability, coalescence occurs after
O ( nT mix ( M ; 1 / 4)) iterations.
From Lemma 9 with (cid:15) = 1 / 4 , we get that the mixing time of the
Markov chain is O (log(1 / D min ) /
Γ( M )) .
Finally, note that in order to perform a single iteration of the CFTP method, it requires to draw a single sample from the Local Sampling Scheme oracle.
This concludes the analysis of the sample complexity.
The time complexity is also polynomial in the number of samples [PW96].
Remark 1.
As we mentioned in the discussion after Informal Theorem 1, in order to obtain Theorem 2 only Assumption 1 is required.
Under only this assumption, a quite large amount of Local Sampling
Schemes is allowed and the resulting sample complexity is (cid:101) O (cid:16) n log(1 / D min )
Γ( M ) (cid:17) .
However, if we additionally assume that Assumption 2 holds, as we did in the statement of Informal Theorem 1, we accept only LSSs whose target distribution satisﬁes D min = Ω(2 − n ) .
This yields the (non-tight) bound (cid:101) O ( n 2 / Γ( M )) of Informal Theorem 1.
We preferred to omit this detail in Informal Theoremand emphasize only on the exponential dependence due to Γ( M ) for many natural instances.
C The Proof of Theorem 3 (Exact Sampling from LSS using Learning)
The main result of this section is the convergence analysis of the Coupling From the Past algorithm when the algorithm performs the rescaling transformation internally in each transition.
Theorem 13 (Convergence of CFTP) .
Assume that Assumption 1 and Assumption 2 both hold.
Algorithm 2, assuming access to a Local Sampling Scheme Samp( Q ; D ) with weighted Laplacian matrix Q (see Equation (2) ) associated with the pair distribution Q over [ n ] × [ n ] , satisﬁes the following:
( i ) Algorithm 2 reaches Line 12 with probability 1 .
( ii )
For any δ ∈ (0 , 1) , a single execution of of Algorithm 2 uses (cid:101) O [Equation] n log
[Equation]
n [Equation] log [Equation]
[Equation]
λ [Equation]
with probability 1 − δ and the running time is polynomial in the number of samples.
( iii ) Algorithm 2, with input the output of the Learning Phase, outputs a state x ∈ [ n ] with probability proportional to D ( x ) or outputs ⊥ .
Moreover, Algorithm 3 outputs a perfect sample from D .
Using the above properties, for any δ > 0 , Algorithm 3 draws, with probability at least 1 − δ ,
N = O (cid:0) n 2 log 2 ( n ) /λ
( Q ) ·
log(1 /δ ) (cid:1) samples from a Local Sampling Scheme Samp( Q ; D ) , runs in time polynomial in N , and outputs a sample distributed as in D .
Proof of Theorem 13.
Consider a target distribution D , supported on [ n ] and sample access to a
Local Sampling Scheme Samp( Q ; D ) for a pair distribution Q .
Let M = Q
⊗ D be the transition matrix of the Markov chain associated with the Local Sampling Scheme graph.
From Theoremwith accuracy (cid:15) > 0 , there exists an algorithm that, with high probability, computes the rescaling of the Markov chain that corresponds to M , which we denote by (cid:102) M , the transition matrix of the rescaled (by 1 / (cid:101) D )
Markov chain.
The learning algorithm uses (cid:101) O ( n 2 / ( λ ( Q )))
with probability 1 − δ and is executed only once.
We now analyze a single execution of Algorithm 2.
Claim 2 (Termination of CFTP) .
Algorithm 2 reaches Line 12 with probability 1 .
Proof.
Since the chain (cid:102) M is ergodic, there is a T such that for any pair of states ( x, y ) , the probability (cid:102) M T ( x, y ) > 0 .
Similarly to the proof sketch of the naive exact sampling algorithm (see also
Theorem 2), let ( X t ) t ∈
Z be a random walk on [ n ] and deﬁne the mapping F ( a,b ] : [ n ] → [ n ] , so that
F ( − t 0 ,T ] ( x 0 ) is the state of the random walk at time t
=
T , i.e., X T ∈ [ n ] , where the random walk begun at time t = − t 0 (in the past) with starting state X − t
= x 0 .
Then, the n random walks have coalesced after T steps if and only if | Image( F ( − T, 0] )
| = 1 .
By the ergodicity of the chain, each one of the events E T = {| Image( F ( − T, 0] ) | = 1 } for T ∈ Z ≥ 0 has strictly positive probability to occur.
Since the events are independent, with probability 1 , there will be an event E T (cid:63) that occurs.
Then, for any T > T (cid:63) , the desired property holds.
Claim 3 (Sample Complexity of a Single Iteration) .
It holds that each execution of Algorithm 2 uses
O (cid:16) [Equation]
(cid:17) samples from the Local Sampling Scheme with probability 1 − δ .
Proof.
The CFTP draws a single sample for each iteration and performs a step according to the matrix (cid:102)
M due to the rescaling.
First, the unique stationary distribution is almost uniform (Lemma 15) and so log(1 / D min ) = Θ(log( n )) .
This implies, by Lemma 9 and Lemma 15, that the mixing time of (cid:102) M for some constant accuracy (cid:15)
= 1 / 4 is T mix ( (cid:102) M ; 1 / 4) =
O (log( n ) /λ
( Q )) .
Hence, by
Proposition 12, we get the desired result.
Moreover, the update time of each execution of the CFTP algorithm is polynomial in n .
Note that, since during the Learning Phase, the algorithm uses O ( n 2 /λ ( Q )) and is executed once, the sample complexity of Algorithm 3 follows, since Algorithm 2 will be executed Θ( n ) times.
Claim 4 (Rejection Sampling) .
Algorithm 2, with input the output of the Learning Phase, outputs a state x ∈ [ n ] with probability proportional to D ( x ) or outputs ⊥ .
Moreover, Algorithm 3 outputs a perfect sample from D .
Proof.
Let us set A = (cid:80) y ∈ [ n ] D ( y ) / (cid:101) D ( y ) .
At the end of Line 11, the parameterized CFTP algorithm (Algorithm 2) outputs a sample x ∝ D ( x ) / (cid:101) D ( x ) .
This is because the unique stationary distribution is (cid:101) π
( x ) = ( D ( x ) / (cid:101) D ( x )) /A and the utility of the CFTP mechanism guarantees that when the Algorithm initiates a CFTP simulation from time T = − T (cid:63) , the generated sample x = F ( − T (cid:63) , 0] = F ( −∞ , 0] will be distributed according to (cid:101) π .
This follows from the observation that the distribution of the state F ( −∞ , 0] is equal to the distribution of the state F [0 , + ∞ ) obtained by running the simulation up to the limit T → + ∞ forward in time.
Hence, we have that x has law the unique stationary distribution (cid:101) π .
We now have to remove the bias induced by the learning step.
For the sample x , we perform a rejection sampling process, with acceptance probability (cid:101) D ( x ) .
The Algorithm 2 has n + 1 potential outputs: It either prints x ∈ [ n ] or ⊥ indicating failure.
The sample x is the output of the algorithm with probability D ( x ) / (cid:101) D ( x ) A · (cid:101)
D ( x ) =
D ( x ) /A .
This holds for any x ∈ [ n ] .
The remaining probability mass is assigned to ⊥ .
Hence, a single execution of Algorithm 2 outputs a point x ∈ [ n ] with probability D ( x ) /A
and outputs “reject” with probability 1 − 1 /A .
As we will see, the accepted sample is distributed according to D .
This is because, conditional on acceptance, its mass is exactly that assigned by D .
Speciﬁcally, we claim that the output of Line 6 of Algorithmhas law D .
Observe that the whole stochastic process of Algorithm 3 outputs x ∈ [ n ] with probability ∞ (cid:88) i = Pr[Reject [Equation] Pr
[Equation]
x is Accepted ] =
[Equation]
We have that Pr[Reject] = 1 A (cid:80) y ∈ [ n ] D ( y ) (cid:101)
[Equation]
( y ) [Equation] − (cid:101) D ( y ))
= 1 − 1 A ∈ (0 , 1) .
Hence, we have that the whole stochastic process of Algorithm 3 outputs x ∈ [ n ] with probability ∞ (cid:88) i = Pr[Reject] [Equation] Pr[ x is Accepted ]
[Equation]
, since (cid:80) i ≥ [-
-]0 λ i = [-
-]1 1 − λ for λ ∈ ( − 1 , 1) .
As a result, we get that the probability that x is the output of the above stochastic process is D ( x ) .
These claims complete the proof.
Speciﬁcally, the total sample complexity is derived as follows:
Since we have to execute the CFTP iterations Θ( n ) times, we should call each CFTP process with δ (cid:48)
=
Cδ/n for some constant C .
Proposition 12 guarantees that the CFTP process will terminate using O ( n log( n ) /λ ( Q ) · log( n/δ ))
samples with probability 1 − Θ( δ/n ) .
Hence, applying the the union bound for the Θ( n ) CFTP calls and the single call of the learning algorithm, gives that, with probability 1 − δ , a number of O ( n 2 log 2 ( n ) /λ ( Q ) · log(1 /δ )) samples sufﬁces in order to get a perfect sample from D .
D The Proof of Lemma 5 (Properties of Rescaled Random Walk)
In the next section, we discuss some useful steps as a warm-up for the proof of Lemma 5.
The proof can be found at the Appendix D.2.
D.1 Sketch of the Idea
By downscaling the transition probabilities (as we will see below), we can decouple the Markov chain from D .
Then, the transition matrix of the modiﬁed Markov chain only depends on the pair distribution Q , and the convergence of the CFTP algorithm is determined by λ ( Q ) .
Hence, we can sample efﬁciently even from target distributions D that may be multimodal or has many low probability points.
E.g., in case of a multimodal stationary distribution D , the spectral properties of the walk would remind these of a disconnected graph and the sample complexity of Algorithm 1 would be quite high.
We use the estimate (cid:101) D (which we obtained from our learning algorithm) of the target distribution D to transform the Markov chain of the Local Sampling Scheme deﬁned in Equation (1) to a modiﬁed Markov chain with an almost uniform stationary distribution.
This transformation can be viewed as a downscaling mechanism:
Deﬁnition 14 (Downscaling) .
Let p ∈
(0 , 1) and let X ∼ Be( p ) be a Bernoulli random variable.
Let λ ∈ (0 , 1) .
The random variable Y ∼ Be( λp ) is called a λ -downscaler of X .
Applying Theorem 4, we can consider that, for any x ∈ [ n ] , there exists a coefﬁcient (cid:101) D
( x ) ≈ D ( x ) .
The idea is to use (cid:101) D ( x ) and make the stationary distribution of the modiﬁed chain close to uniform.
Intuitively, this transformation should speedup the convergence of the CFTP algorithm.
Step 1.
Implementation of the downscaling and the rescaled matrix (cid:102) M .
We can implement the modiﬁed Markov chain via downscaling as follows: Consider an edge { x, y } with transition probability pair ( p xy , p yx ) .
Without loss of generality, we assume that (cid:101) D ( y ) >
(cid:101) D ( x ) (which intuitively means that we should expect that p xy > p yx ).
Then, the downscaler leaves p yx unchanged and reduces the mass of p xy to make the two transitions almost balanced.
Our exact sampling algorithm will perform this downscaling phase to the matrix M (see Equation (1)).
Consider the transition matrix M with M xy =
Q
( x, y )
[Equation]
( x )+ D ( y ) and M xx = 1 − (cid:80) y
(cid:54) =
x M xy .
Also, let (cid:101) D be an estimate for the distribution D with some sufﬁciently small accuracy (cid:15) , to be chosen (see also Theorem 4).
For the pair ( x, y ) , we modify the transition probability p
xy
=
M xy , only if p xy > p yx , to be equal to the following
[Equation]
where we use that (cid:101) D ( x ) ≈ D ( x ) and (cid:101) D ( y ) ≈ D ( y ) .
The transition probability from x to y corresponds to a Bernoulli variable Be( p xy ) , which is downscaled by (cid:101) D ( x ) / (cid:101) D
( y ) <
The modiﬁed transition probability (cid:101) p xy can be implemented by drawing a Λ ∼ Be( (cid:101) D ( x ) / (cid:101) D ( y )) and then drawing a P ∼ Be( p xy ) (from Samp( Q ; D ) ) and, ﬁnally, realizing the transition from x to y only if Λ P = 1 .
This implementation is valid since the two sources of randomness are independent.
The modiﬁed transition matrix (cid:102)
M can be written as:
[Equation]
where (cid:101) D is a symmetric matrix with (cid:101)
D xy = min {D ( x ) , D ( y ) } /
( D ( x )+ D ( y ))
= (cid:101) D yx and Q is the
Laplacian matrix of the graph associated with the Samp( Q ; D ) and Q ◦ [ (cid:15) xy ] denotes the modiﬁed
Hadamard product 3 between the Laplacian Q and the matrix with the estimation error (cid:15) xy is only nonzero in the modiﬁed transitions x → y , i.e., if p xy > p yx then (cid:102) M xy =
Q ( x, y ) · (
D ( x ) D ( x )+ D ( y ) + (cid:15) xy ) .
For this pair, we set (cid:15) yx = 0 .
We denote this error matrix with [ (cid:15) xy ] .
Speciﬁcally, for a pair ( x, y ) , before the downscaling, the pairwise ( x, y ) -comparison corresponds to the random coin (cid:16)
D ( y ) D ( x )+ D ( y ) ,
D ( x ) D ( x )+ D ( y ) (cid:17) and let D ( y ) > D ( x ) .
After the downscaling, this coin becomes (locally) almost fair, i.e., (
D ( x ) D ( x )+ D ( y ) + (cid:15) xy ,
D ( x ) D ( x )+ D ( y ) ) .
Step 2.
Obtaining a simpler matrix (cid:102) M with absolute spectral gap of the same order.
Since the coins are locally fair, we can work with the following matrix that has also almost uniform stationary distribution and conductance of the same order (since making each transition ( x, y ) more lazy (i.e., increasing the probability of x → x and y → y by some constant ) and still keeping the transitions x → y and y
→ x (almost) equal cannot signiﬁcantly affect the conductance, i.e., the desired symmetry is preserved).
Let (cid:102)
M
= I − c · Q + Q ◦ [ (cid:15) xy ] for some constant 0 ≤ c ≤ 1 / 2 .
This matrix with c = min ( x,y ) ∈E D ( x ) /
( D ( x ) + D ( y )) can be obtained by further performing downscaling at each transition (and c is constant due to Assumption 2); then each transition will be equal to the minimum global transition probability (potentially with some O (1 / √ n ) noise term).
For simplicity, we let c = 1 / 2 .
The next proof We get the following matrix
[Equation]
and
[Equation]
Also, we observe that the generator of the Markov chain I − (cid:102)
M is close up to scaling to the Laplacian
Q .
This explains why the convergence time of the Markov chain with transition matrix (cid:102) M spectrally depends only on the distribution Q .
Speciﬁcally, the Laplacian Q corresponds to the dominant component for the convergence of the algorithm and the Hadamard product is the low-order noise induced by the chain transformation.
As we will see, the larger the smallest non-zero eigenvalue of the Laplacian matrix and, hence, the larger the spectral gap of the transition matrix of the transformed chain, the faster the Markov chain converges to its stationary distribution.
D.2 The Proof of Lemma
The following lemma summarizes the key properties of the downscaled random walk.
Lemma 15 (Properties of Rescaled Random Walk) .
Let D be a distribution on [ n ] and consider an (cid:15) -relative approximation (cid:101) D of D , as in Theorem 4 with (cid:15) =
O (1 / √ n ) .
Consider the transition matrix
M of the Local Sampling Scheme Samp( Q ; D ) (see Equation (1) ), and let (cid:102)
M be the (cid:101) D -scaling of
M (see Equation (4) ).
Then, the following hold.
( i )
The transition matrix (cid:102) M has stationary distribution (cid:101)
π 0 ( x ) = Θ(1 /n )
for all x ∈ [ n ] .
( ii )
The absolute spectral gap Γ( (cid:102) M ) of (cid:102) M and the minimum non-zero eigenvalue λ ( Q ) of the Laplacian matrix Q satisfy Γ( (cid:102) M ) =
Ω( λ ( Q )) .
( iii )
For any (cid:15) 0 ∈ (0 , 1) , the mixing time of the transition matrix (cid:102)
M is T mix ( (cid:102) M ; (cid:15) 0 ) =
O (cid:16) log( n/(cid:15) 0 ) λ ( Q ) (cid:17) .
In the above, we can choose (cid:15)
= 1 / 4 .
The proof of the Lemma 15 follows.
Proof of Lemma 15.
We remark that it sufﬁces to work with the matrix of Equation (5), since the results will only change by at most some constant.
Observe that the spectral gap of the matrix of Equation (4) is at least as high as one of the matrix I − c · Q + Q ◦ [ (cid:15) xy ] with constant c = min ( x,y ) ∈E D ( x ) /
( D ( x ) + D ( y ))
=
Θ( φ ) and this matrix has spectral gap of the same order as the matrix of Equation (5).
We break the proof into three claims.
Claim 5.
The transition matrix (cid:102) M has stationary distribution (cid:101) π 0 , that satisﬁes (cid:101)
π 0 ( x ) = Θ(1 /n ) for any x ∈ [ n ] .
Proof.
The chain (cid:102)
M remains irreducible and the detailed balance equations of the matrix (cid:102)
M satisfy:
[Equation]
Since (cid:101) D is an (cid:15) -relative approximation of D with (cid:15) =
O (1 / √ n ) , it holds that for any x ∈ [ n ] :
[Equation]
and hence the unique stationary distribution (cid:101) π 0 satisﬁes: (cid:101) π 0 ( x ) / (cid:101) π 0 ( y ) ∈
[1 − (cid:15), 1 + (cid:15) ] for any x, y ∈ [ n ] .
So, this holds for x = argmax (cid:101) π
0 and y = argmin (cid:101)
π
and, since it should hold that (cid:80) x (cid:101)
π 0 ( x ) = 1 , it must be the case that (cid:101) π
( x ) = Θ(1 /n ) .
Claim 6.
For the absolute spectral gap Γ( (cid:102) M ) of (cid:102) M and the minimum non-zero eigenvalue λ ( Q ) of the Laplacian matrix Q , it holds that: Γ( (cid:102) M ) =
Ω( λ ( Q )) .
Proof.
For x (cid:54) =
y , there exist error estimates (cid:15) xy and (cid:15) yx ; one of them is zero and the other’s absolute value is of order O (1 / √ n ) .
Our goal is to control the absolute spectral gap Γ( (cid:102) M ) .
Recall that the transition matrix can be written as follows
[Equation]
where (cid:15) xy ∈ [ − (cid:15), (cid:15) ] for some (cid:15) =
O (1 / √ n ) .
Note that [ (cid:15) xy ] denotes the n × n error matrix and the operator ◦ denotes the standard Hadamard product.
Note that the matrix I − 12 Q is symmetric, while
Q ◦ [ (cid:15) xy ] needs not to be.
Let λ 1 > λ 2 ≥ · · · ≥ λ n be the spectrum of the transition matrix (cid:102) M .
Also, the matrix is right stochastic with (cid:102) M 1 = 1 and λ 1 = 1 .
Let Γ( (cid:102) M ) be the absolute spectral gap, that is strictly positive by aperiodicity and irreducibility.
The Weyl’s Inequality for general matrices ([HJ12, Tao12]) describes a multiplicative majorization between the ordered absolute eigenvalues and singular values of A and gives a useful perturbation bound.
Recall that σ 2 i ( A )
=
λ i ( AA ∗ )
=
λ i ( A ∗ A ) for an arbitrary matrix A ∈ C m × n .
Fact 2 (Weyl’s Inequality (see [HJ12])) .
Consider the matrices A , E ∈ C n × n .
Deﬁne the singular value of A in decreasing order (counting multiplicity) σ 1 ( A ) ≥ . . .
≥ σ n ( A ) ≥ 0 .
Then, for k = 1 , . . . ,
n , the following hold
• (cid:81) i ∈
[ k ] | λ i ( A ) | ≤ (cid:81) i ∈ [ k ] σ i
( A ) with | λ 1 ( A ) | ≥ | λ 2 ( A ) | ≥ . . .
| λ n ( A ) | .
• | σ k ( A + E ) − σ k
( A ) | ≤ (cid:107) E (cid:107) 2 .
For the matrix (cid:102) M , we have that | λ 1 | = 1 and the second largest absolute eigenvalue is max {| λ 2 | , | λ n |} .
We can apply the ﬁrst property for the singular values from the Fact 2 and get that
[Equation]
where σ 1 , σ 2 correspond to the two largest singular values.
Observe that the largest singular value is σ 1 ( (cid:102) M ) = max (cid:107) v
[Equation]
2 =1 (cid:107) (cid:102)
Mv (cid:107) =
1 and, using the second property of Fact 2, we can control the perturbation of the second largest singular value
[Equation]
In order to lower bound the absolute spectral gap, it sufﬁces to upper bound the second largest singular value.
But, for the second largest singular eigenvalue of the real symmetric matrix I − Q / 2 , it holds that
[Equation]
Since the matrix I − Q / 2 is symmetric, we have that
[Equation]
and, hence, if we let 0 = µ 1 ≤ µ 2 ≤ . . .
≤ µ n be the spectrum of the Laplacian matrix Q , we have that
[Equation]
For the Laplacian matrix Q , observe that | Q xx | = | (cid:80) y ∼ x Q ( x, y ) | ≤ 1 .
Hence, from Gershgorin’s
Theorem (see Lemma 11), we get that all the eigenvalues of the Laplacian lie on the real axis (since the matrix is symmetric) and all the disks B ( x, r ) are centered at points | x | ≤ 1 and the radii r are upper bounded by 1 .
Hence, it holds that 0 ≤ µ 2 ( Q ) ≤ µ n ( Q ) ≤ 2 .
So, we can take that σ 2 ( I − 12 Q )
=
− 12 µ 2 ( Q ) def = 1 − 12 λ ( Q ) ≥ 0 , where λ ( Q ) denotes the smallest non-zero eigenvalue of the Laplacian matrix Q .
Now, we have that (cid:107) Q ◦
[ (cid:15) xy ] (cid:107) 2 = max [Equation] v (cid:107) 2 =1 (cid:107) ( Q ◦ [ (cid:15) xy ]) v (cid:107) [-
-]2 = O (1 / √ n ) · µ n ( Q ) =
O (1 / √ n ) and so we get σ [Equation]
M ) ≤ [Equation] 1 − Θ( λ ( Q [Equation]
+ O (1 / √ n ) .
This implies that [Equation] and so
[Equation]
This implies that Γ( (cid:102) M ) =
Ω( λ ( Q )) for sufﬁciently large n .
The above proof will be similar for any matrix (cid:102)
M
= I − c · Q + Q · [ (cid:15) xy ] for 0 ≤ c ≤ 1 / 2 which is obtained by the discussion of Step 2 .
Such a matrix has an absolute spectral gap that is, on the one side, lower bounded by λ ( Q ) (as we showed above) and, on the other side, upper bounded by the absolute spectral gap of the original matrix of Equation (4).
This completes the proof.
We continue with the mixing time result of the transformed Markov chain.
Since we have shown that the absolute spectral gap is Ω( λ ( Q )) , we can directly get the desired result, whose proof relies on the analysis of Lemma 9 and can be found, e.g., in [LP17].
Claim 7.
The (cid:15) 0 -mixing time of the transition matrix (cid:102)
M is equal to
[Equation]
In the above claim, we choose (cid:15)
= 1 / 4 .
Combining these claims, the proof is completed.
E The Proof of Theorem 4 (Main Result of Learning Phase)
The ﬁrst step of Algorithm 3 is to learn the target distribution D in (cid:15)
-relative error for some (cid:15) >and pass it as input to Algorithm 2.
In this section, we will provide the learning results for abstract (cid:15) ; however, our algorithm applies these results with (cid:15) = 1 / √ n .
For two distributions D , (cid:101) D with ground set [ n ] , we introduce the sequence/list (of length n )
1 − D / (cid:101) D :=
(1 − D ( x ) / (cid:101) D ( x )) x ∈ [ n ] .
Observe that the pair of sequences (1 − D / (cid:101) D , 1 − (cid:101) D / D ) captures the relative error between the two distributions.
The sample complexity of the task of learning D in (cid:15) -relative
error is summarized by the following (restatement of Theorem 4):
Theorem.
For any (cid:15), δ > 0 , there exists an algorithm (Algorithm 5) that draws N =
O (cid:16) n λ ( Q ) (cid:15) 2 log( 1 δ ) (cid:17) samples from a Local Sampling Scheme Samp( Q ; D ) satisfying Assumptions 1 and 2, runs in time polynomial in N , and, with probability at least 1 − δ , computes an estimate (cid:101) D of the target distribution D , that satisﬁes the following relative error guarantee max (cid:8) (cid:107) 1 − D / (cid:101) D(cid:107) ∞ , (cid:107) 1 − (cid:101) D / D(cid:107) ∞ (cid:9) ≤ (cid:15) .
Let us sketch the proof of Theorem 4.
For an arbitrary weight vector w ∈ R n> 0 , we use the reparameterization z i = log( w i ) .
When in addition w ∈ ∆ n , i.e., w is a probability distribution over [ n ] and is usually denoted by D , we call z the natural parameter vector of D .
Recall that Q is a
Laplacian matrix where Q xy =
−Q ( x, y ) , i.e., it is the Laplacian matrix of the graph G Q weighted by the mass assigned by Q .
The proof goes as follows: we draw i.i.d.
samples from the LSS.
We ﬁrst apply the following result which is a modiﬁcation of the results of [SBB + 16].
Theorem 16 (Variant of [SBB + 16]) .
Let G Q be the graph of the support E of Q satisfying Assumption 1 with | V | = n vertices and associated weighted Laplacian matrix Q (see Equation (2) ).
Let L be the associated empirical Laplacian matrix with E [ L ] = Q .
Consider a vector z ∈ R n with (cid:104) 1 , z (cid:105)
=satisfying the constraint max x,y ∈E | z x − z y | ≤ log( φ ) for some constant φ > 1 .
Let Ex( Q ; z ) be the oracle that generates the example { ( x, y ) , q } as follows: the edge ( x, y ) is chosen with probability
Q ( x, y ) and the bit q is set to 1 with probability exp
( z x ) / (exp( z x ) + exp( z y )) ; otherwise it is set
1: procedure L EARN -S HIFT ( (cid:15), δ ) (cid:46)
Sample access to oracle Samp( Q ; D ) .
Set N = Θ (cid:16) n λ ( Q ) (cid:15) 2 log( 1 δ ) (cid:17) .
Draw N samples of the form (( x i , y i ) ,
q i ) ∈
E × { 0 , 1 } . Obtain (cid:98) z using Algorithm 6 (the estimation vector satisﬁes (cid:104) 1 , (cid:98) z (cid:105) = 0 and must be shifted). Compute C = log (cid:16)(cid:80) x ∈ [ n ] exp( (cid:98) z x ) (cid:17) 6:
Set z (cid:48) x
= (cid:98) z x − C for any x ∈ [ n ] (cid:46) See Appendix F.2 .
Output z (cid:48) (cid:46)
The output satisﬁes (cid:107) z − z (cid:48) (cid:107) ∞ ≤ (cid:15) . 8: end procedure to 0 . For any (cid:15), δ > 0 , there is a maximum likelihood estimator of z (Algorithm 6) which draws N = O (cid:16) n λ ( L ) (cid:15) 2 log( 1 δ ) (cid:17) samples from Ex( Q ; z ) and computes, in time polynomial in the number of samples N , an estimate (cid:98) z such that (cid:107) z − (cid:98) z (cid:107) 2 ≤ (cid:15) , with probability at least 1 − δ .
The proof can be found at Appendix F.1.
Some comments are in order: 1.
In the work of [SBB + 16], the target is the (re-parameterized) weights vector z (cid:63) ∈ R n satisfying the conditions (cid:104) 1 , z (cid:63) (cid:105) = [-
-]0 and (cid:107)
z (cid:63) (cid:107) ∞ ≤ B for some constant B .
The provided algorithm minimizes the empirical log-likelihood and, using O ( n/ ( λ ( Q ) · (cid:15) 2 ))) samples, computes an estimate (cid:98) z
so that (cid:107) z (cid:63) − (cid:98) z (cid:107) 2 ≤ (cid:15) .
Our provided Algorithm 6 has the same guarantees as the algorithm of [SBB + 16] but the target weight vector satisﬁes the conditions (cid:104) 1 , z (cid:63) (cid:105) = {+
+} and | z (cid:63)x − z (cid:63)y | ≤ B for some constant B .
The algorithm draws sufﬁciently many samples and then minimizes the empirical negative log-likelihood objective over an appropriately selected constrained set (see Appendix F.1).
Observe that for a Local Sampling Scheme with distribution D , the natural parameter vector z (cid:63) does not satisfy (cid:104) 1 , z (cid:63) (cid:105) = 0 and so Theorem 16 and Algorithm 6 cannot be directly applied.
We will discuss (Appendix F.2) how to apply Algorithm 6 to distributions, i.e., target weight vectors with (cid:104) 1 , z (cid:63) (cid:105) ≤ 0 .
This results in learning the target vector z (cid:63) in L ∞ norm (which is sufﬁcient for Algorithm 3).
The algorithm of Theorem 16 follows:
Algorithm 6 Learning from (Pairwise) Local Sampling Schemes 1: procedure L EARN ( (cid:15), δ ) (cid:46)
Sample access to oracle Ex( Q ; z ) .
Set N = Θ (cid:16) n λ ( Q ) (cid:15) 2 log( 1 δ ) (cid:17) .
Draw N samples of the form (( x i , y i ) ,
q i ) ∈
E × { 0 , 1 } .
Compute the empirical negative log-likelihood objective
L N ( z ; { (( x i , y i ) , q i ) } i ∈ [
[Equation]
Minimize L N using gradient descent in the subspace Ω φ .
(cid:46) See Appendix F.1 .
Output the guess vector (cid:98) z .
7: end procedure To complete the proof of Theorem 4, we combine the upcoming Lemma 17 and Lemma 18 with the main result of [SBB + 16]. Lemma 17 states that, after drawing sufﬁciently many samples from Samp( Q ; D ) , we can construct an empirical Laplacian matrix L , whose Fiedler eigenvalue is of the same order as the one of the unknown matrix Q of the Local Sampling Scheme.
Moreover, notice that Lemma 17 implies that, with high probability, the graph induced by the matrix L is connected, i.e., λ ( L ) > 0 .
Lemma 17 (Concentration of Empirical Fiedler Eigenvalue) .
Let (cid:15) > 0 and V = { v ∈ {± 1 , 0 } n : v = e
i − e j for any i < j } and let Q be a distribution over V .
Let Q be the Laplacian matrix of Q (see Equation (2) ).
There exists an algorithm that uses O (log( n/δ ) /
( λ ( Q ) (cid:15) 2 ))
samples from Q and computes a matrix L that satisﬁes | λ ( L ) − λ ( Q ) | ≤ (cid:15)λ ( Q ) , with probability at least 1 − δ , where λ ( · ) denotes the second smallest eigenvalue of a Laplacian matrix.
Proof.
Let { X i = v i v Ti } be a ﬁnite sequence of m independent symmetric square matrices (of common dimension n ) with v i ∼ Q , let L = 1 m (cid:80) mi =1 X i , and let M = λ max ( X i ) = Θ(1) .
To deal with the second smallest eigenvalue of the Laplacian matrix, we can project the Laplacian matrix L to the orthogonal complement of the vector 1 .
Under this transformation, we can obtain a matrix L (cid:48) that has two crucial properties.
First, it still is a sum of independent positive semideﬁnite terms and its minimum eigenvalue coincides with the Fiedler eigenvalue of L .
To achieve this, as in [Tro15], we introduce the transformation R ∈ R ( n − 1) × n , that satisﬁes RR T = I n − 1 and R 1 = 0 .
Hence, it holds that L (cid:48) =
RLR T and E [ L (cid:48) ] =
RQR T .
Then,
[Equation]
Using Proposition 7, we get that:
[Equation]
To upper bound this probability by δ , it sufﬁces to draw O (log( n/δ ) /
( (cid:15) 2 λ ( Q ))) samples.
So, we get that, with probability at least 1 − δ , the Fiedler eigenvalue of the empirical matrix L satisﬁes λ ( L ) ∈ (1 ± (cid:15) ) λ ( Q ) .
Moreover, the second smallest eigenvalue of the empirical Laplacian matrix
L is strictly positive and thus, the induced graph is connected (recall that the number of connected components in the graph is the dimension of the nullspace of the Laplacian matrix).
The next lemma guarantees that learning the natural parameters of any distribution in L ∞ norm is sufﬁcient for learning the distribution with small relative error over its support.
Lemma 18 (Stability of Relative Error) .
Let D 1 , D 2 be discrete distributions supported on the ground set [ n ] and let z 1 ,
z 2 be the corresponding natural parameter vectors.
For any sufﬁciently small accuracy parameter (cid:15) > 0 , if it holds that (cid:107) z 1 − z 2 (cid:107) ∞ ≤ (cid:15) , then the distributions D 1 , D 2 are close in relative error, i.e., max (cid:110) (cid:107) 1 − D 1 / D 2 (cid:107) ∞ , (cid:107) 1 − D 2 / D 1 (cid:107) ∞ (cid:111) ≤ (cid:15) .
Proof of Lemma 18.
Let (cid:15) > 0 sufﬁciently small and assume that (cid:107) z 1 − z 2 (cid:107) ∞ ≤ (cid:15) , i.e., max x ∈ [ n ]
[Equation]
For x > 0 , it holds that 1 − 1 x ≤ log e ( x ) 4 .
Hence, for any x ∈ [ n ] , we get that
[Equation]
and
[Equation]
This gives that
[Equation]
This implies that both ratios are upper bounded by 1 + (cid:15) .
For contradiction, assume that
[Equation]
But, the Taylor expansion of the function x (cid:55)→ 11+ x is equal to 1 − x
+ O ( x 2 ) for | x | < 1 .
Hence, for sufﬁciently small (cid:15), the desired bound follows.
To wrap up, the proof of Theorem 4 goes as follows: of Theorem 4.
Assume that z (cid:63) ∈ R n is the true natural parameter vector for the target distribution
D .
By Lemma 18, in order to obtain the desired relative error, it sufﬁces to control z (cid:63) in L ∞ .
Also, let z ∈ R n be a shifted variant of z (cid:63) so that (cid:104) 1 , z (cid:105) = 0 , i.e., z (cid:63) =
z + C 1 for some C .
First, the algorithm draws N = Θ (cid:16)
[Equation]
samples from the LSS with Laplacian matrix Q .
Using the concentration of the Fiedler eigenvalue (Lemma 17), we have that this number of samples is sufﬁcient to apply Theorem 16.
This will yield a vector (cid:98) z
so that (cid:107) z − (cid:98) z (cid:107) 2 ≤ (cid:15) with probability 1 − δ .
We then apply the transformation described in Appendix F.2 (which corresponds to Line 5of Algorithm 5).
This yields the desired L ∞ bound and completes the proof.
F The Proof of Theorem 16 (Variant of [SBB + 16])
In this section, we prove a slight variant of the algorithm of [SBB + 16] where the target weight vector satisﬁes the conditions (cid:104) 1 , z (cid:63) (cid:105) = 0 and | z (cid:63)x − z (cid:63)y | ≤ B for some constant B .
F.1 Description of the Learning Algorithm for Abstract Weight Vectors
In this section, we provide a complete description of the learning algorithm of [SBB + 16] (with a slight modiﬁcation).
We remark that the technical steps required in order to establish our result follow the analysis of [SBB + 16].
It sufﬁces to present an efﬁcient learning algorithm that estimates the target weights vector z (cid:63) =
( z (cid:63) 1 , . . .
, z (cid:63)n ) in L ∞ norm.
We underline that for this section, the target vector z (cid:63) is an abstract weight vector and not the natural parameter vector induced by a distribution.
We assume that z (cid:63) lies in the subspace Ω φ , deﬁned in Equation (6).
Hence, it sufﬁces that the learning algorithm computes an estimate (cid:98) z such that (cid:107)
z (cid:63) − (cid:98) z (cid:107) 2 ≤ (cid:15) , with high probability, for some accuracy parameter (cid:15) > 0 . of Theorem 16.
The algorithm minimizes the empirical negative log-likelihood over a subspace Ω φ ⊆ R n of the natural parameter vectors.
The empirical negative log-likelihood objective with N draws from the Local Sampling Scheme corresponds to the function
[Equation]
where x i , y i ∈ [ n ] and q i ∈ { 0 , 1 } for any i ∈ [ N ] .
We optimize this objective using gradient descent in the subspace Ω φ , where
[Equation]
The ﬁrst constraint (cid:104) 1 , z (cid:105) = 0 is imposed in order to work in the subspace where Laplacian matrices are positive deﬁnite (since any Laplacian matrix has its ﬁrst eigenvalue equal to 0 with corresponding eigenvector 1 ).
The second constraint 5 is equivalent to Assumption 2.
Since any valid target distribution satisﬁes 1 φ ≤ D ( x ) D ( y ) ≤ φ for ( x, y ) ∈ E , we have that
[Equation]
) for any x, y ∈ E .
Let L be the empirical Laplacian matrix whose Fiedler eigenvalue λ 2 ( L ) is close to the true λ 2 ( Q ) (see Lemma 17).
We introduce the following notation for the quadratic form (cid:107) v (cid:107) 2 L := v T Lv for any vector v ∈ R n .
The key idea of the algorithm of [SBB + 16] is to compute an estimate (cid:98) z in the (cid:107) · (cid:107) L semi-norm.
Afterwards, by the min-max principle for Hermitian matrices, we have that
[Equation]
Hence, the estimation of the true natural parameter vector in L 2 norm is directly implied.
The analysis for the estimation in the L semi-norm is based on the following fact about M -estimators.
A proof of this result can be found in [SBB + 16].
Fact 3 (see [SBB + 16]) .
Let Ω 1 = { z ∈ R n : (cid:104) 1 , z (cid:105) = 0 } and let Ω ⊆ Ω 1 .
Consider the
M -estimator
[Equation]
and let (cid:96) be a differentiable objective that is κ -strongly convex 6 at z
(cid:63) ∈ Ω with respect to the L semi-norm.
Then, it holds that
[Equation]
where L † is the Moore-Penrose pseudo-inverse of L .
Using the above result, it sufﬁces to verify that the empirical negative log-likelihood objective is strongly convex in the true parameter vector z (cid:63) ∈ Ω φ and to upper bound the dual norm
[Equation]
Recall that L is the empirical estimate of the true
Laplacian matrix Q .
We continue with two claims, that are sufﬁcient in order to control the L semi-norm of the vector z (cid:63) − (cid:98) z .
Claim 8 (Strong Convexity at z (cid:63) ) .
The empirical negative log-likelihood L N ( z ) is κ -strongly convex at z
(cid:63) ∈ Ω φ with respect to the L semi-norm with κ =
poly(1 /φ ) .
Proof.
It sufﬁces to lower bound the quadratic form w T ∇ L N ( z ) w for all vectors w ∈ R n and z ∈ Ω φ .
As in Lemma 17, we introduce the measurement vector notation v i ∈ {− 1 , 0 , 1 } n in order to represent each drawn sample (( x i , y i ) , q i ) (recall that the edges in this training set lie in E and q i ∈ { 0 , 1 } ).
We set v i ( x i ) =
2 q i − 1 , v i
( y i )
=
− v i ( x i ) and the other coordinates are set to 0 .
These measurement vectors are the building blocks of the empirical matrix (see Lemma 17), since we have that
[Equation]
Note that the matrix X ∈ {− 1 , 0 , 1 } N × n has the i -th measurement vector v Ti as its i -th row.
Using this notation and following the computations of [SBB + 16] for the Hessian of L N , we get that
[Equation]
where
[Equation]
z
[Equation] v i [Equation] and F ( x ) = 1 / (1 + exp( − x )) .
The function F ( x ) = 1 / (1 + exp( − x )) is the Bradley-Terry function.
The function F is strongly-log concave in the interval [ − log( φ ) , log( φ )] .
Speciﬁcally, we have that
[Equation]
since the mapping x (cid:55)→ exp( x ) / (1 + exp( x )) 2 is symmetric and, so, focusing on the interval [0 , log( φ )] , its minimum is attained at log( φ ) .
The physical interpretation of these properties is that for the pair i, j , it does not matter whether i (cid:31) j or j (cid:31) i (symmetry) but if the comparison gap (i.e., | z i − z j | ) attains very large values, then the strong convexity will be very low.
Intuitively, note also that, since exp( x ) / (1 + exp( x )) 2 ≤ 1 / 4 , the [Equation] [Figure] desired objective is also smooth and it is known that strong convexity and smoothness of a function over a domain of the form { z ∈ R n : (cid:104) 1 , z (cid:105) =
0 } ∩ Z , where Z is convex, imply that the function satisﬁes the PL inequality over Z .
Hence, for this function, gradient-based methods will converge with fast rates (see e.g., [VYZ20]).
Since any z ∈ Ω φ , we conclude that for any w
[Equation]
where X the above measurement matrix and z ∈ Ω φ .
Hence, if we deﬁne ∆ = (cid:98)
z − z (cid:63) , we get (in order to form the deﬁnition of strong convexity) that
[Equation]
since (cid:107) ∆ (cid:107) 2 L = ∆ T L ∆ = 1 N ∆ T X T X ∆ = 1 N (cid:107) X ∆ (cid:107) 22 .
Hence, the empirical negative loglikelihood objective is κ ( φ ) -strongly convex at the true natural parameter vector z (cid:63) with respect to the L semi-norm.
From the strong-convexity guarantee and, using Fact 3, we get that
[Equation]
[Equation]
Claim 9 (Dual norm) .
Let L be the empirical Laplacian estimate matrix of Q and let L † the
Moore-Penrose pseudo-inverse of L .
There exists a random vector Y ∈ R n such that
[Equation]
where X is the N × n measurements matrix.
Proof.
We ﬁrst provide a compact form the the gradient ∇ L N ( z (cid:63) ) .
Again, following the analysis of [SBB + 16], we deﬁne a random vector Y ∈ R n with independent coordinates ( Y 1 , . . .
, Y n ) where we set Y
i
=
F (cid:48) ( (cid:104) z (cid:63) , v i (cid:105) ) /F
( (cid:104) z (cid:63) , v i (cid:105) ) with
probability F ( (cid:104) z (cid:63) , v i (cid:105) )
and we set Y i = − F (cid:48)
( (cid:104) z (cid:63) , v i (cid:105) ) / (1 − F ( (cid:104) z (cid:63) , v i (cid:105) )) otherwise.
Recall that F , in our setting, is the Bradley-Terry function F ( x )
= 1 / (1 + exp( − x )) and v i ∈ {− 1 , 0 , 1 } n is the i -th comparison vector indicating the drawn edge.
Hence, after computing the gradient at the true parameter vector z (cid:63) , one gets
[Equation]
Claim 9 follows.
We now introduce the matrix J :=
[Equation]
κ [Equation] N 2 XL † X T .
Combining Equation (8) with Claim 9, we get that
[Equation]
Hence, in order to bound the L semi-norm estimation error, it sufﬁces to control the quadratic form of the right hand side.
Similarly to [SBB + 16], our goal is to apply the Hanson-Wright inequality (see Lemma 19) in order to control the quadratic form.
Lemma 19 ([HW71, RV13]) .
Let Y ∈ R n be a random vector with independent zero-mean components, which are sub-Gaussian with parameter K and let J ∈ R n × n be an arbitrary matrix.
Then there is a universal constant c > 0 such that
[Equation]
for any t > 0 .
Observe that E [ Y ] = 0 and, for any i ∈ [ n ] , we have that
[Equation]
where X ⊆ {− 1 , 0 , 1 } n is the space of all valid measurement vectors (they should induce edges in
E ).
Hence, we get that
[Equation]
Note that u ( φ ) ≤ 1 , since, for 0 < b < 1 , it holds that max (cid:110) 1 b , 11 − b (cid:111) ≤ 1 b (1 − b ) .
Since each | Y i | is bounded, we get that the random variables are u ( φ ) -sub-Gaussian.
Also, we have that
[Equation]
It remains to compute the two matrix norms.
As in [SBB + 16], one gets that
[Equation]
and
[Equation]
Applying the Hanson-Wright concentration inequality, we conclude that
[Equation]
for all t ≥ 1 and some universal constant c > 0 .
Recall that φ is the constant of Assumption 2 and hence the term [Equation]
Finally, note that integrating the above tail bound, one gets the desired on expectation bound.
F.2 Application of Algorithm 6 to Distributions
Given a target distribution D , the associated natural parameters vector z = log( D ) satisﬁes (cid:104) 1 , z (cid:105) ≤ 0 .
Hence, we have to shift it appropriately in order to apply Algorithm 6.
Recall that the condition (cid:104) 1 , z (cid:105) = 0 is only useful for identiﬁability purposes of the BTL model.
Given a target distribution D ∈ ∆ n , we convert it to the following weights vector: let A = (cid:80) x ∈ [ n ] log( D ( x )) and set D (cid:48)
( x ) = exp( − A/n ) D ( x ) .
So, we have to shift z in the direction ( − A/n ) · 1 and obtain the weight vector z (cid:48) .
For the weights ( D (cid:48)
( x )) x ∈ [ n ] with weight vector z (cid:48) , it holds that
1. (cid:104) 1 , z (cid:48) (cid:105) =
(cid:80) x ∈ [ n ] ( − A/n + log( D ( x )))
= 0 and 2.
for any x, y ∈ [ n ] , it holds that D (cid:48) ( x ) / D (cid:48)
( y )
=
D ( x ) / D ( y ) .
Given an estimate for z (cid:48) , one can extract a good approximation for the natural parameter vector z of D .
In particular, the following properties hold: ﬁrst, given two distributions p (cid:54) =
q , it holds that z (cid:48) ( p ) (cid:54) =
z (cid:48) ( q ) , i.e., any z (cid:48) is uniquely identiﬁed by its distribution since one of the following will hold: if (cid:80) i ∈ [ n ] log( p i ) =
A = (cid:80) i ∈ [ n ] log( q i ) , then since there exists j ∈ [ n ] so that p j (cid:54) =
q j , we have that z ( p ) (cid:48)
j = log( p j ) − A/n (cid:54) =
log( q j ) − A/n = z ( q ) (cid:48)
j .
Otherwise, if, without loss of generality, A p = (cid:80) i log( p i ) >
(cid:80) i log( q i ) =
A q , we have that there exists j ∈ [ n ] so that p j < q j and so z ( p ) (cid:48)
j = log( p j ) − A p /n < log( q j ) −
A q /n =
z ( q ) (cid:48)
j .
Hence, any distribution D uniquely induces a vector z (cid:48) .
Moreover, the actual natural parameter vector z can be estimated having an estimation for z (cid:48) .
Assume that we have an estimate (cid:98) z (cid:48) that satisﬁes (cid:107) [Table] ≤ (cid:15) .
We can estimate z using a vector (cid:98) z (that can be extracted from our estimate (cid:98) z (cid:48) ) as follows:
Let z (cid:48)
=
z + C 1 , where
C = − A/n.
In order to initiate our exact sampling algorithm, an L ∞ estimate for z is only required.
We have that
[Equation]
where the estimate (cid:98) z (cid:48) satisﬁes (cid:107) z (cid:48) − (cid:98) z (cid:48) (cid:107) ∞ ≤ (cid:107) z (cid:48) − (cid:98) z (cid:48) (cid:107) 2 ≤ (cid:15) via Algorithm 6 and the shift constant can be estimated since we know that D is a distribution:
[Equation]
Note that the right hand side of the above equation contains only our estimates.
We have that (cid:80) x ∈ [ n ] exp( z x ) =
and so we set e C :
= (cid:80) x ∈ [ n ] e z (cid:48)
x .
Since it holds that | z (cid:48) x − (cid:98) z (cid:48) x | ≤ (cid:15) for any x ∈ [ n ] , we have that | C − C (cid:48) | ≤ O ( (cid:15) ) .
This gives that for the actual natural parameters it holds that (cid:107) z − (cid:98) z (cid:107) ∞ ≤ O ( (cid:15) ) .
Hence, given a distribution D , we can learn the weights z (cid:48) in L 2 norm, satisfying the conditions of Algorithm 6.
From this estimation, we can get the estimation for the natural parameters z by removing the introduced shift.
We remark that the above shifting methodology can be applied to any score vector z satisfying (cid:104) 1 , z (cid:105)
=
B (cid:54) = 0 .
G Local Sampling Schemes and Hypergraphs
In this section, we discuss how to extend our analysis to sets of size larger that 2.
The hypergraph structure of Local Sampling Schemes can be settled as follows:
Deﬁnition 20 (Hypergraph Structure of LSS) .
Let Z be a ﬁnite discrete domain and let Q be a distribution supported on subsets of Z .
Then, the hypergraph G =
( V, E ) with vertex set V =
Z and hyperedge set E = supp( Q ) is called a Local Sampling Scheme hypergraph.
If Q is a pair distribution supported on Z × Z , G corresponds to a graph.
The general Markov Chain for distributions Q , supported on sets S ⊆ 2 [ n ] has the following transition probabilities:
[Equation]
For the transition x → y , we can think of a ﬂow f xy with mass p xy and, hence, induce a (simple) graph structure on a ﬂow graph F = ( f xy ) xy .
Observe that any pair ( x, y ) shares the same collection of hyperedges and, hence, we have that:
[Equation]
The ergodicity of the ﬂow graph implies that the stationary distribution corresponds to D and is unique.
For what follows, we consider the case where any hyperedge has size k , i.e., k -uniform hypergraphs.
The Laplacian of the distribution Q can be generalized over hyperedges and have that
[Equation]
any
[Equation]
y [Equation] [Equation]
[Equation]
In the following section, we describe some necessary notation.
G.1 Notation
We denote the set (cid:0) [ n ] k (cid:1) the family of size k subsets of the ground set [ n ] .
Let Q be a distribution on (cid:0) [ n ] k (cid:1) .
Such a distribution will be called a k -set distribution.
Recall that a pair distribution is simply a 2 -set distribution.
A sample from the Local Sampling Scheme Samp k ( Q ; D ) , where Q is a k -set distribution, will be denoted
[Equation]
where S ∈ supp( Q ) and, in the case where i ∈ S won between the elements of S , we have that v = e i ∈ R n ( v is the indicator vector of the winning node) and we have that Pr[ v = e i ] =
D ( i ) / D ( S ) 1 { i ∈ S } .
For the hypergraph case, the canonical transition matrix induced by the
Local Sampling Scheme will be denoted by P .
In the case k = 2 , this matrix was denoted by M (recall Equation (1)).
Let P denote the transition matrix of the Markov chain, associated with the
Local Sampling Scheme Samp k ( Q ; D ) , where Q corresponds to a k -set distribution.
The entries of
P =
[ P xy ] x,y ∈
[ n ] are deﬁned as:
[Equation]
Observe that the transition from x to y is performed when
a hyperedge S ∈ supp( Q ) is chosen (with probability Q ( S ) ) and both x and y lie in S , 2.
and the vertex y ∈ [ n ] is the ’winning’ node among the nodes of S , i.e., y is drawn from the conditional distribution D S .
There is a natural reduction to the graph case.
Consider the marginals of Q to 2 -sets (i.e., edges).
Then, one can deﬁne a Markov chain over a graph with [ n ] nodes, whose transition probabilities are described by P .
The random walk has the following properties: Since any pair of vertices shares the same collection of hyperedges, we get that
[Equation]
for any pair of vertices.
Moreover, the Markov chain is ergodic, since it is irreducible, since by the structure of Q and since D is supported on [ n ] , one eventually can get from every state to every other state with positive probability; and aperiodic, since it contains self-loops.
Hence, it has a unique stationary distribution which coincides with D , using the detailed balance equations.
Modiﬁcations of the LSS Conditions.
We can modify the information-theoretic connectivity condition (Assumption 1) for k -uniform hypergraphs and let E the support of Q .
Also, we have to consider the variation of Assumption 2 over the support E with
[Equation]
for some constant φ .
We remark that the learning results of [SBB + 16] hold for k
= O (1) .
Similar, our learning tools which are modiﬁcations of the work of [SBB + 16] hold for the same regime.
G.2 Learning Phase for Hypergraphs
As in the learning phase of the 2 -set case, the learning algorithm for the k -set problem is essentially a variation of the analysis of [SBB + 16], but the steps are similar.
Consider N i.i.d.
samples ( S i , v i ) drawn from the sampling oracle Samp k ( Q ; D ) with natural parameter vector z (cid:63) ∈ R n .
The analysis that follows assumes that any k -set sample in the empirical likelihood lies in E , i.e., the support of Q .
Our goal is to estimate the true parameter vector.
We consider the negative empirical log-likelihood objective
[Equation]
and we optimize it over the parameter space
[Equation]
Likelihood Objective and PL model.
We now observe that the above likelihood objective is directly connected to the Plackett-Luce model, which captures the process of choosing a single alternative from a given set, i.e., given a set S of m alternatives with values w 1 , . . .
, w m , the likelihood of choosing the i -th item is
[Equation]
Note that the negative log-likelihood of the Plackett-Luce model is exactly the same as the single sample version of our objective function.
Observe that this function is shift-invariant and its value is independent of the ordering of the last ( m − 1) elements.
Shift invariance is crucial: if one does not work in the subspace { z : (cid:104) 1 , z (cid:105) = 0 } , then neither our problem nor the problem of determining the values of the alternatives in the Plackett-Luce model are identiﬁable, since any solution of the form z (cid:63) + c 1 is valid.
Technically, shift invariance implies that 1 lies in the nullspace of the Hessian of the negative log-likelihood (and this is where the ﬁrst constraint of the set Ω φ arises).
Hessian matrix.
Let us introduce the vector e z ( S ) =
(exp( z i ))
i ∈ S ∈ R k for an arbitrary set
S ⊆ (cid:0) [ n ] k (cid:1) .
After standard computations (see also [SBB + 16]), we get that
[Equation]
, and hence
[Equation]
Quantitative Strong Convexity of PL model.
Consider an arbitrary direction v ∈ R k .
Without loss of generality, assume that S =
[ k ] .
Then, it holds that
[Equation]
using the Cauchy-Schwarz inequality, where equality holds if and only if v ∈ span( 1 ) .
Hence, for the Plackett-Luce function F : R k
→ R , we have that [Equation] any [Equation]
∈ [Equation] In order to quantify the strong convexity parameter, we will make use of the second constraint.
For the Plackett-Luce function F , we have that
[Equation]
for some k × k symmetric matrix H with λ 2 ( H ) > 0 .
We can observe that the matrix
[Equation] where β ( φ ) =
min
[Equation]

 , satisﬁes the strong log-concavity condition for the Plackett-Luce model, i.e., the function F : R k
→ R .
Underlying Laplacian matrix and estimation.
Similarly to the case k = 2 , our goal is to establish strong convexity for the empirical negative log-likelihood around the true parameters z (cid:63) ∈
Ω φ with respect to the L semi-norm.
Hence, we have to ﬁrst introduce the appropriate Laplacian matrix L .
In the k -set setting, we have that
[Equation]
Observe that Q 1 =
(since each set S ⊆ (cid:0) [ n ] k (cid:1) that contains x , also contains other k − 1 elements).
Recall that for the case k = 2 , the estimate for the Laplacian matrix Q is given by the empirical matrix L = 1 N (cid:80) Ni
=
v i v Ti = 1 N X T X (see Appendix F.1).
In the case k > 2 case, the Laplacian estimate for the matrix Q is given by the n × n matrix
[Equation]
where the vectors v i are substituted by the ( n × k ) matrices
E i , where each one of the k columns of
E i is a unit vector.
The k non-zero elements of E i correspond to the k elements that lie in the i -th drawn set S
i , i.e., E i = (cid:104) e [Equation] (cid:12)(cid:12)(cid:12) e [Equation]
2 (cid:12)(cid:12)(cid:12) . . .
(cid:12)(cid:12)(cid:12) e i k (cid:105) for S
i =
{ i 1 , i 2 , . . .
, i k } .
In the estimation part 7 , our goal is to estimate the unknown matrix Q .
Using concentration results of the Fiedler eigenvalue of sums of Hermitian matrices [Tro15], using O (log( n ) /λ ( Q )) samples, one can compute an estimate L (that corresponds to the above empirical estimate) that approximates the true λ ( Q ) with (1 + (cid:15) ) multiplicative error.
We have that
[Equation]
where 8 X i = E i ( k I − 11 T ) E Ti , i.e., the estimate L is a sum of independent Hermitian matrices with λ max ( X i ) =
Θ( k ) .
Using Proposition 7 in a similar manner as in Lemma 17, we get that
[Equation]
Hence, it sufﬁces to draw at least
[Equation]
in order to guarantee the desired concentration bound with conﬁdence at least 1 − δ .
Strong Convexity of Empirical NLL w.r.t.
(cid:107) · (cid:107) L .
Having introduced the empirical Laplacian matrix, we are able to get the desired strong convexity result for the empirical negative log-likelihood with respect to the L semi-norm.
Following the analysis of [SBB + 16], we get that, for any vector w ∈ R d and parameter z ∈ Ω φ ,
[Equation]
where H = β ( φ )( I − 11 T ) .
Hence, we get that
[Equation]
Note that λ 2 ( I − 11 T )
= 1 .
Consequently, the empirical negative log-likelihood is β ( φ ) /k
-strongly convex around the true parameters z (cid:63) ∈
Ω φ .
Using Fact 3, we get that
[Equation]
of
We have that E =
{+[ e 1 | e 2 | e 3 ] =   1 0 0 0 10 0 1 0 
+}
0  and X =   − 1 − 1 0 0 − − 1 0 0 − 1 − 0 0  0 0  0 
Bounding the (expected) dual norm (cid:107) · (cid:107) L † .
Following the exact analysis of [SBB + 16] for the expected value of the dual norm, we get that for our setting
[Equation]
where F is the Plackett-Luce function and assuming that S
=
[ k ] (and S satisﬁes the modiﬁed mass condition, i.e., S lies in the support of Q ).
We have that
[Equation]
for some i ∈ [ k ] .
Hence, we get that
[Equation]
Over the optimization space Ω φ and since the natural parameters z x = log( D ( x )) ≤ 0 , we have that
[Equation]
[-X =   − 1 − 1 0 0 − − 
-]
Hence, we have that
[Equation]
and
[Equation]
Finally, we get that
[Equation]
Conclusion of the Learning Phase in k -sets.
We get that we can estimate the true parameters in
L semi-norm, i.e.,
[Equation]
Hence, we can transfer this result to an L 2 bound and get
[Equation]
Using N = O (cid:16)
[Equation]
nk
[Equation]
samples from the oracle Samp k
[Equation] Q [Equation] D ) on the support set E , we can learn the parameter vector in L 2 .
At this point, the proof of the learning phase is similar to the k = 2 case: We can apply the trick of Appendix F.2 and get an L ∞ bound.
Hence, using Lemma 18, we can estimate the target distribution with relative error.
G.3 Downscaling Phase for Hypergraphs
Let (cid:98) z be the estimate of the natural parameters vector and let (cid:101) D the correspond distribution.
We use the estimate (cid:101) D of the target distribution D to transform the Markov chain of the k -set Local Sampling
Scheme into another almost uniform stationary distribution, as in the case k = 2 .
Using the Bernoulli downscaling mechanism, for the pair ( x, y ) , we have that
[Equation]
We have that
[Equation]
The modiﬁed transition matrix (as in the k = 2 case) has an almost uniform stationary distribution.
Also, it has an absolute spectral gap of same order as the matrix (cid:101) P that can be expressed as
[Equation]
Q
[Equation] and (cid:101) P xx = 1 − (cid:80) y (cid:54) =
x (cid:101) P xy .
Hence, we get that
[Equation]
The expression of the modiﬁed transition matrix (cid:101) P is exactly similar to the k = 2 case (see Equation (5)).
Spectral analysis as in Lemma 15 of this matrix will result to the following properties:
( i )
The transition matrix (cid:101) P has stationary distribution (cid:101)
π 0 ( x ) = Θ(1 /n )
for all x ∈ [ n ] , i.e., an almost uniform probability measure.
( ii )
The absolute spectral gap Γ( (cid:101) P ) of (cid:101) P and the minimum non-zero eigenvalue λ ( Q ) of the Laplacian matrix Q satisfy Γ( (cid:101) P ) =
Ω( λ ( Q )) .
( iii )
The mixing time of the transition matrix (cid:101) P is T mix ( (cid:101) P ; 1 / 4) =
O (log( n ) /λ
( Q )) , where λ ( Q ) is the minimum non-zero eigenvalue of the Laplacian matrix Q .
G.4 CFTP Phase for Hypergraphs
The structure of the k -set algorithm remains similar to the k = 2 case, with the exception that the drawn samples are now hyperedges.
We continue with the modiﬁed Algorithm 7.
Recall that, for each drawn sample ( S, v ) ∼ Samp k ( Q ; D ) , the vector v ∈ { e 1 , ..., e n } is the indicator vector of the winning node of the set S ⊆ (cid:0) [ n ] k (cid:1) .
G.5 Conclusion
Let N Learn and N CFTP be the expected number of samples required for a single execution of the learning algorithm for k -sets and of the parameterized CFTP algorithm respectively (see Algorithm 7). Under Assumption 1 (for the distribution Q over hyperedges) and the modiﬁed version of Assumption 2, the expected sample complexity of the generalized algorithm for the k -set case is N [Equation]
Note that β ( φ ) is constant under Assumption 2.
As explained in [SBB + 16], it is natural to consider the regime where
k
= O (1) .
In this regime, the learning bounds match the CFTP dependence on n under (cid:101)
O notation.
Corollary 21 (Exact Sampling from k -Set LSS) .
For any positive constant integer k > 2 , under
Assumption 1 (for the distribution Q over k -sets) and Assumption 2, there exists an algorithm (Algorithm 7) that draws an expected number of (cid:101) O (cid:16) n 2 k 2 λ
1: procedure E XACT S AMPLER - k -S ET ( p ) (cid:46)
Sample access to the LSS oracle Samp k ( Q ; D ) .
2: t ←3:
F 0 ( x ) ← x , for any x ∈ [ n ] 4: while F t has not coalesced do (cid:46) While no coalescence has occured.
5: t ← t −6:
Draw ( S, v ) where S ∈ E and v = e i for some i ∈ [ n ] 7: y ← position where v [ y ] =(cid:46) Find the winning node y ∈ S .
8: for x = 1 . . .
n do (cid:46)
In order to update state x .
9: if x / ∈ S or x = y then F t ( x ) ← F t +1 ( x ) 10: else
F t ( x ) ← (cid:26) F t +1 ( w ) , with probability min { p ( x ) /p ( w ) , 1 }
F t +1 ( x ) , otherwise 11: end 12: end 13:
Draw C ∼ Be( p ( F t (1)))
(cid:46)
Remove bias using rejection sampling.
14: if C = 1 then Output F t (1) else Output ⊥ (cid:46) Output the perfect sample or reject.
15: end procedure 16: procedure E XACT S AMPLER - k -S ET
W ITH L EARNING ( ) (cid:46)
The algorithm of Corollary 21 .
17: (cid:101) D ← L EARN ( (cid:15) :
= 1 / √ n, δ ) (cid:46)
Learn D in relative error as in Appendix G.2 .
18: x ←⊥ 19: while x = ⊥ repeat 20: x ← E XACT S AMPLER - k -S ET ( (cid:101) D ) 21:
Output x 22: end procedure