SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Note that  $w \in \mathbb{R}^{\binom{n}{k}}$  can be written as  $w = (w_{\emptyset}, w_{-\emptyset})$  where  $w_{\emptyset}$  corresponds to the constant term of the Fourier expansion and  $w_{-\emptyset} = (w_S)_{\emptyset \neq S \subseteq [n]:|S| \leq k}$  is the vector of the remaining coordinates. The Fourier expansion implies that
$$
\mathbf {V a r} _ {x \sim U (\{- 1, 1 \} ^ {n})} [ w \cdot x ^ {\otimes k} ] = \| w _ {- \emptyset} \| _ {2} ^ {2},
$$
which yields the desired equality for the variance.
G.1.1 Maximum Cut
Let us consider a graph with  $n$  nodes and weighted adjacency matrix  $A$  with non-negative weights. Maximum cut is naturally associated with the Ising model and, intuitively, our approach does not yield an efficient algorithm for solving Max-Cut since we cannot efficiently sample from the Ising model in general. To provide some further intuition, consider a single-parameter Ising model for  $G = (V,E)$  with Hamiltonian  $H_{G}(x) = \sum_{(i,j)\in E}\frac{1 + x_{i}x_{j}}{2}$ . Then the partition function is equal to  $Z_{G}(\beta) = \sum_{x\in \{-1,1\}^{V}}\exp (\beta H_{G}(x))$ . Note that when  $\beta >0$ , the Gibbs measure favours configurations with alligned spins (ferromagnetic case) and when  $\beta < 0$ , the measure favours configurations with opposite spins (anti-ferromagnetic case). The antiferromagnetic Ising model appears to be more challenging. According to physicists the main reason is that its Boltzmann distribution is prone to a complicated type of long-range correlation known as 'replica symmetry breaking' [COLMS22]. From the TCS viewpoint, observe that as  $\beta$  goes to  $-\infty$ , the mass of the Gibbs distribution shifts to spin configurations with more edges joining vertices with opposite spins and concentrates on the maximum cuts of the graph. Hence, being able to efficiently approximate the log-partition function for general Ising models, would lead to solving the Max-Cut problem.
Theorem 2 (Max-Cut has a Compressed and Efficiently Optimizable Solution Generator). Consider a prior over Max-Cut instances with  $n$  nodes. For any  $\epsilon > 0$ , there exists a solution generator  $\mathcal{P} = \{p(w) : w \in \mathcal{W}\}$  such that  $\mathcal{P}$  is complete, compressed with description  $\mathrm{poly}(n) \log(1 / \epsilon)$  and  $\mathcal{L} + \lambda R : \mathcal{W} \mapsto \mathbb{R}$  is efficiently estimable via projected stochastic gradient descent in  $\mathrm{poly}(n, 1 / \epsilon)$  steps for some  $\lambda > 0$ .
Proof of Theorem 2. It suffices to show that Max-Cut satisfies Assumption 1. Consider an input graph  $G$  with  $n$  nodes and Laplacian matrix  $L_{G}$ . Then
$$
\operatorname {M A X C U T} = \frac {1}{4} \max  _ {s \in \{- 1, 1 \} ^ {n}} s ^ {\top} L _ {G} s = \frac {1}{4} \min  _ {s \in \{- 1, 1 \} ^ {n}} - s ^ {\top} L _ {G} s.
$$
We show that there exist feature mappings so that the cost of every solution  $s$  under any instance/graph  $G$  is a bilinear function of the feature vectors (cf. Item 2 of Assumption 1). We consider the correlation-based feature mapping  $\psi_S(s) = (ss^\top)^\flat \in \mathbb{R}^{n^2}$ , where by  $(\cdot)^\flat$  we denote the vectorization/ flattening operation and the negative Laplacian for the instance (graph),  $\psi_{\mathcal{I}}(G) = (-L_G)^\flat \in \mathbb{R}^{n^2}$ . Then simply setting the matrix  $M$  to be the identity  $I \in \mathbb{R}^{n^2 \times n^2}$  the cost of any solution  $s$  can be expressed as the bilinear function  $\psi_{\mathcal{I}}(G)^\top M\psi_S(s) = (-L_G^\flat)^\top (ss^T)^\flat = -s^\top L_G s$ . We observe that (for unweighted graphs) with  $n$  nodes the bit-complexity of the family of all instances  $\mathcal{I}$  is roughly  $O(n^2)$ , and therefore the dimensions of the  $\psi_S, \psi_{\mathcal{I}}$  feature mappings are clearly polynomial in the bit-complexity of  $\mathcal{I}$ . Moreover, considering unweighted graphs, it holds  $\| \psi_{\mathcal{I}}(G) \|_2$ ,  $\| \psi_S(s) \|_2$ ,  $\| M \|_{\mathrm{F}} \leq \mathrm{poly}(n)$ . Therefore, the constants  $D_S, D_{\mathcal{I}}, C$  are polynomial in the bit-complexity of the instance family.
It remains to show that our solution feature mapping satisfy the variance preservation assumption. For any  $v$ , we have that  $\mathbf{Var}_{s\sim U(S)}[v\cdot \psi_S(s)] = \mathbf{Var}_{s\sim U(\{-1,1\} ^n)}[v\cdot (ss^\top)^\flat ] = \Omega (\| v\| _2^2)$ , using Lemma 9 with  $k = 2$ , since  $c_{\emptyset} = 0$  with loss of generality.
G.1.2 Minimum Cut/Maximum Flow
Let us again consider a graph with  $n$  nodes and Laplacian matrix  $L_{G}$ . It is known that the minimum cut problem is solvable in polynomial time when all the weights are positive. From the discussion of the maximum cut case, we can intuitively relate minimum cut with positive weights to the ferromagnetic Ising setting [dPS97]. We remark that we can consider the ferromagnetic parameter space  $\mathcal{W}_{\mathrm{fer}} = \mathbb{R}_{\geq 0}^{\binom{n}{2}}$  and get the variance lower bound from Lemma 9. We constraint projected SGD in  $\mathcal{W}_{\mathrm{fer}}$ . This means that during any step of SGD our algorithm has to sample from a mixture of ferromagnetic models with known mixture weights. The state of the art approximate sampling algorithm from ferromagnetic Ising models achieves the following performance, improving on prior work [JS93, LSS19, CLV22, CGG+19].
Proposition 9 (Theorem 1.1 of [CZ22]). Let  $\delta_{\beta}, \delta_{\lambda} \in (0,1)$  be constants and  $\mu$  be the Gibbs distribution of the ferromagnetic Ising model specified by graph  $G = (V,E), |V| = n, |E| = m$ , parameters  $\beta \in [1 + \delta_{\beta}, +\infty)^{m}$  and external field  $\lambda \in [0,1 - \delta_{\lambda}]^{n}$ . There exists an algorithm that
samples  $X$  satisfying  $\operatorname{TV}(X, \mu) \leq \epsilon$  for any given parameter  $\epsilon \in (0, 1)$  within running time
$$
m \left(\frac {\log n}{\epsilon}\right) ^ {O _ {\delta_ {\beta}, \delta_ {\lambda}} (1)}.
$$
This algorithm can handle general instances and it only takes a near-linear running time when parameters are bounded away from the all-ones vector. Our goal is to sample from a mixture of two such ferromagnetic Ising models which can be done efficiently. For simplicity, we next restrict ourselves to the unweighted case.
Theorem 3 (Min-Cut has a Compressed, Efficiently Optimizable and Samplable Solution Generator). Consider a prior over Min-Cut instances with  $n$  nodes. For any  $\epsilon > 0$ , there exists a solution generator  $\mathcal{P} = \{p(w) : w \in \mathcal{W}\}$  such that  $\mathcal{P}$  is complete, compressed with description  $\mathrm{poly}(n) \log(1/\epsilon)$ ,  $\mathcal{L} + \lambda R : \mathcal{W} \mapsto \mathbb{R}$  is efficiently estimable via projected stochastic gradient descent in  $\mathrm{poly}(n, 1/\epsilon)$  steps for some  $\lambda > 0$  and efficiently samplable in  $\mathrm{poly}(n, 1/\epsilon)$  steps.
$$
\text {M I N C U T} = \frac {1}{4} \min  _ {x \in \{- 1, 1 \} ^ {n}} x ^ {\top} L _ {G} x.
$$
The analysis (i.e., the selection of the feature mappings) is similar to the one of Theorem 2 with the sole difference that the parameter space is constrained to be  $\mathcal{W}_{\mathrm{fer}}$  and  $\psi_{\mathcal{I}}(G) = (L_G)^{\flat}$ . We note that Proposition 9 is applicable during the optimization steps. Having an efficient approximate sampler for solutions of Min-Cut, it holds that the runtime of the projected SGD algorithm is poly  $(n,1 / \epsilon)$ . We note that during the execution of the algorithm we do not have access to perfectly unbiased samples from mixture of ferromagnetic Ising models. However, we remark that SGD is robust to that inaccuracy in the stochastic oracle. For further details, we refer e.g., to [d'A08]).
G.1.3 Max-k-CSPs
In this problem, we are given a set of variables  $\{x_u\}_{u\in \mathcal{U}}$  where  $|\mathcal{U}| = n$  and a set of Boolean predicates  $P$ . Each variable  $x_{u}$  takes values in  $\{-1,1\}$ . Each predicate depends on at most  $k$  variables. For instance, Max-Cut is a Max-2-CSP. Our goal is to assign values to variables so as to maximize the number of satisfied constraints (i.e., predicates equal to 1). Let us fix a predicate  $h\in P$ , i.e., a Boolean function  $h:\{-1,1\}^n\to \{0,1\}$  which is a  $k$ -junta. Using standard Fourier analysis, the number of satisfied predicates for the assignment  $x\in \{-1,1\}^n$  is
$$
F (x) = \sum_ {j = 1} ^ {| P |} \sum_ {S \subseteq [ n ], | S | \leq k} \widehat {h} _ {j} (S) \prod_ {u \in S} x _ {u},
$$
where  $\widehat{h}_j(S)$  is the Fourier coefficient of the predicate  $h_j$  at  $S$ .
Theorem 4 (Max- $k$ -CSPs have a Compressed and Efficiently Optimizable Solution Generator). Consider a prior over Max- $k$ -CSP instances with  $n$  variables, where  $k \in \mathbb{N}$  can be considered constant compared to  $n$ . For any  $\epsilon > 0$ , there exists a solution generator  $\mathcal{P} = \{p(w) : w \in \mathcal{W}\}$  such that  $\mathcal{P}$  is complete, compressed with description  $O(n^{k})\mathrm{polylog}(1 / \epsilon)$  and  $\mathcal{L} + \lambda R : \mathcal{W} \mapsto \mathbb{R}$  is efficiently estimable via projected stochastic gradient descent in  $\mathrm{poly}(n^{k}, 1 / \epsilon)$  steps for some  $\lambda > 0$ .
Proof. Any instance of Max- $k$ -CSP is a list of predicates (i.e., Boolean functions) and our goal is to maximize the number of satisfied predicated with a single assignment  $s \in \{-1, 1\}^n$ . We show that there exist feature mappings so that the cost of every solution  $s$  under any instance/predicates list  $P$  is a bilinear function of the feature vectors (cf. Item 2 of Assumption 1). We consider the order  $k$  correlation-based feature mappings  $\psi_S(s) = (s^{\otimes k})^\flat \in \mathbb{R}^{n^k}$ , where by  $(\cdot)^\flat$  we denote the flattening operation of the order  $k$  tensor, and,  $\psi_{\mathcal{I}}(P) = \psi_{\mathcal{I}}(h_1, \ldots, h_{|P|}) = -\sum_{j=1}^{|P|} ((\widehat{h}_j(S))_{S \subseteq [n], |S| \leq k})^\top \in \mathbb{R}^{n^k}$ , where  $(\widehat{h}_j(S))_{S \subseteq [n], |S| \leq k}$  is a vector of size  $n^k$  with the Fourier coefficients of the  $j$ -th predicate. We take  $\psi_{\mathcal{I}}$  being the coordinate-wise sum of these coefficients. The setting the matrix  $M$  to be the identity matrix  $I \in \mathbb{R}^{n^k \times n^k}$ , we get that the cost of any solution  $s$  can be expressed as the bilinear function  $\psi_{\mathcal{I}}(P)^\top M \psi_S(s) =$
$-\sum_{j=1}^{|P|}\sum_{S \subseteq [n], |S| \leq k} \widehat{h}_j(S) \prod_{u \in S} x_u$ . For any  $h: \{-1,1\}^n \to \{0,1\}$ , we get that the description size of any  $\widehat{h}(S)$  is  $\mathrm{poly}(n,k)$  and so the dimensions of the  $\psi_S, \psi_{\mathcal{I}}$  feature mappings are polynomial in the description size of  $\mathcal{I}$ . Moreover, we get that  $\|\psi_{\mathcal{I}}(P)\|$ ,  $\|\psi_S(s)\|$ ,  $\|M\| \leq \mathrm{poly}(n^k)$ . Hence, the constants  $D_S, D_{\mathcal{I}}, C$  are polynomial in the description size of the instance family. Finally, we have that for any  $v$ ,  $\mathbf{Var}_{s \sim U(S)}[v \cdot \psi_S(s)] = \mathbf{Var}_{s \sim U(\{-1,1\}^n)}[v \cdot s^{\otimes k}] = \Omega(\|v\|_2^2)$ , using Lemma 9, assuming that  $v_\emptyset$  is 0 without loss of generality. This implies the result.
G.2 Bipartite Matching and TSP
G.2.1 Maximum Weight Bipartite Matching
In Maximum Weight Bipartite Matching (MWBM) there exists a complete bipartite graph  $(A,B)$  with  $|A| = |B| = n$  (the assumptions that the graph is complete and balanced is without loss of generality) with weight matrix  $W$  where  $W(i,j)$  indicates the value of the edge  $(i,j), i \in A, j \in B$  and the goal is to match the vertices in order to maximize the value. Hence the goal is to maximize  $L(\Pi) = W \cdot \Pi$  over all permutation matrices. By the structure of the problem some maximum weight matching is a perfect matching. Furthermore, by negating the weights of the edges we can state the problem as the following minimization problem: given a bipartite graph  $(A,B)$  and weight matrix  $W \in (\mathbb{R} \cup \{\infty\})^{n \times n}$ , find a perfect matching  $M$  with minimum weight. One of the fundamental results in combinatorial optimization is the polynomial-time blossom algorithm for computing minimum-weight perfect matchings by [Edm65].
We begin this section by showing a variance lower bound under the uniform distribution over the permutation group.
Lemma 10 (Variance Lower Bound). Let  $U(\mathbb{S}_n)$  be the uniform distribution over  $n \times n$  permutation matrices. For any matrix  $W \in \mathbb{R}^{n \times n}$ , with  $\sum_{i} W_{ij} = 0$  and  $\sum_{j} W_{ij} = 0$  we have
$$
\operatorname {V a r} _ {\Pi \sim U (\mathbb {S} _ {n})} [ W \cdot \Pi ] = \frac {\| W \| _ {\mathrm {F}} ^ {2}}{n - 1}.
$$
Proof. We have that  $\mathbf{E}_{\Pi \sim U(\mathbb{S}_n)}[\Pi_{ij}] = 1 / n$  and  $\mathbf{E}_{\Pi \sim U(\mathbb{S}_n)}[\Pi_{ij}\Pi_{ab}] = \frac{1\{i\neq a\wedge j\neq b\}}{n(n - 1)} +\frac{1\{i = a,j = b\}}{n}.$  We have
$$
\begin{array}{l} \mathbf {V a r} _ {\Pi \sim U (\mathbb {S} _ {n})} [ W \cdot \Pi ] = \underset {\Pi \sim U (\mathbb {S} _ {n})} {\mathbf {E}} [ (W \cdot \Pi) ^ {2} ] - \left(\underset {\Pi \sim U (\mathbb {S} _ {n})} {\mathbf {E}} [ W \cdot \Pi ]\right) ^ {2} \\ = \sum_ {i, j, a, b} W _ {a b} W _ {i j} \left(\frac {\mathbb {1} \{i \neq a , j \neq b \}}{n (n - 1)} + \frac {\mathbb {1} \{i = a , j = b \}}{n}\right) - \left(\sum_ {i, j} \frac {W _ {i j}}{n}\right) ^ {2} \\ = \frac {1}{n} \sum_ {i, j} W _ {i j} ^ {2} + \sum_ {i, j, a, b} W _ {i j} W _ {a b} \frac {\mathbb {1} \{i \neq a , j \neq b \}}{n (n - 1)} \\ = \frac {\| W \| _ {\mathrm {F}} ^ {2}}{n} + \sum_ {i, j, a, b} W _ {i j} W _ {a b} \frac {\mathbb {1} \{i \neq a , j \neq b \}}{n (n - 1)}, \\ \end{array}
$$
where to obtain the third equality we used our assumption that  $\sum_{ij}W_{ij} = 0$ . We observe that, by our assumption that  $\sum_{b}W_{ab} = 0$  for all  $a$  it holds  $\sum_{b\neq j}W_{ab} = -W_{aj}$  and therefore, we have
$$
\sum_ {b} W _ {a b} \mathbb {1} \{i \neq a, j \neq b \} = \mathbb {1} \{i \neq a \} \sum_ {b} W _ {a b} \mathbb {1} \{j \neq b \} = - \mathbb {1} \{i \neq a \} W _ {a j}.
$$
Similarly, using the fact that  $\sum_{a\neq i}W_{aj} = -W_{ij}$  we obtain that
$$
\sum_ {a b} W _ {a b} \mathbb {1} \{i \neq a, j \neq b \} = \sum_ {a} - W _ {a j} \mathbb {1} \{i \neq a \} = W _ {i j}.
$$
Therefore, using the above identity, we have that
$$
\sum_ {i, j, a, b} W _ {i j} W _ {a b} \frac {\mathbb {1} \{i \neq a , j \neq b \}}{n (n - 1)} = \sum_ {i, j} \frac {W _ {i j} ^ {2}}{n (n - 1)} = \frac {\| W \| _ {\mathrm {F}} ^ {2}}{n (n - 1)}.
$$
Combining the above we obtain the claimed identity.
Remark 8. We note that in MWBM the conditions  $\sum_{i}W_{ij} = 0$  and  $\sum_{j}W_{ij} = 0$  are without loss of generality.
We next claim that there exists an efficient algorithm for (approximately) sampling such permutation matrices.
Lemma 11 (Efficient Sampling). There exists an algorithm that generates approximate samples from the Gibbs distribution  $p(\cdot; W)$  with parameter  $W$  over the symmetric group, i.e.,  $p(\Pi; W) \propto \exp(W \cdot \Pi) 1\{\Pi \in \mathbb{S}_n\}$ , in poly(n) time.
Proof. This lemma essentially requires approximating the permanent of a weighted matrix, since this would imply that one has an approximation of the partition function. Essentially, our goal is to generate a random variable  $X$  that is  $\epsilon$ -close in statistical distance to the probability measure
$$
p (\Pi ; W) \propto \exp (W \cdot \Pi) 1 \{\Pi \in \mathbb {S} _ {n} \}.
$$
Note that the partition function is
$$
Z (W) = \sum_ {\Pi \in \mathbb {S} _ {n}} e ^ {W \cdot \Pi} = \sum_ {\Pi \in \mathbb {S} _ {n}} \prod_ {(i, j)} e ^ {W _ {i j} \Pi_ {i j}} = \sum_ {\sigma \in \mathbb {S} _ {n}} \prod_ {i \in [ n ]} A _ {i, \sigma (i)},
$$
where  $A$  is a non-negative real matrix with entries  $A_{ij} = \exp (W_{ij})$ . Note that in the third equality, we used the isomorphism between permutations and permutation matrices. Hence,  $Z(W)$  is exactly the permanent of the matrix  $A$ .
Proposition 10 ([JSV04]). There exists a fully polynomial randomized approximation scheme for the permanent of an arbitrary  $n \times n$  matrix  $A$  with non-negative entries.
To conclude the proof of the lemma, we need the following standard result.
Proposition 11 (See Appendix H and [Sin12, Jer03]). For self-reducible problems, fully polynomial approximate integration and fully polynomial approximate sampling are equivalent.
This concludes the proof since weighted matchings are self-reducible (see Appendix H).
The above lemma establishes our goal:
Theorem 5 (MWBM has a Compressed, Efficiently Optimizable and Samplable Solution Generator). Consider a prior over MWBM instances with  $n$  nodes. For any  $\epsilon > 0$ , there exists a solution generator  $\mathcal{P} = \{p(w) : w \in \mathcal{W}\}$  such that  $\mathcal{P}$  is complete, compressed with description  $\mathrm{poly}(n)\mathrm{polylog}(1 / \epsilon), \mathcal{L} + \lambda R : \mathcal{W} \mapsto \mathbb{R}$  is efficiently estimable via projected stochastic gradient descent in  $\mathrm{poly}(n, 1 / \epsilon)$  steps for some  $\lambda > 0$  and efficiently samplable in  $\mathrm{poly}(n, 1 / \epsilon)$  steps.
Proof. Consider an input graph  $G$  with  $n$  nodes and adjacency matrix  $E$ . The feature vector corresponding to a matching can be represented as a binary matrix  $\Pi \in \{0,1\}^{n\times n}$  with  $\sum_{j}\Pi_{ij} = 1$  for all  $i$  and  $\sum_{i}\Pi_{ij} = 1$  for all  $j$ , i.e.,  $\Pi$  is a permutation matrix. Then
$$
\mathsf {M W B M} = \max  _ {\Pi \in \mathbb {S} _ {n}} E \cdot \Pi = \min  _ {\Pi \in \mathbb {S} _ {n}} - E \cdot \Pi .
$$
Therefore, for a candidate matching  $s$ , we set  $\psi_S(s)$  to be the matrix  $\Pi$  defined above. Moreover, the feature vector of the graph is the negative (flattened) adjacency matrix  $-E^{\flat}$ . The cost oracle is then  $L(R; E) = -\sum_{ij} E_{ij} M_{ij} R_{ij}$  perhaps for an unknown weight matrix  $M_{ij}$  (see Remark 6). This means that the dimensions of the feature mappings  $\psi_S, \psi_{\mathcal{I}}$  are polynomial in the bit complexity of  $\mathcal{I}$ . Moreover, we get that  $\| \psi_{\mathcal{I}}(I) \|_F$ ,  $\| \psi_S(s) \|_F$ ,  $\| M \|_F \leq \mathrm{poly}(n)$ . We can employ Lemma 10 to get the variance lower bound under the uniform probability distribution in the subspace induced by the matrices satisfying Lemma 10, i.e., the matrices of the parameter space (see Remark 9). Finally, (approximate) sampling from our solution generators can be done efficiently using Lemma 11 and hence (noisy) projected SGD will have a runtime of order  $\mathrm{poly}(n, 1/\epsilon)$  (as in the case of Min-Cut).
We close this section with a remark about Item 3 of Assumption 1.
Remark 9. We note that Item 3 of Assumption 1 can be weakened. We use our variance lower bound in order to handle inner products of the form  $w \cdot x$  where  $w$  will lie in the parameter space and  $x$  is the featurization of a solution that lies in some space  $X$ . Hence it is possible that  $w$  lies in a low-dimensional subspace of  $X$ . For our optimization purposes, it suffices to provide variance lower bounds only in the subspace where  $w$  lies into.
G.2.2 Travelling Salesman Problem
Let us consider a weighted clique  $K_{n}$  with  $n$  vertices and weight matrix  $W \in \mathbb{R}^{n \times n}$ . A solution to the TSP instance  $W$  is a sequence  $\pi : [n] \to [n]$  of the  $n$  elements indicating the TSP tour  $(\pi(1), \pi(2), \ldots, \pi(n), \pi(1))$  and suffers a cost
$$
L (\pi) = \sum_ {i = 1} ^ {n - 1} W _ {\pi (i), \pi (i + 1)} + W _ {\pi (n), \pi (1)}.
$$
Crucially, the allowed sequences are a proper subset of all possible permutations. For instance, the permutations with fixed points or small cycles are not allowed. In particular, the solution space of TSP corresponds to the set of cyclic permutations with no trivial cycles, i.e., containing an  $n$ -cycle. Clearly, the number of  $n$ -cycles is  $(n - 1)!$ . The goal is to find a tour of minimum cost. Our first goal is to write the cost objective as a linear function of the weight matrix  $W$  and the feasible solutions, which correspond to cyclic permutations. To this end, we can think of each cyclic permutation  $\pi$  as a cyclic permutation matrix  $\Pi \in \{0,1\}^{n\times n}$ . Then, the desired linearization is given by  $L(\Pi) = W\cdot \Pi$  (for a fixed graph instance).
Our next task is to provide a Gibbs measure that generates random cyclic permutations. Let  $\mathbb{C}_n$  be the space of  $n\times n$  cyclic permutation matrices. Then we have that the tour  $\Pi$  is drawn from
$$
p _ {W} (\Pi) = \frac {\exp (W \cdot \Pi) 1 \left\{\Pi \in \mathbb {C} _ {n} \right\}}{\sum_ {\Pi^ {\prime} \in \mathbb {C} _ {n}} \exp (W \cdot \Pi^ {\prime})},
$$
where  $W$  is the weight matrix. The following key lemma provides guarantees for the performance of our approach to TSP. This lemma allows us to show that the number of optimization steps is  $\mathrm{poly}(n, 1 / \epsilon)$ .
Lemma 12 (Variance Lower Bound). Let  $U(\mathbb{C}_n)$  be the uniform distribution over  $n \times n$  cyclic permutation matrices. For any matrix  $W \in \mathbb{R}^{n \times n}$ , with  $\sum_{i} W_{ij} = 0$  and  $\sum_{j} W_{ij} = 0$  we have
$$
\mathbf {V a r} _ {\Pi \sim U (\mathbb {C} _ {n})} [ W \cdot \Pi ] \geq \frac {\| W \| _ {\mathrm {F}} ^ {2}}{(n - 1) (n - 2)}.
$$
Proof. The first step is to compute some standard statistics about cyclic permutations (see Lemma 13). Lemma 13 and the analysis of Lemma 10 gives us that  $\mathbf{Var}_{\Pi \sim U(\mathbb{C}_n)}[W \cdot \Pi]$  is equal to
$$
\frac {\| W \| _ {\mathrm {F}} ^ {2}}{n - 1} + \sum_ {i, j, a, b} W _ {i j} W _ {a b} \left(\frac {1 \{i \neq a = j \neq b \}}{(n - 1) (n - 2)} + \frac {1 \{j \neq b = i \neq a \}}{(n - 1) (n - 2)} + \frac {1 \{i \neq a \neq j \neq b \neq i \}}{(n - 1) (n - 3)}\right).
$$
Let us set
$$
A _ {1} = \sum_ {i, j, a, b} W _ {i j} W _ {a b} 1 \{i \neq a = j \neq b \}.
$$
$$
\sum_ {b} W _ {a b} 1 \{i \neq a \} 1 \{a = j \} 1 \{j \neq b \} = 1 \{i \neq a \} 1 \{a = j \} \sum_ {b} W _ {a b} 1 \{j \neq b \} = - 1 \{i \neq a \} 1 \{a = j \} W _ {a j}.
$$
$$
A _ {1} = - \sum_ {i, j, a} W _ {i j} W _ {a j} 1 \{i \neq a \} 1 \{a = j \} = - \sum_ {i, j} W _ {i j} W _ {j j} 1 \{i \neq j \} = \sum_ {j} W _ {j j} ^ {2}.
$$
Due to symmetry,  $A_{2} = \sum_{i,j,a,b} W_{ij} W_{ab} 1 \{j \neq b = i \neq a\} = \sum_{j} W_{jj}^{2}$ . It remains to argue about
$$
A _ {3} = \sum_ {i, j, a, b} W _ {i j} W _ {a b} 1 \{i \neq a \neq j \neq b \neq i \}.
$$
$$
\sum_ {b} W _ {a b} 1 \{i \neq a \} 1 \{a \neq j \} 1 \{j \neq b \} 1 \{b \neq i \} = - 1 \{i \neq a \} 1 \{a \neq j \} (W _ {a j} + W _ {a i}).
$$
$$
A _ {3} = - \sum_ {i, j, a} W _ {i j} (W _ {a j} + W _ {a i}) 1 \{i \neq a \} 1 \{a \neq j \}.
$$
$$
\sum_ {i, j, a} W _ {i j} W _ {a j} 1 \{i \neq a \} 1 \{a \neq j \} = \sum_ {j, a} W _ {a j} 1 \{a \neq j \} \sum_ {i} W _ {i j} 1 \{i \neq a \} = - \sum_ {j, a} W _ {a j} ^ {2} 1 \{a \neq j \}.
$$
$$
A _ {3} = \sum_ {j \neq a} W _ {a j} ^ {2} + \sum_ {i \neq a} W _ {a i} ^ {2} = 2 \sum_ {j \neq a} W _ {a j} ^ {2}.
$$
In total, this gives that
$$
\mathbf {V a r} _ {\Pi \sim U (\mathbb {C} _ {n})} [ W \cdot \Pi ] = \frac {\| W \| _ {\mathrm {F}} ^ {2}}{n - 1} + \frac {2 \sum_ {j} W _ {j j} ^ {2}}{(n - 1) (n - 2)} + \frac {2 \sum_ {i \neq j} W _ {i j} ^ {2}}{(n - 1) (n - 3)} \geq \frac {\| W \| _ {\mathrm {F}} ^ {2}}{n - 1} + \frac {\| W \| _ {\mathrm {F}} ^ {2}}{(n - 1) (n - 2)}.
$$
Remark 10. We note that in TSP the conditions  $\sum_{i}W_{ij} = 0$  and  $\sum_{j}W_{ij} = 0$  are without loss of generality.
The next lemma is a generic lemma that states some properties of random cyclic permutation matrices.
Lemma 13. Consider a uniformly random cyclic permutation matrix  $\Pi$ . Let us fix  $i \neq j$  and  $a \neq b$ . Then