SlowGuess/NeurIPS_2023_OCR huggingface dataset:
$\mathbf{E}[\Pi_{ij}] = \frac{1}{n - 1}$
-  $\mathbf{E}[\Pi_{ij}\Pi_{ab}] = \frac{1\{i\neq a = j\neq b\}}{(n - 1)(n - 2)} +\frac{1\{j\neq b = i\neq a\}}{(n - 1)(n - 2)} +\frac{1\{i\neq a\neq j\neq b\neq i\}}{(n - 1)(n - 3)} +\frac{1\{i = a,j = b\}}{n - 1}.$
Proof. First, note that any matrix that corresponds to a cyclic permutation does not contain fixed points and so the diagonal elements are 0 deterministically. For the first item, the number of cyclic permutations such that  $i \to j$  (i.e.,  $\Pi_{ij} = 1$ ) is  $(n - 2)!$ . This implies that the desired expectation is  $(n - 2)! / |\mathbb{C}_n| = 1 / (n - 1)$ . For the second item, if  $i = a, j = b$ , we recover the first item. Otherwise if  $i = a$  or  $j = b$ , then the expectation vanishes since we deal with permutation matrices. Finally, let us consider the case where  $i \neq a$  and  $j \neq b$ . Our goal is to count the number of cyclic permutations with  $i \to j$  and  $a \to b$ .
- If  $i \neq a \neq j \neq b \neq i$ , then there are  $n$  choices to place  $i$  and  $n - 2$  choices to place  $a$ . Then there are  $(n - 4)!$  possible orderings for the remaining elements. This gives an expectation equal to  $1 / ((n - 1)(n - 3))$ .
- If  $i \neq a = j \neq b$  or  $j \neq b = i \neq a$ , then there are  $n$  choices for  $i$  and  $(n - 3)!$  orderings for the remaining elements. Hence, the expectation is  $1 / ((n - 1)(n - 2))$ .
We note that sampling from our solution generators is the reason that we cannot find an optimal TSP solution efficiently. In general, an algorithm that has converged to an almost optimal parameter  $W^{\star}$  has to generate samples from the Gibbs measure that is concentrated on cycles with minimum weight. In this low-temperature regime, sampling is NP-hard. We are now ready to state our result.
Theorem 6 (TSP has a Compressed, Efficiently Optimizable Solution Generator). Consider a prior over TSP instances with  $n$  nodes. For any  $\epsilon > 0$ , there exists a solution generator  $\mathcal{P} = \{p(w): w \in \mathcal{W}\}$  such that  $\mathcal{P}$  is complete, compressed with description  $\mathrm{poly}(n)$  polylog(1/ε) and  $\mathcal{L} + \lambda R: \mathcal{W} \mapsto \mathbb{R}$  is efficiently estimable via projected stochastic gradient descent in  $\mathrm{poly}(n, 1/\epsilon)$  steps for some  $\lambda > 0$ .
Proof. Consider an input graph  $G$  with  $n$  nodes and weighted adjacency matrix  $E$ . The feature vector is again a permutation matrix  $\Pi$  with the additional constraint that  $\Pi$  has to represent a single cycle (a tour over all cities). Then
$$
\mathrm{TSP} = \min_{\Pi \in \mathbb{C}_{n}}E\cdot \Pi .
$$
The cost function for TSP is  $L(\Pi; E) = \sum_{ij} E_{ij} M_{ij} \Pi_{ij}$ . We refer to Theorem 5 for the details about the feature mappings. We can finally use Lemma 12 to obtain a variance lower bound (in the subspace induced by the parameters satisfying this lemma, see Remark 9) under the uniform measure over the space of cyclic permutation matrices  $\mathbb{C}_n$ .
H Sampling and Counting
In this section, we give a quick overview of the connections between approximate sampling and counting. For a formal treatment, we refer to [Sin12].
In what follows,  $\sigma$  may be thought of as an encoding of an instance of some combinatorial problem, and the  $\omega$  of interest are encodings of the structures we wish to generate. Consider a weight function  $W$  and assume that  $W(\sigma, \omega)$  is computable in time polynomial in  $|\sigma|$ .
Definition 4 (Approximate Sampling). A fully polynomial approximate sampler for  $(\Omega_{\sigma},\pi_{\sigma})$  is a Probabilistic Turing Machine which, on inputs  $\sigma$  and  $\epsilon \in \mathbb{Q}_{+}$ $(0 < \epsilon \leq 1)$ , outputs  $\omega \in \Sigma^{\star}$ , according to a measure  $\mu_{\sigma}$  satisfying  $\mathrm{TV}(\pi_{\sigma},\mu_{\sigma})\leq \epsilon$ , in time bounded by a bivariate polynomial in  $|\sigma|$  and  $\log(1/\epsilon)$ .
One of the main applications of sampling is to approximate integration. In our setting this means estimating  $Z(\sigma)$  to some specified relative error.
Definition 5 (Approximate Integration). A fully polynomial randomized approximation scheme for  $Z(\sigma)$  is a Probabilistic Turing Machine which on input  $\sigma, \epsilon$ , outputs an estimate  $\widehat{Z}$  so that
$$
\operatorname * {P r} [ Z / (1 + \epsilon) \leq \widehat {Z} \leq (1 + \epsilon) Z ] \geq 3 / 4,
$$
and which runs in time polynomial in  $|\sigma|$  and  $1 / \epsilon$ .
Definition 6 (Self-Reducible Problems). An NP search problem is self-reducible if the set of solutions can be partitioned into polynomially many sets each of which is in a one-to-one correspondence with the set of solutions of a smaller instance of the problem, and the polynomial size set of smaller instances are efficiently computable.
For instance, consider the relation MATCH which associates with an undirected graph  $G$  all matchings (independent sets of edges) of  $G$ . Then MATCH is self-reducible since, for any edge  $e = (u, v) \in E(G)$ , we have that
$$
\operatorname {M A T C H} (G) = \operatorname {M A T C H} \left(G _ {1}\right) \cup \left\{M \cup \{e \}: M \in \operatorname {M A T C H} \left(G _ {2}\right) \right\},
$$
where  $G_{1}$  is the graph obtained by deleting  $e$  and  $G_{2}$  is the graph obtained by deleting both  $u$  and  $v$  together with all their incident edges.
Theorem 7 (See Corollary 3.16 in [Sin12]). For self-reducible problems, approximate integration and good sampling are equivalent.
We remark that the above result holds for the more general class of self-partitionable problems.
I Details of the Experimental Evaluation
We investigate the effect of the entropy regularizer (see Equation (2)) in a very simple setting: we try to find the Max-Cut of a fixed graph  $G$ , i.e., the support of the prior  $\mathcal{R}$  is a single graph. We show that while the unregularized objective is often "stuck" at sub-optimal solutions – and this happens even for very small instances (15 nodes) – of the Max-Cut problem, the regularized version (with the fast/slow mixture scheme) is able to find the optimal solutions. We consider an instance randomly generated by the Erdős-Rényi model  $G(n,p)$  and then optimize the "vanilla" loss  $\mathcal{L}$  and the regularized loss  $\mathcal{L}_{\lambda}$  defined in Equation (3). The solutions are vectors  $s \in \{\pm 1\}^n$ . We first use the feature mapping  $\psi_S(s) = (ss^\top)^b$  described in Section 1.1 and an exponential family solution generator that samples a
class FastSlowMixture(torch.nn-module):   
def __init__(self, dimension, rho):   
```
The Model parameters.   
```
super().__init__()   
self.l1 = torch.nn_PARAMETER(torch.empty(30, dimension)) torch.nn.init.kaiming.uniform_(self.l1, a=5**0.5)   
self.l2 = torch.nn_PARAMETER(torch.empty(10, 30)) torch.nn.init.kaiming.uniform_(self.l2, a=5**0.5)   
self.l3 = torch.nn_PARAMETER(torch.empty(1, 10)) torch.nn.init.kaiming.uniform_(self.l3, a=5**0.5)   
self.a2 = torch.nn.ReLU()   
self.a1 = torch.nn.ReLU()   
self.rho = rho   
def forward(self, x, is_cold=True):   
temp = self.rho * (1. - is_cold) + is_cold   
out = x   
out = torch.nnfunctional.Linear(out, temp * self.l1)   
out = self.a1(out)   
out = torch.nnfunctional.Linear(out, temp * self.l2)   
out = self.a2(out)   
out = torch.nnfunctional.Linear(out, temp * self.l3)   
return out
Figure 4: Our implementation of the fast/slow network. The output of the network is the log-density (score) of a solution  $s \in \{\pm 1\}^{\mathrm{dimension}}$ . If evaluated with the is-cold set to False, the parameters of every linear layer are re-scaled by the inverse temperature rho.
solution  $s$  with probability  $\propto \exp (w\cdot \psi_S(s))$  for some weight vector  $w\in \mathbb{R}^{n^2}$ . We also consider optimizing a simple 3-layer ReLU network as solution generator with input  $s\in \{\pm 1\} ^n$  on the same random graphs. We generate 100 random  $G(n,p)$  graphs with  $n = 15$  nodes and  $p = 0.5$  and train solution generators using both the "vanilla" and the entropy-regularized loss functions. We perform 600 iterations and, for the entropy regularization, we progressively decrease the regularization weight, starting from 10, and dividing it by 2 every 60 iterations. We used a fast/slow mixing with mixture probability 0.2 and inverse temperature rho=0.03 (see Figure 4).
For convenience, we present a pytorch implementation of our simple 3-layer ReLU network here. For more details we refer to our full code submitted in the supplementary material.
Out of the 100 trials we found that our proposed objective was always able to find the optimal cut while the model trained with the vanilla loss was able to find it for approximately  $65\%$  of the graphs (for 65 out of 100 using the linear network and for 66 using the ReLU network). In Figure 2 we show two instances where the model trained with the "vanilla" loss gets stuck on a sub-optimal solution while the entropy-regularized one succeeds in finding the optimal solution. Our experiments show that the regularization term and the fast/slow mixture scheme that we introduced to achieve our main theoretical convergence result, see Section 3 and Proposition 4, are potentially useful for training more realistic models for bigger instances and we leave more extensive experimental evaluation as an interesting direction for future work.
We note that, similarly to our theoretical results, our sampler in this experimental section is of the form  $e^{\mathrm{score}(s;w)}$ , where  $s \in \{-1,1\}^n$  (here  $n$  is the number of nodes in the graph) is a candidate solution of the Max-Cut problem. The function used is a 3-layer MLP (see Figure Figure 4). Since the instances that we consider here are small ( $n = 15$ ) we can explicitly compute the density (score) of every solution and use that to compute the expected gradient. For larger instances, one could use some approximate sampler (e.g., via Langevin dynamics) to generate samples. The main message of the current experimental section is that even for very small instances of Max-Cut (i.e., with 15 nodes), optimizing the vanilla objective is not sufficient and the iteration gets trapped in local optima. In contrast, our entropy regularized always manages to find the optimal cut.