SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Input:  $\mathbf{V}^t$ : current sample,  $U$ : potential energy function,  $\varepsilon$ : step size
Output: next sample:  $\mathbf{V}^{t + 1}$
1:  $\mathbf{r} \sim \mathcal{N}(\mathbf{0}, \mathbb{I})$
2:  $\mathbf{r} \gets \mathbf{r} - \frac{\varepsilon}{2} \nabla U(\mathbf{V}^t)$
3:  $\mathbf{V}^{t + 1}\gets \operatorname {Proj}_{\mathcal{B}}(\mathbf{V}^t +\varepsilon \mathbf{r})$
4: return  $\mathbf{V}^{t + 1}$
Algorithm 4 Structured Voronoi Sampling
Input:  $\mathbf{x}^t$ : current embeddings,  $U$ : potential function,  $\varepsilon$ : step size
1:  $\mathbf{r} \sim \mathcal{N}(\mathbf{0}, \varepsilon \mathbb{I})$
2:  $H^{t}\gets U(\mathbf{V}_{\boldsymbol{m}^{\star}(\mathbf{x}^{t})}) + K(\mathbf{r})$
3:  $\mathbf{r} \gets \mathbf{r} - \frac{\varepsilon}{2} \nabla U\left(\mathbf{V}_{\mathbf{m}^* (\mathbf{x}^t)}\right)$
4:  $\mathbf{x}^{t + 1}\gets$  FINDDISC(xt,r)
5:  $\mathbf{r} \gets \mathbf{r} - \frac{\varepsilon}{2} \nabla U(\mathbf{V}_{\mathbf{m}^{\star}(\mathbf{x}^{t + 1})})$
6:  $H_{t + 1}\gets \bar{U} (\mathbf{x}^{t + 1}) + K(\mathbf{r})$
7:  $\Delta H\gets H^{t + 1} - H^t$
8: if  $\mathbf{s} \sim \mathcal{U}(0,1) < e^{-\Delta H}$ :
9: return  $\mathbf{x}^{t + 1}$
11: return  $\mathbf{x}^{t + 1}\gets \mathbf{x}^t$
$\triangleright$  Metropolis criterion
Algorithm 5 REFRACTREFLECT
Input:  $\mathbf{r}^t$ : current momentum,  $b$  normal vector of the boundary,  $\Delta U$ : difference in potential energy  
Output: next momentum:  $\mathbf{r}^{t + 1}$
1:  $\mathbf{r}_{\perp}^{t}\gets \frac{\mathbf{b}^{T}\mathbf{r}^{t}}{\|\mathbf{b}\|_{2}^{2}}\mathbf{b}$
2:  $\mathbf{r}_{\parallel}^{t}\gets \mathbf{r}^{t} - \mathbf{r}_{\perp}^{t}$
3: if  $\| \mathbf{r}^{t}_{1}\|_{2}^{2} > 2\Delta U$
4:  $\mathbf{r}_{\perp}^{t + 1}\gets \sqrt{\|\mathbf{r}_{\perp}^{t}\|_{2}^{2} - 2\Delta U}\cdot \frac{\mathbf{r}_{\perp}^{t}}{\|\mathbf{r}_{\perp}^{t}\|_{2}^{2}}$
5: else
6:  $\mathbf{r}_{\perp}^{t + 1}\gets -\mathbf{r}_{\perp}^{t}$
7:  $\mathbf{r}^{t + 1}\gets \mathbf{r}_{\perp}^{t + 1} + \mathbf{r}_{\parallel}^{t}$
8: return  $\mathbf{r}^{t + 1}$
$\triangleright$  refract
$\triangleright$  reflect
Split	Chinese	English	Fast food	French	Indian	Italian	Japanese	Total
train	2929	4100	5891	5889	4412	5909	5996	42061
valid	1489	1780	-	-	-	-	-	4672
test	492	613	632	639	497	608	638	4693
Table 2: Number of restaurant reviews in each split and food type. Note that some reviews do not represent any specific food type, therefore, the total count is bigger than the sum count of reviews in each food category.
F.1 Efficiently Finding Discontinuities
As explained in §6, the key insight in SVS is to adjust the momentum when facing discontinuity in the potential function. Therefore, we first need to find discontinuities efficiently and then reflect/refract on the boundary of the discontinuity. Remember that at each leapfrog step, we move from  $\mathbf{x}^t$  to  $\mathbf{x}^{t + 1} = \mathbf{x}^t +\varepsilon \mathbf{r}$ . To find discontinuities along this trajectory, we divide each leapfrog step into fractions of a step and check for discontinuity. Concretely, we only advance the embedding by a fraction  $\alpha$ , i.e.  $\mathbf{x}' = \mathbf{x} + \alpha \varepsilon \mathbf{r}$  (line 4 in Algorithm 6). Then, we check for discontinuity by looking at the difference between the potential energies. If we don't observe any changes in the potential function, we take the next fraction of a step. Otherwise, we find the boundary and adjust the momentum (line 10 in Algorithm 6). Note that in SVS, finding the boundary is straightforward, and it is a hyperplane characterized by the normal vector, as the line goes through the two Voronoi centers on each side of the hyperplane.
Algorithm 6 Find Discontinuity
Input:  $\mathbf{x}^t$ : current embeddings, r: current momentum,  $U$ : potential function,  $\varepsilon$ : step size,  $\alpha$ : discontinuity step size  
Output: next sample:  $\mathbf{x}^{t+1}$   
1:  $\tau \gets 0$   
2:  $\mathbf{x} \gets \mathbf{x}^t$   
3: while  $\tau < 1$ :  
4:  $\mathbf{x}' \gets \mathbf{x} + \alpha \varepsilon \mathbf{r}$   
5:  $\Delta U \gets U(\mathbf{V}_{m^*(\mathbf{x}')}) - U(\mathbf{V}_{m^*(\mathbf{x})})$   
6: if  $\Delta U = 0$ : there is no discontinuity  
7:  $\mathbf{x} \gets \mathbf{x}'$   
8:  $\tau \gets \tau + \alpha$   
9: else  
10:  $\mathbf{b}, \alpha' \gets \mathrm{FINDB}(\mathbf{x}, \mathbf{x}')$  Returns the intersection  $\alpha'$  and the normal vector of the boundary b  
11:  $\mathbf{x} \gets \mathbf{x} + \alpha' \varepsilon \mathbf{r}$   
12:  $\tau \gets \tau + \alpha'$   
13:  $\mathbf{r} \gets \mathrm{REFRACTREFLECT}(\mathbf{r}, \mathbf{b}, \Delta U)$   
14: return  $\mathbf{x}^{t+1} \gets \mathbf{x}$
G Dataset Statistics
This dataset is made available under the CC BY-SA 4.0 license. Our use of this dataset is for i) fine-tuning GPT-2, and ii) training classifiers for topic control, which clearly matches the intended use of this dataset mentioned by the creators as end-to-end training and generating text with content selection. A summary of dataset statistics can be found in Table 2.
Sampling algorithms. Hyperparameters for each experiment are reported in Table 4. Following prior work [46], in algorithms based on Langevin dynamics, we apply an exponential decay to the step size by decreasing it to 0.05 after 500 steps. In all settings, we take 500 burn-in steps.
Toy Example	LM Sampling	Controlled Generation
ε	α	ε	α	ε	α	γ
VS	0.1	0.1	-	-	-	-	-
SVS	0.1	0.1	1.	0.4	1.5	0.3	1.5
HMC	0.1	0.1	-	-	-	-	-
LANGEVIN	-	-	1.	-	1.5	-	1.5
MUOLA	0.1	0.1	1.	-	1.	-	2.
Table 4: Hyperparameters used in sampling algorithms.
Food Classifiers. We train 3 classifiers. First, for experiments with FUDGE, we follow the experimental detail as in the original paper [48] and train a 3-layered BILSTM classifier with 0.5 dropout. The hidden dimension is set to 300, and the embedding layer is trained from scratch. Second, in experiments with MUCoLA, LANGEVIN, and SVS, to make the setup as close as to FUDGE, we train a 3-layered BILSTM classifier with 0.5 dropout. However, this time the BILSTM is trained on top of frozen GPT-2 representations, thus sharing the embedding layer that is necessary for these methods to work. Finally, to evaluate the success of the methods in following the control targets,
we finetune a RoBERTA [25] base model. The accuracy of all the classifiers and the number of trained parameters are reported in Table 5. We train all the models on a single gtx_1080_ti GPU with approximately 2 hours of total computational budget.
Table 3: Inference time for different methods on the controlled generation experiments. All experiments are done on a single A100-40GB GPU.
method	sec/batch
FUDGE	10
MUCoLA	30
LANGEVIN (ours)	31
SVS (ours)	84
Inference times. We report inference times based on sec/batch. The number of decoded sentences per batch depends on the GPU memory and the size of the model. As depicted in table 3, using Voronoi measures does not increase the inference time (compare MUCOLA and LANGEVIN). We observe that SVS inference time is longer, because of the extra momentum adjustment steps. However, one can reduce the inference time by increasing  $\alpha$ .
model	f1-score	precision	recall	# params
BiLSTM	0.87	0.87	0.87	17M
BiLSTMPROBE	0.84	0.84	0.84	37M
ROBERTA	0.90	0.91	0.90	124M
BiLSTMPROBE	0.90	0.90	0.90	878M
Table 5: Performance of food classifiers, and their number of learnable parameters, used in controlled generation experiment. All classifiers are trained and tested on E2E dataset.
I More Experiments on The Toy Model
To better understand the advantages of Voronoi sampling over HMC or MUCOLA, we further look at the JS divergence between the reference distribution and the distribution of samples when increasing the number of iterations. As depicted in Fig. 4, while the divergence decreases with more iterations across all sampling methods, Voronoi sampling converges to the reference distribution with fewer iterations. We then look at the distribution of sampled elements in Fig. 5. We observe that with 100 iterations, MUCOLA undersamples the element with the maximum probability while oversampling other elements. Finally, we extend the toy model from 4 squared cells in  $\mathbb{R}^2$  to  $2^{k}$  hypercube cells in  $\mathbb{R}^k$  (Fig. 6). As the dimensionality increases, the divergence between the samples' distribution and the reference distribution also increases in all sampling methods. Importantly, Voronoi sampling consistently converges faster across different values for  $k$ .
(a) 500 iterations
(b) 1000 iterations
(c) 100 iterations
Figure 4: Comparing JS divergence of methods using different numbers of iterations. In general, Voronoi sampling converge to the true distribution faster compared to HMC and MUCOLA. As the number of iterations increases, the divergence between the samples' distribution and the true distribution decreases across all sampling methods.
J Controlled Generation Results
Table 6, Fig. 7 show the performance of 4 sampling methods on different metrics. We show a sample of generations for each control target in Table 7.
FUDGE	MUCoLA	LANGEVIN	SVS
Success	PPL	Dist-1	Dist-2	Dist-3	Success	PPL	Dist-1	Dist-2	Dist-3	Success	PPL	Dist-1	Dist-2	Dist-3	Success	PPL	Dist-1	Dist-2	Dist-3
Chinese	0.30	5.41	0.38	0.53	0.63	0.30	10.90	0.21	0.35	0.47	1.00	16.21	0.22	0.36	0.48	0.90	12.42	0.22	0.36	0.49
English	0.15	5.82	0.41	0.57	0.67	0.45	17.76	0.26	0.39	0.49	0.70	15.82	0.27	0.43	0.55	0.85	15.00	0.25	0.41	0.54
Fast food	0.25	6.44	0.41	0.57	0.68	0.95	5.39	0.26	0.38	0.47	1.00	10.09	0.23	0.37	0.49	1.00	11.43	0.23	0.38	0.50
French	0.25	6.02	0.39	0.55	0.66	0.75	88.19	0.29	0.43	0.54	0.80	12.87	0.21	0.36	0.49	0.95	17.23	0.21	0.35	0.47
Indian	0.55	4.52	0.41	0.55	0.64	0.40	7.87	0.25	0.40	0.51	0.95	17.67	0.26	0.41	0.54	0.95	12.89	0.19	0.35	0.47
Italian	0.35	5.49	0.37	0.53	0.64	0.50	18.19	0.27	0.42	0.53	0.95	14.29	0.26	0.41	0.53	0.90	15.47	0.26	0.41	0.52
Japanese	0.30	5.44	0.42	0.55	0.65	0.75	83.37	0.27	0.42	0.54	1.00	12.90	0.21	0.36	0.47	0.95	12.89	0.18	0.33	0.45
Table 6: Evaluation of different sampling methods on controlled generation, using three criteria: success in following the control target (measured by the evaluator classifier), fluency (measured by perplexity), and diversity.
(a) 500 iterations
(b) 1000 iterations
(c) 100 iterations
Figure 5: Comparing the distribution of sampled elements at temperature 0.25. With 100 iterations, MUCoLA undersamples the element with the highest probability while oversampling other elements.
Figure 6: Comparing the distribution of sampled elements with the true distribution after 100 iterations, at temperature 0.5. There are  $2^{k}$  Voronoi cells with equal base measures in  $\mathbb{R}^k$ , where elements to sample from are located at the center of each Voronoi cell. Voronoi sampling converges faster to the true distribution across all  $k$  values. As the dimensionality of the sample space increases, the divergence of all methods increases.
MuCoLa
Figure 7: Evaluation of different sampling methods on restaurant review generation, along 6 axes: mean and standard deviation of negative perplexity $^{19}$ (-ppl-mean ↑, -ppl-standard ↑), the percentage of generated sentences adhering to the control target (success ↑), and diversity metrics (dist-1 ↑, dist-2 ↑, dist-3 ↑). For most control targets, SVS achieves the highest success rate, with relatively low perplexity.
Chinese
FUDGE MUCOLA LANGEVIN SVS	In the city centre near Yippee Noodle Bar Chinese, is Alimentum. It has moderate prices and and has a 1 out of 5. It has food and high customer rating. The Rice Boat is It serves Chinese food with a low customer rating. The fast food and restaurant The Golden Curry is a It has a low customer rating and a price. The highly rated Chinese restaurant The Phoenix has a high English
FUDGE MUCOLA LANGEVIN SVS	It has an average customer Rating. Bibimbap House has English food in the riverside area near and has a low customer rating. The Golden Curry is a children friendly, serving English food, with It has low rating and is located near the to the city centre. The Phoenix is a English food Alimentum in the city centre near the a moderate price range. It serves English food, is
Fast food
FUDGE MUCOLA LANGEVIN SVS	A fast food, coffee shop, Strada has a low customer rating, has a price range of over £30. It is and is family friendly and serves fast food. The Wrestlers is a fast food coffee shop in the It is located near the riverside, is a cheap family friendly fast food restaurant, and is called It is located near the river. The Mill is a cheap, fast food and coffee shop near the French
FUDGE MUCOLA LANGEVIN SVS	It has a low-priced Inn French food. It is near Café Rouge.The Alimentum is a kid friendly fast food The French restaurant The Waterman is located in the city centre. The price range is less than It is a restaurant located in the riverside, the restaurant, offers French food with a price It is a family restaurant that serves French food with a price range and has a low customer rating.
Indian
FUDGE MUCOLA LANGEVIN SVS	The Phoenix Indian restaurant has moderate prices with a 3 out of 5 rating. Located on the It is in the city and has a low customer rating. The Waterman is a low priced It is not child friendly and it is near the river. It serves Indian food and a customer rating It is located in the city centre near The Portland Arms Indian food and has a low customer rating.
Italian
FUDGE MUCOLA LANGEVIN SVS	It has family Italian food and has a low a moderate price range. The Rice Boat has an average is a high priced Italian food restaurant with a customer rating of average. The Phoenix is a high It is located in the city centre, it is not family friendly and is a coffee shop serving Italian It is located in the the city centre near The Portland Arms.The Eagle is an Italian restaurant.
Japanese
FUDGE MUCOLA LANGEVIN SVS	Japanese food. Its customer rating is 3 out of 5.The Phoenix is Japanese in the city centre for Japanese food is located in the city centre. It has a low customer rating. The Golden It is located in the riverside. It is a Japanese food. It is a pub restaurant It is located in the riverside. It is a low rated Japanese restaurant, and coffee shop.
Table 7: Examples of sampled sentences from different control food targets.