SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Oren Mangoubi  
Worcester Polytechnic Institute
Given a Lipschitz or smooth convex function  $f: K \to \mathbb{R}^d$  for a bounded polytope  $K := \{\theta \in \mathbb{R}^d : A\theta \leq b\}$ , where  $A \in \mathbb{R}^{m \times d}$  and  $b \in \mathbb{R}^m$ , we consider the problem of sampling from the log-concave distribution  $\pi(\theta) \propto e^{-f(\theta)}$  constrained to  $K$ . Interest in this problem derives from its applications to Bayesian inference and differential privacy. We present a generalization of the Dikin walk to this setting that requires at most  $O((md + dL^2 R^2) \times md^{\omega - 1} \log(\frac{w}{\delta}))$  arithmetic operations to sample from  $\pi$  within error  $\delta > 0$  in the total variation distance from a  $w$ -warm start. Here  $L$  is the Lipschitz constant of  $f$ ,  $K$  is contained in a ball of radius  $R$  and contains a ball of smaller radius  $r$ , and  $\omega \approx 2.37$  is the matrix-multiplication constant. This improves on the running time of prior works for a range of structured settings important for the aforementioned inference and privacy applications. Technically, we depart from previous Dikin walks by adding a soft-threshold regularizer derived from the Lipschitz or smoothness properties of  $f$  to a barrier function for  $K$  that allows our version of the Dikin walk to propose updates that have a high Metropolis acceptance ratio for  $f$ , while at the same time remaining inside the polytope  $K$ .
We consider the problem of sampling from a log-concave distribution supported on a polytope: Given a polytope  $K \coloneqq \{\theta \in \mathbb{R}^d : A\theta \leq b\}$ , where  $A \in \mathbb{R}^{m \times d}$  and  $b \in \mathbb{R}^m$ , and a convex function  $f: K \to \mathbb{R}$ , output a sample  $\theta \in K$  from the distribution  $\pi(\theta) \propto e^{-f(\theta)}$ . Our interest in this problem derives from its applications to Bayesian inference and differentially private optimization. In Bayesian inference, the ability to sample from  $\pi(\theta) \propto e^{-f(\theta)}$  allows one to compute Bayesian confidence intervals and other statistics for the Bayesian posterior distribution of many machine learning models (see e.g. [17, 20, 6]). In differentially private optimization, sampling from the "exponential mechanism" [36] allows one to get optimal utility bounds for the problem of minimizing  $f$  under  $\varepsilon$ -differential privacy [2].
The instances of the polytope-constrained sampling problem that arise in these applications are more general than the two well-studied special cases-- the uniform density case ( $f \equiv 0$ ) and the unconstrained case ( $K = \mathbb{R}^d$ ). However, they still have more structure than the case of a general log-concave function supported on an arbitrary convex body. For instance, in Bayesian Lasso logistic regression,  $f(\theta) = \sum_{i=1}^{n} \ell(\theta; x_i)$ , where  $\ell$  is the logistic loss and  $x_i$  are datapoints with  $\|x_i\|_2 \leq 1$ , and  $K = \{\theta \in \mathbb{R}^d : \| \theta \|_1 \leq O(1)\}$ ; see [44, 43, 26, 47]. Since the logistic function is both  $O(1)$ -smooth and  $O(1)$ -Lipschitz,  $f$  is both  $O(n)$ -Lipschitz and  $O(n)$ -smooth, and  $K$  is defined by  $O(d)$  inequalities and contained in a ball of radius  $O(1)$ .
To obtain an  $\varepsilon$ -differentially private mechanism for the Lasso logistic regression problem, using the exponential mechanism, the goal is to sample from  $\exp \left(-\frac{\varepsilon}{R}\sum_{i=1}^{n}\ell(\theta; x_i)\right)$ , where  $\ell$  is the logistic loss and  $K$  is contained in a ball of radius  $R$ . Thus, the log-density is both
$\beta$ -smooth and  $L$ -Lipschitz for  $\beta = L = n\varepsilon / R = O(d)$  if  $n = d$  and  $\varepsilon < 1$ , since  $R = O(1)$ . Another example is a result of [28] that reduces the problem of  $\varepsilon$ -differentially private low-rank approximation of a symmetric  $p \times p$  matrix to a constrained sampling problem where  $f$  is linear (and thus 0-smooth) of dimension  $d = p^2$  and  $K$  is the Gelfand-Tsetlin polytope (a generalization of the probability simplex). Here  $K$  has  $d$  inequalities and diameter  $O(\sqrt{d})$ .
Importantly, when sampling from the exponential mechanism in privacy applications, sampling with total variation (TV) bounds—the case that has received the most attention—is insufficient to guarantee  $\varepsilon$ -differential privacy, the strongest notion of differential privacy; see [14]. Instead, one requires bounds in the stronger infinity-distance metric  $\mathrm{d}_{\infty}(\nu, \pi) := \sup_{\theta \in K} |\log^{(\nu(\theta) / \pi(\theta))}|$ . A recent work [35] showed how to convert samples within  $O(\delta)$ -TV distance from continuous log-Lipschitz densities  $\pi$ , into samples with  $O(\varepsilon)$ -infinity-distance bounds, but it requires the TV distance  $\delta$  to be very small—roughly  $\delta = O(\varepsilon e^{-d - LR})$ , raising the question of designing Markov chains whose runtime bounds also have a low-order dependence on  $\log^{1/\delta}$ . Thus, for the aforementioned applications to Bayesian inference and privacy which give rise to structured instances of sampling from log-concave densities over polytopes, it is desirable to design sampling algorithms that have a low-order polynomial dependence not only on the parameters  $d, L, R, \beta$ , but also on  $\log^{1/\delta}$ .
Main related works. A line of work has developed algorithms for sampling in the general setting when  $K$  is an arbitrary convex body given by a membership oracle [16, 1, 15, 33, 34]. [33] use the "hit-and-run" framework to give an algorithm to sample from a log-concave distribution  $\pi \propto e^{-f}$  on a convex body  $K$  which also contains a ball of radius  $r$  with TV-error  $\delta > 0$  in  $O(d^{2}(R / r)^{2}\log^{2}\bigl (wdR / (r\delta)\bigr)\log^{3}(w / \delta))(T_{f} + \hat{T}_{K}))$  arithmetic operations from a  $w$ -warm start (their Theorem 1.1) and  $O(d^{3}(R / r)^{2})\log^{5}(dR^{2} / (\delta r)(T_{f} + \hat{T}_{K}))$  arithmetic operations from a cold start (Corollary 1.2). Here  $T_{f}$  is the time required to evaluate  $f$  and  $\hat{T}_K$  is the time for a membership oracle query. Here, a distribution  $\nu$  is  $w$ -warm for  $w \geq 1$  w.r.t. the stationary distribution  $\pi$  if  $\sup_{z\in K}\nu (z) / \pi (z)\leq w$ .
[38] give a "Dikin-walk"-based algorithm to sample from any log-concave  $\pi \propto e^{-f}$  on  $K$  where  $f$  is  $L$ -Lipschitz or  $\beta$ -smooth. The Dikin walk Markov chain was introduced in [24] in the special case where  $f \equiv 0$  (see also [37]). Their runtime is  $O((d^{5} + d^{3}L^{2}R^{2})\log(w / \delta)(T_{f} + md^{\omega - 1}))$  arithmetic operations, where  $\omega = 2.37\dots$  is the matrix-multiplication constant. From a cold start, their runtime is  $O((d^{5} + d^{3}L^{2}R^{2})\log(1 / \delta)(d\log(R / r) + M + \log(1 / \delta))(T_{f} + md^{\omega - 1})$ , where  $M := \log(\max_{\theta \in K}e^{f(a) - f(\theta)})$ . Their bounds when  $f$  is  $\beta$ -smooth are the same, but with each  $L^{2}$  term replaced with  $\beta$ . We discuss additional related work in Appendix A.
Our contributions. We present a Markov chain sampling algorithm that generates samples from an  $L$ -log-Lipschitz or  $\beta$ -log-smooth log-concave distribution  $\pi \propto e^{-f}$  on an  $R$ -bounded polytope  $K$  given by  $m$  inequalities, with an error bounded in the TV distance (Algo-1 and Theorem 2.1). Our algorithm requires  $O((md + dL^2 R^2)\log (w / \delta))(T_f + md^{\omega -1})$  arithmetic operations to sample with TV error  $O(\delta)$  from  $\pi \propto e^{-f}$  when  $f$  is  $L$ -Lipschitz, and  $O((md + d\beta R^2)\log (w / \delta))(T_f + md^{\omega -1})$  arithmetic operations in the setting where  $f$  is  $\beta$ -smooth, where  $T_{f}$  is the number of arithmetic operations to compute the value of  $f$  and  $md^{\omega -1}$  is the number of arithmetic operations to compute the Hessian of the log-barrier of  $K$ .
In comparison to [33], we improve dependence of the running time on the parameters  $R / r$  and  $\log^{1 / \delta}$ . In comparison to [38], we improve the dependence on  $d$  while retaining the same dependence on  $1 / \delta$ . Our result directly implies faster runtimes with improved dependence on the dimension  $d$  for structured inference problems such as Bayesian Lasso logistic regression where e.g.  $R / r = \Omega (\sqrt{d})$  (Corollary B.1). Moreover, plugging our algorithm into the TV-toinfinity distance bound converter of [35], we obtain an algorithm to sample from a logconcave density constrained to a polytope with error bounded by infinity distance that improves upon prior work of [2]; see Corollary C.1 and the subsequent discussion. Corollary C.1, along with the exponential mechanism [36], allows us to obtain faster runtime bounds with improved
dependence on  $d$  for applications to differentially private empirical risk minimization [2] and matrix approximation [28]; see Corollary D.1 and the subsequent discussion.
Technically, our algorithm is a Markov chain inspired by the Dikin walk [24, 37], whose steps are determined by a barrier function that generalizes the log-barrier function by adding a "soft-threshold"  $\ell_2$ -norm regularizer. The regularized barrier allows our Markov chain to take larger steps, while still retaining a high acceptance probability on Lipschitz or smooth log-densities  $f$  -allowing our Markov chain to sample from these distributions with a faster runtime. A key technical step in obtaining our results is to show that our self-concordant barrier function is the limit of an infinite sequence of log-barrier functions for  $K$  "padded" with additional redundant inequalities. This allows us to leverage well-known properties of the log-barrier to bound the acceptance probability and mixing time of our Markov chain.
While  $\ell_2$  regularization is optimal for classes of functions  $f$  which are Lipschitz or smooth in the  $\ell_2$ -norm, for other classes of functions (e.g., Lipschitz in the  $\ell_1$ -norm),  $\ell_2$  regularization may not be optimal. Moreover, it remains open to obtain runtime bounds for the Dikin walk that do not require  $f$  to be Lipschitz or smooth, and/or depend polynomially on  $\log R$ . This leads to the question of whether one can design other tractable self-concordant barriers to obtain further runtime improvements for sampling log-concave distributions on a polytope; we discuss this in Appendix F.
2 Results
Our main result (Theorem 2.1 and Algorithm 1) is a Markov chain algorithm that generates TV-error bounded samples from  $L$ -log-Lipschitz or  $\beta$ -log-smooth log-concave distributions on a polytope. As explained later in this section, Theorem 2.1 often results in the fastest known algorithm for some of the applications to Bayesian inference and differentially private optimization mentioned in the introduction.
Notation. In the following,  $f$  is  $L$ -Lipschitz or  $\beta$ -smooth for some  $L, \beta > 0$ .  $T_{f}$  denotes the number of arithmetic operations to evaluate  $f$ ,  $T_{K}$  is the number of arithmetic operations to compute the Hessian of the log-barrier function,  $\hat{T}_{K}$  the operations for a membership oracle query, and  $\tilde{T}_{K}$  the operations to compute a projection oracle for  $K$ . When  $K$  is a polytope  $K = \{\theta \in \mathbb{R}^{d} : A\theta \leq b\}$  given by  $A \in \mathbb{R}^{m \times d}$  and  $b \in \mathbb{R}^{m}$ , one has  $T_{K} = O(md^{\omega - 1})$ ,  $\hat{T}_{K} = O(md)$ , and  $\tilde{T}_{K} = O(md^{\omega - 1})$ . For every  $j \in \{1, \dots, m\}$ , we denote the  $j$ 'th row of  $A$  by  $a_{j}$  and the  $j'$ th entry of  $b$  by  $b_{j}$ . When comparing runtimes, we often assume for simplicity that  $T_{f} = \Theta(d^{2})$  unless otherwise stated, which is the case, e.g., in logistic regression with  $n = \Theta(d)$  datapoints. For any two distributions  $\mu, \nu$  on  $\mathbb{R}^{d}$ , we denote their total variation distance by  $\| \mu - \nu \|_{\mathrm{TV}} := \sup_{S \subseteq \mathbb{R}^{d}} |\mu(S) - \nu(S)|$ . For  $\theta \in \mathbb{R}^{d}$ ,  $t > 0$ , denote the ball of radius  $t$  at  $\theta$  by  $B(\theta, t) := \{z \in \mathbb{R}^{d} : \| z - \theta \|_{2} \leq t\}$  where  $\| \cdot \|_{2}$  is the Euclidean norm. Denote the interior of any  $S \subseteq \mathbb{R}^{d}$  by  $\operatorname{Int}(S) := \{\theta \in S : B(\theta, t) \subseteq S \text{ for some } t > 0\}$ .  $\alpha > 0$  is a step-size hyperparameter shared by our algorithm and the original Dikin walk of [24, 37],  $\gamma > 0$  is the hyperparameter in [38], and  $\eta > 0$  a hyperparameter for our algorithm's regularizer.
Theorem 2.1 (Sampling with TV bounds via a soft-threshold Dikin Walk)
There exists an algorithm (Algorithm 1) which, given  $\delta, R > 0$  and either  $L > 0$  or  $\beta > 0$ ,  $A \in \mathbb{R}^{m \times d}$ ,  $b \in \mathbb{R}^m$  that define a polytope  $K := \{\theta \in \mathbb{R}^d : A\theta \leq b\}$  such that  $K$  is contained in a ball of radius  $R$  and has nonempty interior, an oracle for the value of a convex function  $f: K \to \mathbb{R}^d$ , where  $f$  is either  $L$ -Lipschitz or  $\beta$ -smooth, and an initial point sampled from a distribution supported on  $K$  which is  $w$ -warm with respect to  $\pi \propto e^{-f}$  for some  $w > 0$ , outputs a point from a distribution  $\mu$  where  $\| \mu - \pi \|_{\mathrm{TV}} \leq \delta$ . Moreover, this algorithm takes  $O((md + dL^2 R^2)\log (w / \delta)) \times (T_f + T_K)$  arithmetic operations in the setting where  $f$  is  $L$ -Lipschitz, or  $O((md + d\beta R^2)\log (w / \delta)) \times (T_f + T_K)$  arithmetic operations when  $f$  is  $\beta$ -smooth, where  $T_f$  is the number of operations to evaluate  $f$  and  $T_K = O(md^{\omega - 1})$ .
Theorem 2.1 improves on the previous bound of ([33]; Theorem 1.1) of  $O(d^{2}(R / r)^{2}\log^{2}(w d R / (r\delta))\log^{3}(w / \delta))(T_{f} + \hat{T}_{K})$  arithmetic operations, for sampling from  $\propto e^{-f}$
with TV error  $O(\delta)$  from a  $w$ -warm start, by a factor of  $\min(d^{2 - \omega}(R / r)^2, d^{3 - \omega} / (r^2 L^2)) \log^4(w / \delta)$  when  $f$  is  $L$ -Lipschitz on a polytope  $K$  defined by  $m = O(d)$  inequalities (see Table 1). Thus,  $\hat{T}_K = O(md)$  and if, e.g., we also have  $LR = O(\sqrt{d})$  and  $R / r = \sqrt{d}$ , the improvement is  $d^{3 - \omega}$  arithmetic operations. When  $f$  is  $\beta$ -smooth, the improvement is  $\min(d^{2 - \omega}(R / r)^2, d^{3 - \omega} / (r^2\beta)) \log^4(w / \delta)$  arithmetic operations.
When a warm start is not provided, Algorithm 1 takes at most  $O((md + dL^2 R^2) \times (d\log(R / r) + M + \log(1 / \delta))) (T_f + T_k)$  arithmetic operations when  $f$  is  $L$ -Lipschitz (or  $O((md + d\beta R^2) \times (d\log(R / r) + M + \log(1 / \delta))) (T_f + T_k)$ ) arithmetic operations when  $f$  is  $\beta$ -smooth), since an  $e^{d\log(R / r) + M}$ -warm start can be obtained by sampling uniformly from the ball  $B(a, r) \subseteq K$ , where  $M = \log(\max_{\theta \in K} e^{f(a) - f(\theta)}) \leq LR$ . In comparison, the work of [33] for the hit-and-run algorithm gives a bound of  $O(d^3 (R / r)^2) \log^5 (dR^2 / (\delta r))(T_f + \hat{T}_K)$  arithmetic operations to sample with TV error  $O(\delta)$  without a warm start. Thus, when we are not given a warm start and  $\pi \propto e^{-f}$  is constrained to a polytope  $K$  defined by  $m = O(d)$  inequalities, Theorem 2.1 improves on the bounds of [33] for the hit-and-run algorithm by a factor of  $\min(d^{2 - \omega}(R / r)^2, d^{3 - \omega} / (rL^2)) \log^4 (1 / \delta)$  arithmetic operations in the setting when  $f$  is  $L$ -Lipschitz, and  $\min(d^{2 - \omega}(R / r)^2, d^{3 - \omega} / (r\beta)) \log^4 (1 / \delta)$  when  $f$  is  $\beta$ -smooth. On the other hand, we note that the results of [33] apply more generally when  $\pi$  is a log-concave distribution on a convex body, while our bounds for the soft-threshold Dikin walk (Theorem 2.1) apply to the setting where  $\pi$  is a log-Lipschitz (or log-smooth) log-concave distribution on a polytope. In the example of Bayesian Lasso logistic regression, our algorithm takes  $O(d^{3 + \omega} \log(d / \delta))$  arithmetic operations without a warm start since  $f$  is both  $\beta$ -smooth and  $L$ -Lipschitz, with  $\beta = L = n = m = d$ , and  $R = 1$  and  $r = 1 / \sqrt{d}$  (Corollary B.1 in Appendix B); this improves by a factor of  $d^{3 - \omega} \log^4 (d / \delta)$  arithmetic operations on the bound of  $d^6 \log^5 (d / \delta)$  arithmetic operations for the hit-and-run algorithm of [33]. This is because, while it only takes  $\hat{T}_K = O(d)$  arithmetic operations to establish membership in the  $\ell_1$  ball, computing the function  $f$  in logistic regression requires  $T_f = nd = d^2$  arithmetic operations.
Algorithm	Iterations for L-Lipschitz f	Iterations for β-smooth f	Arithmetic operations per iteration	Iterations if m = O(d), L, r = O(1), R = O(d)	Iterations if m = O(d), r = O(1/d) β = L = O(d), R = O(1)
Proximal Langevin MC [3]	\(\tilde{O}(d^5\delta^{-6}L^2(\frac{R}{r})^4)\)	—	\(O(md^{\omega-1}) + T_{\nabla f}\)	\(\tilde{O}(d^9\delta^{-6})\)	\(\tilde{O}(d^9\delta^{-6})\)
Hit-and-run [32]	\(\tilde{O}(d^2(\frac{R}{r})^2)\)	same	\(O(md) + Tf\)	\(\tilde{O}(d^4)\)	\(\tilde{O}(d^3)\)
Dakin Walk of [37]	\(\tilde{O}(d^5 + d^3L^2R^2)\)	\(\tilde{O}(d^5 + d^3\beta R^2)\)	\(O(md^{\omega-1}) + Tf\)	\(\tilde{O}(d^5)\)	\(\tilde{O}(d^5)\)
Soft Threshold Dikin Walk (this paper)	\(\tilde{O}(md + dL^2R^2)\)	\(\tilde{O}(md + d\beta R^2)\)	\(O(md^{\omega-1}) + Tf\)	\(\tilde{O}(d^3)\)	\(\tilde{O}(d^2)\)
Table 1: Number of iterations (and arithmetic operations per-iteration) of different algorithms which imply bounds for sampling within TV error  $O(\delta)$  from a logconcave  $\pi \propto e^{-f}$  on a polytope  $K$  when  $f$  is L-Lipschitz or  $\beta$ -smooth, from a  $w$ -warm start.  $T_f$  and  $T_{\nabla f}$  are, respectively, the number of operations to compute  $f$  or  $\nabla f$ . The  $\tilde{O}$  notation hides logarithmic factors of  $d, \delta, r, W$ . The fifth column gives runtimes when  $K$  is a polytope with  $R = O(d)$  that contains the unit ball (and is thus not well-rounded), and  $f$  is  $O(1)$ -Lipschitz. The sixth column corresponds to sampling a Bayesian Lasso logistic regression posterior distribution with  $O(d)$  datapoints, where  $K$  is the unit  $\ell_1$ -ball.
We also compare to the bound of  $O((d^5 + d^3 L^2 R^2) \log^{(w / \delta)}) (T_f + T_k)$  arithmetic operations to sample from an  $L$ -log-Lipschitz density on a polytope defined by  $m$  inequalities with TV error  $O(\delta)$  from a  $w$ -warm start of [38].<sup>2</sup> We note that while this bound has a larger dependence on  $d$  (and  $L$ ) than the bounds for the hit-and-run algorithm, the dependence on  $\log^{1 / \delta}$  is much smaller—by a factor of  $\log^{4} 1 / \delta$ —which can allow for faster runtimes when the goal is to sample with infinity-distance error. We explain this in detail below. Theorem 2.1 improves on the bound of [38] bound by a factor of  $d^2$  arithmetic operations if  $m = O(d)$ ,
<sup>1</sup>Any convex body  $K$  contained in a ball of radius  $R > 0$  and containing a ball of smaller radius  $r > 0$  satisfies the lower bound  ${}^R / r \geq 1$ . Moreover, for any convex body  $K$ , there always exists a linear transformation  $T$  for which  $TK$  is contained in a ball of radius  $\hat{R}$  and contains a ball of radius  $\hat{r}$  such that  $\hat{R} / \hat{r} \leq O(\sqrt{d})$  ( $TK$  is referred to as a "well-rounded" convex body).
2When setting their scalar step size hyperparameter  $\alpha^{1 / 2}$  to  $O(\min (^{1 / d},^{1} / LR))$  , [38] get a bound of  $\phi \geq \alpha^{1 / 2} / (\kappa \sqrt{d})$  on the conductance  $\phi$  of their Dikin walk (their Lemma 4). Here  $\kappa$  is the selfconcordance parameter of the barrier function; for the log-barrier  $\kappa = m$  , although there are other barriers for which  $\kappa = O(d)$  . Plugging their conductance bound into Corollary 1.5 of [31] implies a bound of  $\phi^{-2}\log (w / \delta)(T_f + T_k) = O((d^5 +d^3 L^2 R^2)\log (w / \delta)(T_f + T_k))$  arithmetic operations from a  $w$  -warm start for their walk to sample with  $O(\delta)$  TV error from  $\pi$  . See Appendix A for details.
and  $d^3$  if we also have that  $LR = O(\sqrt{d})$ . When  $f$  is instead  $\beta$ -smooth their bound is  $O((d^5 + d^3 \beta R^2) \log^{(w / \delta)}(T_f + T_k))$ , and our improvement on this bound is  $d^2$  arithmetic operations if  $m = O(d)$ , and  $d^3$  if we also have that  $\beta R^2 = O(d)$ .
We also note that while many works, e.g. [24, 37, 27, 8], give faster bounds for the Dikin walk and its variants than the bounds in [38], these only apply in the special case when  $\pi$  is the uniform distribution on  $K$ . The proof of Theorem 2.1 appears in Appendix E. We give an overview of the main ideas in the proof of Theorem 2.1 in Section 3. In Appendix F, we give an axiomatic approach to arrive at our barrier function and discuss possible extensions.
Infinity-distance sampling. In applications of sampling to differentially private optimization [36, 23, 2, 18, 28], bounds in the total variation (TV) distance are insufficient to guarantee "pure"  $\varepsilon$ -differential privacy, and one instead requires bounds in the infinity-distance  $\mathrm{d}_{\infty}(\nu, \pi) := \sup_{\theta \in K} |\log^{(\nu(\theta) / \pi(\theta))}|$ ; see e.g. [14]. [35] give an algorithm that converts samples from TV bounds to those bounded in infinity-distance. Namely, given  $\varepsilon > 0$  and a sample from a distribution  $\mu$  within TV distance  $\delta \leq O\left(\varepsilon \left(R(d\log^{(R / r)} + LR)^2 / \varepsilon r\right)^{-d}e^{-LR}\right)$  of  $\pi$ , this post-processing algorithm outputs a sample within infinity-distance  $O(\varepsilon)$  from  $\pi$ . Plugging the TV bounds from our Theorem 2.1 into their Theorem 2.2 gives a faster algorithm to sample from a log-concave and log-Lipschitz (or log-smooth) distribution constrained to a polytope  $K$ , with  $O(\varepsilon)$  error in  $\mathrm{d}_{\infty}$  (Corollary C.1 in Appendix C). In particular, Corollary C.1 gives a bound of  $O((md + dL^2 R^2) \times [LR + d\log^{(Rd + LRd / r\varepsilon)}](T_f + T_K)$  arithmetic operations to sample within  $O(\varepsilon)$  error in  $\mathrm{d}_{\infty}$  from a log-concave distribution  $\propto e^{-f}$  constrained to a polytope  $K$  from a cold start when  $f$  is  $L$ -Lipschitz (when  $f$  is also  $\beta$ -smooth, the bound is  $O((md + d\beta R^2) \times [LR + d\log^{(Rd + LRd / r\varepsilon)}](T_f + T_K)$ ). This improves on the bound of  $O((m^2 d^3 + m^2 dL^2 R^2) \times [LR + d\log^{(Rd + LRd / r\varepsilon)}](T_f + T_K)$  arithmetic operations in Theorem 2.1 of [35] by a factor of  $d^3$ . Moreover, it further improves on the bound of  $O((md^9 + md^5 L^4 R^4)(1 / \varepsilon^2) \times \text{polylog}(1 / \varepsilon, 1 / r, R, L, d))(T_f + \tilde{T}_K)$  operations implied by [2]. Corollary C.1 improves on this bound by a factor of  $d^{9 - \omega} / \varepsilon^2$  when, e.g., each function evaluation takes  $T_f = O(d^2)$  operations and  $m = O(d)$  as may be the case in privacy applications.
Differentially private optimization. A randomized mechanism  $h: \mathcal{D}^n \to \mathcal{R}$  is  $\varepsilon$ -differentially private ( $\varepsilon$ -DP) if for any datasets  $x, x' \in \mathcal{D}$  which differ by a single datapoint, and any  $S \subseteq \mathcal{R}$ , we have  $\mathbb{P}(h(x) \in S) \leq e^{\varepsilon} \mathbb{P}(h(x') \in S)$ ; see [14].  $\varepsilon$ -differential privacy is the strongest notion of differential privacy, holds several advantages over weaker notions of differential privacy (see Remark 2.2), and has been widely studied in the literature [14, 25, 2].