SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Lemma E.11 (Isoperimetric inequality for cross-ratio distance (Theorem 2.2 of [32])) Let  $\pi : \mathbb{R}^d \to \mathbb{R}$  be a log-concave density, with support on a convex body  $K$ . Then for any partition of  $\mathbb{R}^d$  into measurable sets  $S_1, S_2, S_3$ , the induced measure  $\pi^\star$  satisfies
$$
\pi^ {\star} (S _ {3}) \geq \sigma (S _ {1}, S _ {2}) \pi^ {\star} (S _ {1}) \pi^ {\star} (S _ {2}).
$$
For any  $\theta \in \mathrm{Int}(K)$ , define the random variable  $Z_{\theta}$  to be the step taken by the Markov chain in Algorithm 1 from the point  $\theta$ . That is, set  $z = \theta + \Phi(\theta)^{-\frac{1}{2}}\xi$  where  $\xi \sim N(0, I_d)$ . If  $z \in K$ , set  $Z_{\theta} = z$  with probability  $\min \left(\frac{\pi(z)\sqrt{\operatorname*{det}(\Phi(z))}}{\pi(\theta)\sqrt{\operatorname*{det}(\Phi(\theta))}}\exp\left(\|z-\theta\|_{\Phi(\theta)}^2 -\|\theta-z\|_{\Phi(z)}^2\right),1\right)$ . Else, set  $z = \theta$ .
For any  $\theta$ ,  $S \subseteq \mathbb{R}^d$ , define the one-step transition probability of the soft-threshold Dikin walk Markov chain to be
$$
P _ {\theta} (S) = \mathbb {P} (Z _ {\theta} \in S).
$$
The next proposition shows that the soft-threshold Dikin walk Markov chain is reversible and  $\pi$  is a stationary distribution of this Markov chain:
Proposition E.12 (Reversibility and stationary distribution) For any  $S_1, S_2 \subseteq K$  we have that
$$
\int_ {\theta \in S _ {1}} \pi (\theta) P _ {\theta} (S _ {2}) \mathrm {d} \theta = \int_ {z \in S _ {2}} \pi (z) P _ {z} (S _ {1}) \mathrm {d} z.
$$
Proof: Let  $\rho_{\theta}(z) := \frac{\sqrt{\operatorname*{det}(\Phi(\theta))}}{(2\mathrm{pi})^{\frac{d}{2}}} e^{-\frac{1}{2} (\theta - z)^{\top} \Phi(\theta)(\theta - z)}$  for any  $\theta, z \in \operatorname{Int}(K)$  be the density of the  $N(\theta, \Phi(\theta)^{-1})$  distribution.
$$
\begin{array}{l} \int_ {\theta \in S _ {1}} \pi (\theta) P _ {\theta} (S _ {2}) \mathrm {d} \theta = \int_ {\theta \in S _ {1}} \pi (\theta) \int_ {z \in S _ {2}} \rho_ {\theta} (z) \times \\ \min  \left(\frac {\pi (z) \sqrt {\det  (\Phi (z))}}{\pi (\theta) \sqrt {\det  (\Phi (\theta))}} \exp \left(\| z - \theta \| _ {\Phi (\theta)} ^ {2} - \| \theta - z \| _ {\Phi (z)} ^ {2}\right), 1\right) \mathrm {d} z \mathrm {d} \theta \\ = \int_ {\theta \in S _ {1}} \int_ {z \in S _ {2}} \pi (\theta) \rho_ {\theta} (z) \min  \left(\frac {\pi (z) \rho_ {z} (\theta)}{\pi (\theta) \rho_ {\theta} (z)}, 1\right) \mathrm {d} z \mathrm {d} \theta \\ = \int_ {z \in S _ {2}} \int_ {\theta \in S _ {1}} \pi (\theta) \rho_ {z} (\theta) \min  \left(\frac {\pi (\theta) \rho_ {\theta} (z)}{\pi (z) \rho_ {z} (\theta)}, 1\right) d \theta d z \\ = \int_ {\theta \in S _ {1}} \pi (\theta) P _ {\theta} (S _ {2}) \mathrm {d} \theta . \\ \end{array}
$$
Define the conductance  $\phi$  of the Markov chain to be
$$
\phi = \inf  _ {S \subseteq K: \pi^ {\star} (S) \leq \frac {1}{2}} \frac {1}{\pi^ {\star} (S)} \int_ {S} P _ {\theta} (K \backslash S) \pi (\theta) \mathrm {d} \theta .
$$
Lemma E.13 The conductance  $\phi$  satisfies
$$
\phi \geq \frac {1}{1 0 ^ {4}} \frac {1}{\sqrt {2 m \alpha^ {- 1} + \eta^ {- 1} R ^ {2}}}.
$$
Proof: The proof follows the general format for conductance proofs for geometric Markov chains. Let  $S_{1} \subseteq K$  and let  $S_{2} = K \backslash S_{1}$ . Without loss of generality, we may assume that  $\pi(S_{1}) \leq \frac{1}{2}$  (since otherwise we could just swap the names "  $S_{1}$ " and "  $S_{2}$  " ). Let
$$
\begin{array}{l} S _ {1} ^ {\prime} = \left\{\theta \in S _ {1}: P _ {\theta} (S _ {2}) \leq \frac {1}{7 0} \right\}, \\ S _ {2} ^ {\prime} = \left\{z \in S _ {2}: P _ {z} \left(S _ {1}\right) \leq \frac {1}{7 0} \right\}, \tag {51} \\ \end{array}
$$
$$
S _ {3} ^ {\prime} = (K \backslash S _ {1} ^ {\prime}) \backslash S _ {2} ^ {\prime}.
$$
By Proposition E.12 we have that
$$
\int_ {\theta \in S _ {1}} \pi (\theta) P _ {\theta} (S _ {2}) \mathrm {d} \theta = \int_ {\theta \in S _ {2}} \pi (\theta) P _ {\theta} (S _ {1}) \mathrm {d} \theta . \tag {52}
$$
Moreover, by Lemma E.10, for any  $u,v\in \mathrm{Int}(K)$  we have that
$$
\left\| P _ {u} - P _ {v} \right\| _ {\mathrm {T V}} \leq \frac {2 9}{3 0} \text {w h e n e v e r} \| u - v \| _ {\Phi (u)} \leq \frac {1}{2}. \tag {53}
$$
Thus, on the one hand, by Lemma E.2, Inequality (53) implies that for any  $u, v \in \operatorname{Int}(K)$ ,
$$
\left\| P _ {u} - P _ {v} \right\| _ {\mathrm {T V}} \leq \frac {2 9}{3 0}, \quad \text {w h e n e v e r} \quad \sigma (u, v) \leq \frac {1}{2 \sqrt {2 m \alpha^ {- 1} + \eta^ {- 1} R ^ {2}}}. \tag {54}
$$
On the other hand, Equations (51) imply that, for any  $u \in S_1'$ ,  $v \in S_2'$  we have that
$$
\left\| P _ {u} - P _ {v} \right\| _ {\mathrm {T V}} \geq 1 - P _ {u} \left(S _ {2}\right) - P _ {v} \left(S _ {1}\right) \geq \frac {6 8}{7 0}. \tag {55}
$$
Thus, Inequalities (54) and (55) together imply that
$$
\sigma \left(S _ {1} ^ {\prime}, S _ {2} ^ {\prime}\right) > \frac {1}{2 \sqrt {2 m \alpha^ {- 1} + \eta^ {- 1} R ^ {2}}}. \tag {56}
$$
Moreover by Lemma E.11 we have that
$$
\pi^ {\star} \left(S _ {3} ^ {\prime}\right) \geq \sigma \left(S _ {1} ^ {\prime}, S _ {2} ^ {\prime}\right) \pi^ {\star} \left(S _ {1} ^ {\prime}\right) \pi^ {\star} \left(S _ {2} ^ {\prime}\right). \tag {57}
$$
First, we assume that both  $\pi^{\star}(S_1^{\prime})\geq \frac{1}{4}\pi^{\star}(S_1)$  and  $\pi^{\star}(S_2^{\prime})\geq \frac{1}{4}\pi^{\star}(S_2)$ . In this case we have
$$
\begin{array}{l} \int_ {S _ {1}} P _ {\theta} (S _ {2}) \pi (\theta) \mathrm {d} \theta \stackrel {\text {E q .}} {=} 5 2 \frac {1}{2} \int_ {S _ {1}} P _ {\theta} (S _ {2}) \pi (\theta) \mathrm {d} \theta + \frac {1}{2} \int_ {S _ {2}} P _ {\theta} (S _ {1}) \pi (\theta) \mathrm {d} \theta \\ \geq \frac {1}{1 4 0} \pi^ {\star} \left(S _ {3} ^ {\prime}\right) \\ \geq \frac {1}{1 4 0} \sigma \left(S _ {1} ^ {\prime}, S _ {2} ^ {\prime}\right) \pi^ {\star} \left(S _ {1} ^ {\prime}\right) \pi^ {\star} \left(S _ {2}\right) \\ \stackrel {\text {E q .} 5 6} {\geq} \frac {1}{2 8 0} \frac {1}{\sqrt {2 m \alpha^ {- 1} + \eta^ {- 1} R ^ {2}}} \pi^ {\star} \left(S _ {1} ^ {\prime}\right) \pi^ {\star} \left(S _ {2} ^ {\prime}\right) \\ \geq \frac {1}{5 6 0} \frac {1}{\sqrt {2 m \alpha^ {- 1} + \eta^ {- 1} R ^ {2}}} \min  \left(\pi^ {\star} \left(S _ {1} ^ {\prime}\right), \pi^ {\star} \left(S _ {2} ^ {\prime}\right)\right) \\ \geq \frac {1}{2 0 2 4} \frac {1}{\sqrt {2 m \alpha^ {- 1} + \eta^ {- 1} R ^ {2}}} \min  \left(\pi^ {\star} \left(S _ {1}\right), \pi^ {\star} \left(S _ {2}\right)\right). \tag {58} \\ \end{array}
$$
Now suppose that instead either  $\pi^{\star}(S_1') < \frac{1}{4}\pi^{\star}(S_1)$  or  $\pi^{\star}(S_2') < \frac{1}{4}\pi^{\star}(S_2)$ . If  $\pi^{\star}(S_1') < \frac{1}{4}\pi^{\star}(S_1)$  then we have
$$
\begin{array}{l} \int_ {S _ {1}} P _ {\theta} (S _ {2}) \pi (\theta) \mathrm {d} \theta \stackrel {\text {E q .} 5 2} {=} \frac {1}{2} \int_ {S _ {1}} P _ {\theta} (S _ {2}) \pi (\theta) \mathrm {d} \theta + \frac {1}{2} \int_ {S _ {2}} P _ {\theta} (S _ {1}) \pi (\theta) \mathrm {d} \theta \\ \geq \frac {1}{2} \int_ {S _ {1} \backslash S _ {1} ^ {\prime}} P _ {\theta} (S _ {2}) \pi (\theta) \mathrm {d} \theta \\ \geq \frac {1}{2} \times \frac {3}{4} \times \frac {3 4}{7 0} \pi^ {\star} (S _ {1}) \\ \geq \frac {1}{1 0} \min  \left(\pi^ {\star} \left(S _ {1}\right), \pi^ {\star} \left(S _ {2}\right)\right). \tag {59} \\ \end{array}
$$
Similarly, if  $\pi^{\star}(S_2^{\prime}) < \frac{1}{4}\pi^{\star}(S_2)$  we have
$$
\begin{array}{l} \int_ {S _ {1}} P _ {\theta} (S _ {2}) \pi (\theta) \mathrm {d} \theta \stackrel {\text {E q .} 5 2} {=} \frac {1}{2} \int_ {S _ {1}} P _ {\theta} (S _ {2}) \pi (\theta) \mathrm {d} \theta + \frac {1}{2} \int_ {S _ {2}} P _ {\theta} (S _ {1}) \pi (\theta) \mathrm {d} \theta \\ \geq \frac {1}{2} \int_ {S _ {2} \backslash S _ {2} ^ {\prime}} P _ {\theta} (S _ {1}) \pi (\theta) \mathrm {d} \theta \\ \geq \frac {1}{2} \times \frac {3}{4} \times \frac {3 4}{7 0} \pi^ {\star} (S _ {2}) \\ \geq \frac {1}{1 0} \min  \left(\pi^ {\star} \left(S _ {1}\right), \pi^ {\star} \left(S _ {2}\right)\right). \tag {60} \\ \end{array}
$$
Therefore, Inequalities (58), (59), and (60) together imply that
$$
\frac {1}{\min \left(\pi^ {\star} \left(S _ {1}\right) , \pi^ {\star} \left(S _ {2}\right)\right)} \int_ {S _ {1}} P _ {\theta} \left(S _ {2}\right) \pi (\theta) \mathrm {d} \theta \geq \frac {1}{1 0 ^ {4}} \frac {1}{\sqrt {2 m \alpha^ {- 1} + \eta^ {- 1} R ^ {2}}}. \tag {61}
$$
for every partition  $S_{1} \cup S_{2} = K$ . Hence, Inequality (61) implies that
$$
\phi = \inf _ {S \subseteq K: \pi^ {\star} (S) \leq \frac {1}{2}} \frac {1}{\pi^ {\star} (S)} \int_ {S} P _ {\theta} (K \backslash S) \pi (\theta) \mathrm {d} \theta \geq \frac {1}{6 0} \frac {1}{\sqrt {2 m \alpha^ {- 1} + \eta^ {- 1} R ^ {2}}}.
$$
Definition E.3 We say that a distribution  $\nu$  is  $w$ -warm for some  $w \geq 1$  with respect to the stationary distribution  $\pi$  if  $\sup_{z \in K} \frac{\nu(z)}{\pi(z)} \leq w$ .
Lemma E.14 (Corollary 1.5 of [31]) Suppose that  $\mu_0$  is the initial distribution of a lazy reversible Markov chain with conductance  $\phi > 0$  and stationary distribution  $\pi$ , and let  $\mu_i$  be the distribution of this Markov chain after  $i \geq 0$  steps. Suppose that  $\mu_0$  is  $w$ -warm with respect to  $\pi$  for some  $w \geq 1$ . Then for all  $i \geq 0$  we have
$$
\| \mu_ {i} - \pi \| _ {\mathrm {T V}} \leq \sqrt {w} \left(1 - \frac {\phi^ {2}}{2}\right) ^ {i}.
$$
Lemma E.15 Let  $\delta > 0$ . Suppose that  $f: K \to \mathbb{R}$  is either  $L$ -Lipschitz (or has  $\beta$ -Lipschitz gradient). Suppose that  $\theta_0 \sim \mu_0$  where  $\mu_0$  is a wwarm distribution with respect to  $\pi \propto e^{-f}$  with support on  $K$ . Let  $\mu$  denote the distribution of the output of Algorithm 1 with hyperparameters  $\alpha \leq \frac{1}{10^5d}$  and  $\eta \leq \frac{1}{10^4dL^2}$  (or  $\eta \leq \frac{1}{10^4d\beta}$ ), if it is run for  $T$  iterations. Then if  $T \geq 10^9$  ( $2m\alpha^{-1} + \eta^{-1}R^2$ )  $\times \log \left(\frac{w}{\delta}\right)$  we have that
$$
\| \mu - \pi \| _ {\mathrm {T V}} \leq \delta .
$$
Proof: By Lemma E.13 we have that the conductance  $\phi$  of the Markov chain in Algorithm 1 satisfies
$$
\phi \geq \frac {1}{1 0 ^ {4}} \frac {1}{\sqrt {2 m \alpha^ {- 1} + \eta^ {- 1} R ^ {2}}},
$$
and hence that
$$
T = 1 0 ^ {9} \left(2 m \alpha^ {- 1} + \eta^ {- 1} R ^ {2}\right) \times \log (\frac {w}{\delta}) \geq 2 \phi^ {- 2} \times \log (\frac {w}{\delta}).
$$
Thus, by Lemma E.13 we have that
$$
\begin{array}{l} \left\| \mu_ {T} - \pi \right\| _ {\mathrm {T V}} \leq \sqrt {w} \left(1 - \frac {\phi^ {2}}{2}\right) ^ {T} \\ = \sqrt {w} \left(1 - \frac {\phi^ {2}}{2}\right) ^ {2 \phi^ {- 2} \log (\frac {w}{\delta})} \\ \leq \sqrt {w} e ^ {- \log (\frac {w}{\delta})} \\ \leq \delta . \\ \end{array}
$$
Proof: [of Theorem 2.1]
Total variation bound. Recall that we have set the step size parameters  $\alpha = \frac{1}{10^5d}$  and either  $\eta = \frac{1}{10^4dL^2}$  (if  $f$  is  $L$ -Lipschitz) or  $\eta = \frac{1}{10^4d\beta}$  (if  $f$  is  $\beta$ -smooth). Thus, after running Algorithm 1 for  $T = 10^{9}\left(2m\alpha^{-1} + \eta^{-1}R^{2}\right)\times \log (\frac{w}{\delta})$  iterations, by Lemma E.15 we have that the distribution  $\mu$  of the output of Algorithm 1 satisfies
$$
\left\| \mu - \pi \right\| _ {\mathrm {T V}} \leq \delta . \tag {62}
$$
Bounding the number of operations. Moreover, by Lemma E.1 we have that each iteration of Algorithm 1 can be implemented in  $O(md^{\omega - 1})$  arithmetic operations plus  $O(1)$  calls to the oracle for the value of  $f$ . Thus, the number of iterations  $T$  taken by Algorithm 1 is  $O((md + dL^2 R^2) \times \log(\frac{w}{\delta}))$  iterations in the setting where  $f$  is  $L$ -Lipschitz and  $O((md + d\beta R^2) \times \log(\frac{w}{\delta}))$  iterations in the setting where  $f$  is  $\beta$ -smooth, where each iteration takes one function evaluation and  $md^{\omega - 1}$  arithmetic operations.
F Extension to general barrier functions?
From any point  $\theta$ , our algorithm proposes a step with Gaussian distribution  $N(\theta, \frac{1}{d} (\nabla^2 g(\theta))^{-1})$ , where  $g$  is the following barrier function
$$
g (\theta) = \varphi (\theta) + \hat {\eta} ^ {- 1} \theta^ {\top} \theta , \tag {63}
$$
where  $\varphi(\theta)$  is a barrier function for the polytope  $K$ , and the parameter  $\hat{\eta}^{-1} = \Omega\left(\frac{1}{L^2}\right)$  in the setting where  $f$  is guaranteed to be  $L$ -Lipschitz and  $\hat{\eta}^{-1} = \Omega\left(\frac{1}{\beta}\right)$  in the setting where  $f$  is guaranteed to be  $\beta$ -smooth. Thus, most of the probability mass of the Gaussian distribution concentrates in the Dikin ellipsoid  $\hat{D}_{\theta} = \theta + \{w : w^{\top}(\nabla^{2}g(\theta))^{-1}w \leq 1\}$  for the barrier function  $g$ . For simplicity, in our main result, we assume that  $\varphi(\theta) := -\sum_{j=1}^{m} \log (b_j - a_j^\top \theta)$  (which has self-concordance parameter  $m$ ), however, we can in principle choose any barrier function  $\varphi$  for the polytope  $K$ , such as the entropic barrier [4] or the Lee-Sidford Barrier [29] which have self-concordance parameter roughly  $\nu = d$ .
To arrive at our barrier function from a more axiomatic approach, we first consider the definition of  $\nu$ -self concordant barrier function:
Definition F.1 ( $\nu$ -self-concordant barrier function for  $K$ ) We say that  $g$  is a  $\nu$ -self-concordant barrier function if  $g: \operatorname{Int}(K) \to \mathbb{R}$  and  $g$  satisfies the following conditions:
1. Convex and differentiable barrier:  $g$  is convex and third-order differentiable, and  $g(x) \to +\infty$  as  $x \to \partial K$ .
2. Self-concordance:  $\nabla^3 g(x)[h,h,h] \leq 2(\nabla^2 g(x)[h,h])^{3/2}$  for all  $h \in \mathbb{R}^d$  (this ensures that the Hessian of the barrier function does not change too much each time the Dikin walk takes a step from  $x$  to  $z$ , that is,  $\frac{1}{2}\nabla^2 g(z) \preceq \nabla^2 g(x) \preceq 2\nabla^2 g(z)$ )
3.  $g$  is  $\nu$ -self concordant:  $h^\top \nabla g(x) \leq \sqrt{\nu h^\top \nabla^2 g(x)h}$  for every  $x \in \operatorname{Int}(K)$ ,  $h \in \mathbb{R}^d$
The fact that our barrier function (63) satisfies parts (1) and (2) of Definition F.1 follows from the fact that  $\varphi$  satisfies Definition F.1 and that  $\nabla^3 (\theta^\top \theta) = 0$ . We discuss the self-concordance parameter  $\nu$  for which our barrier function satisfies (63) below.
To ensure that the steps  $z \sim N(\theta, (\frac{1}{d}\nabla^2 g(\theta))^{-1})$  proposed by our Dikin walk Markov chain arising from the barrier function  $g$  has an  $\Omega(1)$  acceptance ratio  $e^{f(z) - f(\theta)}$ , we require that the function  $g$  satisfies the following property. This property says that at least  $\frac{1}{4}$  of the volume of the Dikin ellipsoid  $\hat{D}_{\theta}$  is contained in the sublevel set  $\{z \in K : f(\theta) < f(\theta) + 2\}$  where the value of  $f$  does not increase by more than 2.
Property F.1 Dikin ellipsoid mostly contained in sublevel set: At every  $\theta \in \operatorname{Int}(K)$ , the Dikin Ellipsoid, satisfies
$$
\operatorname {V o l} \left(\hat {D} _ {\theta} \cap \{z \in K: f (z) <   f (\theta) + 2 \}\right) \geq \frac {1}{4} \operatorname {V o l} (\hat {D} _ {\theta}).
$$
When designing a barrier function  $g$ , there is a trade-off between choosing  $g$  such that the self-concordance parameter  $\nu$  is small, while at the same time ensuring that Property F.1 holds. Roughly, to make the parameter  $\nu$  as small as possible, we would like the Dikin ellipsoid to be large relative to the Hilbert-distance metric for the convex body  $K$  (This is because, by Proposition 2.3.2(iii) of [39], any  $\nu$ -self concordant function  $g$  satisfies  $(h^{\top}\nabla^{2}g(x)h)^{-\frac{1}{2}}\leq |h|_{x}\leq (1 + 3\nu)(h^{\top}\nabla^{2}g(x)h)^{-\frac{1}{2}}$ , for any  $h\in \mathbb{R}^d$  where  $|h|_x\coloneqq \sup \{\alpha >0: x\pm \alpha h\in \{z\in K\} \}$ . On the other hand, if we make the Dikin ellipsoid too large (with respect to the sublevel set  $\{z\in K:f(z) < f(x) + 2\}$ ) then Property F.1 will not be satisfied, and the steps proposed by the Dikin walk will have a very low acceptance probability. Setting the hyperparameter  $\hat{\eta} = \frac{1}{L^2}$  when  $f$  is  $L$ -Lipschitz or  $\hat{\eta} = \frac{1}{\beta}$  when  $f$  is  $\beta$  smooth ensures that our barrier function in (63) satisfies Property F.1.
In the special case where  $f$  is the log-density of the uniform distribution  $(f \equiv 0)$ , or when  $f$  is any linear function, we have that  $f$  is  $\beta$ -smooth for  $\beta = 0$ . Thus, our barrier function (63) we use to encode the geometry of  $f$  on  $K$  is the same as the barrier function  $\varphi$  for the polytope  $K$ . This is because, since the level sets of linear functions  $f$  are half-planes, any ellipsoid centered at  $\theta$  has at least half of its volume in the sublevel set  $\{z \in K : f(z) \leq f(\theta)\} \subseteq \{z \in K : f(z) < f(\theta) + 2\}$ , satisfying Property F.1.
The following lemma shows that our barrier function in (63) is  $\nu$ -self concordant with  $\nu = O(\nu' + \hat{\eta}R^2)$ , where  $\nu'$  is the self-concordance parameters of  $\varphi$  (which is  $\nu' = m$  if we choose  $\varphi$  to be log-barrier function). Thus, our barrier function in (63) is  $\nu = O(\nu' + L^2R^2)$  self-concordant in the setting where  $f$  is  $L$ -Lipschitz, and  $\nu = O(\nu' + \beta R^2)$  in the setting where  $f$  is  $\beta$ -smooth:
Lemma F.2 Suppose that  $\phi(x)$  is a  $\nu'$ -self concordant barrier function for a convex body  $K \subset \mathbb{R}^d$  where  $B(0, r) \subseteq K \subseteq B(0, R)$  for some  $R > r > 0$ . Let  $g(x) = \phi(x) + \frac{\alpha}{2} x^\top x$  for some  $\alpha > 0$ . Then  $g$  is  $\nu$ -self concordant for  $\nu = 4\nu' + 4\alpha R^2$ .
The proof of Lemma F.2 is given in Appendix F.1. The polynomial dependence of  $\nu$  on  $LR$  or  $\beta R^2$  is a necessary feature of any objective function satisfying Definition F.1 and Property F.1. In appendix F.2, we give explicit examples of classes of objective functions  $f$  and polytopes  $K$  for which the minimum value of  $\nu$  depends polynomially on  $LR$  (and classes of smooth functions  $f$  where  $\nu$  depends polynomially on  $\sqrt{\beta} R$ ).
An open problem is whether one can design versions of the Dikin walk which sample from log-concave distributions with a runtime that depends only on the dimension  $d$ , and is independent of  $L$ ,  $R$  or  $\beta$  and which are invariant to affine transformations. The difficulty in achieving this lies in the fact that (e.g., in the setting where  $f$  is  $L$ -Lipschitz), on the one hand, the level sets of  $f$  where most of the probability mass of  $\propto e^{-f}$  concentrates may have a diameter roughly  $RL$  times smaller than the diameter  $R$  of  $K$ . Thus, to have an  $\Omega(1)$  acceptance probability the Dikin walk may need to take steps that are roughly  $RL$  times smaller than the diameter of  $K$ . On the other hand, the isoperimetric inequality currently used to bound the mixing time of the Dikin walk uses a metric—the cross-ratio distance for  $K$ -which, roughly speaking, defines distances between steps by how quickly these steps approach the boundary of  $K$ . Thus, measured in the cross-ratio distance, the steps proposed by the Dikin walk are of size roughly proportional to  $\frac{1}{RL}$ , and mixing time bounds obtained with this isoperimetric inequality thus depend polynomially on  $RL$ . To obtain mixing time bounds independent of the  $R, L, \beta$  one may need to show a new isoperimetric inequality which is based on a different metric that encodes the geometry of all of the level sets of  $f$ -rather than just the geometry of its support  $K$ .
F.1 Proof of Lemma F.2
Proof: [Proof of Lemma F.2] For any  $x \in \operatorname{int}(K)$ , we have
$$
\nabla g (x) = \nabla \phi (x) + \alpha x
$$
Thus, for any  $h\in \mathbb{R}^d$
$$
h ^ {\top} \nabla g (x) = h ^ {\top} \nabla \phi (x) + \alpha h ^ {\top} x \leq h ^ {\top} \nabla \phi (x) + \alpha \| h \| \| x \| \leq h ^ {\top} \nabla \phi (x) + \alpha R \| h \| \tag {64}
$$
$$
\begin{array}{l} \left(h ^ {\top} \nabla g (x)\right) ^ {2} \stackrel {\text {E q .} (6 4)} {\leq} \left(h ^ {\top} \nabla \phi (x) + \alpha R \| h \|\right) ^ {2} \tag {65} \\ \leq 4 \left(h ^ {\top} \nabla \phi (x)\right) ^ {2} + 4 \left(\alpha R \| h \|\right) ^ {2} \\ = 4 \left(h ^ {\top} \nabla \phi (x)\right) ^ {2} + 4 \alpha^ {2} R ^ {2} h ^ {\top} I _ {d} h \\ = 4 \left(h ^ {\top} \nabla \phi (x)\right) ^ {2} + 4 \alpha R ^ {2} h ^ {\top} (\alpha I _ {d}) h \\ \leq 4 \nu^ {\prime} h ^ {\top} \nabla^ {2} \phi (x) h + 4 \alpha R ^ {2} h ^ {\top} (\alpha I _ {d}) h \\ \leq \left(4 \nu^ {\prime} + 4 \alpha R ^ {2}\right) \left(h ^ {\top} \nabla^ {2} \phi (x) h + h ^ {\top} \left(\alpha I _ {d}\right) h\right) \\ = \left(4 \nu^ {\prime} + 4 \alpha R ^ {2}\right) \left(h ^ {\top} \left(\nabla^ {2} \phi (x) + \alpha I _ {d}\right) h\right) \\ = \left(4 \nu^ {\prime} + 4 \alpha R ^ {2}\right) \left(h ^ {\top} \left(\nabla^ {2} g (x)\right) h\right) \\ \end{array}
$$
where the third inequality holds since  $\phi$  is  $\nu'$  self-concordant. Thus, plugging in  $\nu = 4\nu' + 4\alpha R^2$  to (65), we get that
$$
h ^ {\top} \nabla g (x) \leq \sqrt {\nu \left(h ^ {\top} \left(\nabla^ {2} g (x)\right) h\right)}.
$$
F.2 Lower bounds for self-concordance parameter
Lower bounds on  $\nu$  for worst-case  $L$ -Lipschitz  $f$ . Consider the  $L$ -Lipschitz function  $f(\theta) = L\| \theta \| _2$  constrained to the convex body  $K = \frac{R}{2\sqrt{d}} [-1,1]^d$  which is contained in a ball of radius  $R$ . In this case, any barrier function  $g$  which satisfies Property F.1 has Dikin Ellipsoid  $\hat{D}_{\theta}$  which is contained in the ball  $B(0,\frac{8}{L})$ . Thus, we have that  $\nu \geq \Omega (RL)$ . (This is because, by Proposition 2.3.2(iii) of [39], any  $\nu$ -self concordant function  $g$  satisfies  $(h^{\top}\nabla^{2}g(\theta)h)^{-\frac{1}{2}}\leq |h|_{\theta}\leq (1 + 3\nu)(h^{\top}\nabla^{2}g(\theta)h)^{-\frac{1}{2}}$ , for any  $h\in \mathbb{R}^d$  where  $|h|_{\theta}:= \sup \{\alpha >0:\theta \pm \alpha h\in \{z\in K\}$ . Thus, at  $\theta = 0$  and choosing  $h = (1,\dots ,1)$  we have  $\frac{|h|_{\theta}}{\|h\|_2} = R$  and  $(h^{\top}\nabla^{2}g(\theta)h)^{-\frac{1}{2}}\leq O(\frac{1}{L})$ . Thus,  $\nu \geq \Omega \left(\frac{|h|_{\theta}}{(h^{\top}\nabla^{2}g(\theta)h)^{-\frac{1}{2}}}\right)\geq LR$ .
Since there exists a convex body  $K$  for which any self-concordant barrier function satisfying Definition F.1 has self-concordance parameter at least  $\nu \geq d$ , for any  $L, R > 0$  there exists a function  $f$  and convex body  $K \subset B(0, R)$  such that the self-concordance parameter of every barrier function satisfying both Definition F.1 and Property F.1 is at least  $\nu \geq \Omega(\max(d, LR))$ .
Lower bounds on  $\nu$  for worst-case  $\beta$ -smooth  $f$ . Consider the  $\beta$ -smooth function  $f(\theta) = \frac{1}{2}\beta\theta^{\top}\theta$  constrained to the convex body  $K = \frac{R}{2\sqrt{d}}[-1,1]^d$ . At  $\theta = 0$ , any ellipsoid  $\hat{D}_{\theta}$  satisfying Property F.1 is contained in the ball  $\frac{8}{\sqrt{\beta}}B(0,1)$ . Thus, we have that  $\nu \geq \frac{R}{\sqrt{\beta}}\sqrt{\beta} = \Omega(\sqrt{\beta}R)$  (This is because at  $\theta = 0$  we have  $\frac{|h|_{\theta}}{\|h\|_2} = R$  and  $(h^{\top}\nabla^{2}g(\theta)h)^{-\frac{1}{2}} \leq O\left(\frac{1}{\sqrt{\beta}}\right)$  and thus, by Proposition 2.3.2(iii) of [39], we have  $\nu \geq \Omega\left(\frac{|h|_{\theta}}{(h^{\top}\nabla^{2}g(\theta)h)^{-\frac{1}{2}}}\right) \geq \sqrt{\beta}R$ ). Thus, for any  $\beta, R > 0$  there exists a function  $f$  and convex body  $K \subset B(0,R)$  such that the self-concordance parameter of every barrier function satisfying both Definition F.1 and Property F.1 is at least  $\nu \geq \max(d,\Omega(\beta R))$ .