SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Figure 12: Set of valid diagonal preconditioners (step-sizes  $\mathbf{p}_1$  and  $\mathbf{p}_2$ ) for the quadratic in Equation (11). Preconditioned gradient descent can use a larger step-size in the first coordinate.
B Optimal preconditioners, valid preconditioners and competitive ratios
In Section 2, we defined the optimal preconditioner  $\mathbf{P}_{*}$  as the preconditioner that is the best overall approximation to the inverse Hessian. Formally, we define the optimal diagonal preconditioner  $\mathbf{P}_{*}$  as
$$
\mathbf {P} _ {*} := \underset {\mathbf {P} \succ 0, \text {d i a g o n a l}} {\arg \min } \kappa \quad \text {s u c h t h a t} \quad \frac {1}{\kappa} \mathbf {P} ^ {- 1} \preceq \nabla^ {2} f (\mathbf {x}) \preceq \mathbf {P} ^ {- 1} \text {f o r a l l} \mathbf {x}. \tag {1}
$$
One way to interpret this definition is that  $\mathbf{P}_*^{-1}$  is the tightest diagonal approximation to  $\nabla^2 f(\mathbf{x})$ .
We remark that we do not need  $f$  to be (strongly-)convex to define the theoretically optimal step-size of  $1 / L$  for gradient descent. Thus, one may wonder why we need strong-convexity (although we relax this to requiring  $f$  to be PL in Appendix B.1) to define what an optimal preconditioner is in (1).
The main difference between the scalar step-size and per-coordinate step-sizes settings is whether the "largest" step-size or preconditioner is well-defined. In the scalar setting, the largest step-size that is guaranteed to lead to progress everywhere (i.e., a step-size that satisfies the Armijo condition (3) for all  $\mathbf{x}$ ) is well-defined and equal to  $\alpha_{*} \coloneqq 1 / L$  for  $L$ -smooth function  $f$ . Equivalently,
$$
\alpha_ {*} = \sup  \left\{\alpha > 0: \nabla^ {2} f (\mathbf {x}) \preceq \frac {1}{\alpha} \mathbf {I} \right\} = \sup  _ {\mathbf {x} \in \mathbb {R} ^ {d}} \lambda_ {\max } (\nabla^ {2} f (\mathbf {x})),
$$
where  $\lambda_{\max}(\nabla^2 f(\mathbf{x}))$  is the largest eigenvalue of  $\nabla^2 f(\mathbf{x})$ . But in the case of preconditioners, the ordering on positive definite matrices is not complete, so there is no single "largest" preconditioner  $\mathbf{P}$  that satisfies  $\nabla^2 f(\mathbf{x}) \preceq \mathbf{P}^{-1}$ . We can still describe "good" preconditioners, that are guaranteed to satisfy the Armijo condition (Equation (4)) everywhere; this is the notion of valid preconditioners defined in Definition 4.1, which in set notation is  $\mathcal{V} \coloneqq \{\mathbf{P} \succ 0 : \nabla^2 f(\mathbf{x}) \preceq \mathbf{P}^{-1}\}$ . With this definition, we can consider the set of valid preconditioners  $\mathbf{P}$  for which there are no bigger valid preconditioners, that is,  $\mathcal{P} \coloneqq \{\mathbf{P} \in \mathcal{V} : \exists \mathbf{P}' \in \mathcal{V} \text{ s.t. } \mathbf{P} \prec \mathbf{P}'\}$ . However,  $\mathcal{P}$  contains incomparable preconditioners, that is, distinct matrices  $\mathbf{A}, \mathbf{B} \in \mathcal{P}$  that neither  $\mathbf{A} \succeq \mathbf{B}$  nor  $\mathbf{A} \preceq \mathbf{B}$  hold.
Let us look at an example with a quadratic function (illustrated in Figure 12)
$$
f (\mathbf {x}) = \frac {1}{2} \langle \mathbf {x}, \mathbf {A x} \rangle \quad \text {w i t h H e s s i a n} \quad \mathbf {A} = \left[ \begin{array}{l l} . 5 & . 1 \\ . 1 & 1. 0 \end{array} \right]. \tag {11}
$$
There are many preconditioners that are valid, for example using the per-coordinate step-sizes
$$
\mathbf {P} _ {L} \approx \left[ \begin{array}{c c} . 9 1 & 0 \\ 0 & . 9 1 \end{array} \right], \quad \mathbf {P} _ {1} = \left[ \begin{array}{c c} 2. 0 & 0 \\ 0 & 0. 0 \end{array} \right], \quad \mathbf {P} _ {2} = \left[ \begin{array}{c c} 0. 0 & 0 \\ 0 & 1. 0 \end{array} \right], \quad \mathbf {P} _ {*} \approx \left[ \begin{array}{c c} 1. 7 5 & 0 \\ 0 & 0. 8 7 \end{array} \right].
$$
The preconditioner  $\mathbf{P}_L$  corresponds to the  $1 / L$  step-size,  $\mathbf{P}_1$  and  $\mathbf{P}_2$  take the largest possible stepsize in each coordinate, and  $\mathbf{P}_{*}$  is the optimal preconditioner according to Equation (1). Those preconditioners are not comparable to each other, as neither  $\mathbf{P}_L\prec \mathbf{P}_*$  nor  $\mathbf{P}_{*}\prec \mathbf{P}_{L}$  hold. Instead of looking at the matrices themselves, we use in (1) the condition number4 of  $\mathbf{P}^{1 / 2}\nabla^2 f(\mathbf{x})\mathbf{P}^{1 / 2}$  as a measure of quality of  $\mathbf{P}$ . This allows for a well-defined optimal preconditioner as this condition number can be maximized.
<sup>3</sup>Up to invertibility issues which we address in the next subsection.
4Our definition is slightly different, but both notions are equivalent for positive definite  $\mathbf{P}$
B.1 Defining optimal preconditioners without twice-differentiability or strong-convexity
Although we used twice-differentiability of  $f$  to define the optimal preconditioner, this is not necessary. If  $f$  is not twice-differentiable but still strongly-convex, the definition in Equation (1) can be replaced by Equation (2), as finding the  $\mathbf{P}$ -norm under which the function is most strongly-convex.
$$
\mathbf{P}_{*} = \operatorname *{arg  min}_{\substack{\mathbf{P}\succ 0,\text{diagonal}}}\kappa
$$
$$
\text {s u c h t h a t} \left\{ \begin{array}{l} \frac {1}{\kappa} \frac {1}{2} \| \mathbf {x} - \mathbf {y} \| _ {\mathbf {P} ^ {- 1}} ^ {2} \leq f (\mathbf {y}) - f (\mathbf {x}) - \langle \nabla f (\mathbf {x}), \mathbf {y} - \mathbf {x} \rangle , \\ f (\mathbf {y}) - f (\mathbf {x}) - \langle \nabla f (\mathbf {x}), \mathbf {y} - \mathbf {x} \rangle \leq \frac {1}{2} \| \mathbf {y} - \mathbf {x} \| _ {\mathbf {P} ^ {- 1}} ^ {2}, \end{array} \right. \quad \text {f o r a l l} \mathbf {x}, \mathbf {y}.
$$
To avoid strong-convexity, we can instead use the PL inequality. A function  $f$  is  $\mu$ -PL if
$$
\frac {1}{\mu} \frac {1}{2} \| \nabla f (\mathbf {x}) \| ^ {2} \geq f (\mathbf {x}) - f \left(\mathbf {x} _ {*}\right). \tag {12}
$$
This property is implied by  $\mu$ -strong convexity. We refer to the work of Karimi et al. (2016) for the properties of PL functions and its relation to other assumptions. To adapt Equation (12) to our results, we can measure the PL constant  $\mu$  in the norm induced by  $\mathbf{P}$ , and say that  $f$  is  $\mu$ -PL in  $\| \cdot \|_{\mathbf{P}}$  if
$$
\frac {1}{\mu} \frac {1}{2} \| \nabla f (\mathbf {x}) \| _ {\mathbf {P}} ^ {2} \geq f (\mathbf {x}) - f \left(\mathbf {x} _ {*}\right). \tag {13}
$$
We use this inequality in the convergence proof in Proposition 3.2 since it is a consequence of strong-convexity. As this property is the only property of strong-convexity needed for our results, we can adapt our results to be competitive with the optimal preconditioner defined using the PL inequality, using the definition
$$
\mathbf{P}_{*}^{\text{PL}}:= \operatorname *{arg  min}_{\substack{\mathbf{P}\succ 0,\text{diagonal}}}\kappa
$$
$$
\text {s u c h} \left\{ \begin{array}{l l} \frac {1}{\kappa} \| \nabla f (\mathbf {x}) \| _ {\mathbf {P}} ^ {2} \geq f (\mathbf {x}) - f \left(\mathbf {x} _ {*}\right) & \text {f o r a l l} \mathbf {x}, \\ f (\mathbf {y}) - f (\mathbf {x}) - \langle \nabla f (\mathbf {x}), \mathbf {y} - \mathbf {x} \rangle \leq \frac {1}{2} \| \mathbf {y} - \mathbf {x} \| _ {\mathbf {P} ^ {- 1}} ^ {2}, & \text {f o r a l l} \mathbf {x}, \mathbf {y}. \end{array} \right. \tag {14}
$$
If  $f$  is  $\mu$ -PL and  $L$ -smooth, Equation (14) has a feasible solution at  $\mathbf{P} = 1 / L\mathbf{I}$  number  $\kappa = L / \mu$ . The constraint based on the  $\mu$ -PL condition in Equation (14) is weaker than the definition using strong-convexity, as strong-convexity implies the PL inequality. The optimal preconditioner defined using the PL inequality (14) might thus achieve a lower condition number than the one using strong-convexity (1). For example, the quadratic  $f(\mathbf{x}) = (1 / 2)\langle \mathbf{x},\mathbf{A}\mathbf{x}\rangle$  with a positive semi-definite  $\mathbf{A}$  is not strongly convex if the smallest eigenvalue of  $\mathbf{A}$  is 0. The optimal preconditioner in Equation (1) is ill-defined (or has condition number  $\kappa_{*} = \infty$ ). In contrast, the optimal preconditioner defined using the PL inequality in Equation (14) has a finite condition number, as  $\mathbf{P} = 1 / L\mathbf{I}$  is a feasible solution with condition number  $\kappa = L / \lambda_{\min}^{+}(\mathbf{A})$  where  $\lambda_{\min}^{+}(\mathbf{A})$  is the smallest non-zero eigenvalue of  $\mathbf{A}$ . As our proofs only use the properties guaranteed by Equation (14), our results also apply to PL functions.
B.2 Valid and optimal preconditioners with singular matrices
In the main text, we defined valid preconditioners (Definition 4.1) only for positive definite matrices for ease of presentation. The notion of valid preconditioners can be extended to general positive semidefinite matrices. In the diagonal case, the convention  $1 / 0 = +\infty$  is a useful mental model but can cause inconsistencies (such as  $\infty \cdot 0$ ). To extend the notion of valid preconditioners to general positive semidefinite matrices, we can use the definition
Definition B.1. A preconditioner  $\mathbf{P} \succeq 0$  is valid if  $\mathbf{P}^{1/2}\nabla^2 f(\mathbf{x})\mathbf{P}^{1/2} \preceq I$  for all  $\mathbf{x} \in \mathbb{R}^d$ .
The above is well-defined for all positive semidefinite matrices. An alternative to arrive at a definition closer to Definition 4.1 is to consider the projection matrix  $\Pi_{\mathbf{P}}$  onto the image of  $\mathbf{P}$ , given by  $\Pi_{\mathbf{P}} = \mathbf{P}^{1/2} (\mathbf{P}^{1/2})^\dagger$  where  $\mathbf{P}^\dagger$  is the Moore-Penrose pseudo-inverse of  $\mathbf{P}$ . Using that, one can show that  $\mathbf{P}$  is valid (according to Definition B.1) if and only if
$$
\Pi_ {\mathbf {P}} \nabla^ {2} f (\mathbf {x}) \Pi_ {\mathbf {P}} \preceq \mathbf {P} ^ {\dagger} \quad \text {f o r a l l} \mathbf {x} \in \mathbb {R} ^ {d}.
$$
An example of a valid preconditioner that is covered by Definition B.1 but not 4.1 is the all-zeroes matrix. Definition B.1 can seamlessly replace 4.1, and all the results follow similarly. Moreover, notice that the optimization problem defining the optimal preconditioner (1) may not attain its minima on positive definite matrices when  $f$  is not strongly convex. In this case, we can define an optimal
Figure 13: Set of valid diagonal preconditioners (step-sizes  $\mathbf{p}_1$  and  $\mathbf{p}_2$ ) for the quadratic in Equation (15). The set of valid preconditioners (Definition 4.1) is the white region in the right figure.
preconditioner as a limit point of a sequence that attains in the limit the value in (1) by replacing the minimum with an infimum. In this case, an optimal preconditioner may be singular, but the results in the main body also follow seamlessly using this definition. We decided to restrict our attention to non-singular preconditioners in the main paper for ease of exposition, since when  $f$  is strongly-convex, an optimal preconditioner is always non-singular.
B.3 Best competitive ratio achievable by the optimal preconditioner
In Section 3, we mentioned that the optimal preconditioner  $\mathbf{P}_{*}$  could be only  $1 / d$ -competitive. In fact, the competitive ratio of  $\mathbf{P}_{*}$  can be arbitrarily bad. The reason for this is that the competitive ratio  $\gamma$  does not compare against  $\mathbf{P}_{*}$ , but rather against any  $\mathbf{P}$  in the set  $\mathcal{S}$  of potentially valid preconditioners. Moreover, this definition only takes into account the norm  $\| \nabla f(\mathbf{x})\|_{\mathbf{P}}$  at a fixed  $\mathbf{x}$ , while the optimal preconditioner needs to have large norm for all  $\mathbf{x}$ .
For example, consider the scalar step-size case. If our current interval of candidate step-sizes to try is  $S = [0,1]$  but the optimal step-size  $\alpha_{*}$  is small, let us say  $\alpha_{*} = 1 / 10$ , then  $\alpha_{*}$  is only  $1 / 10$ -competitive in  $S$ . The motivation for this definition of competitive ratio is that we cannot check whether  $\alpha$  is large compared to  $\alpha_{*}$  (as we do not know  $\alpha_{*}$ ) but we can more easily ensure that a candidate step-size  $\alpha$  is  $\gamma$ -competitive in  $S$  (for example  $\alpha = 1 / 2$  is  $1 / 2$ -competitive in  $[0,1]$ ).
In the previous example, the bad competitive ratio of  $\alpha_{*}$  in  $S$  was mostly due to the fact that  $S$  was large and that, for some  $\mathbf{x}$ , step sizes larger than  $\alpha_{*}$  could satisfy the Armijo condition (3). Even if  $\alpha_{*}$  is globally optimal, we could make more progress by using a larger step-size if they were to be accepted, and we have not yet ruled out those step-sizes. However, as  $S$  shrinks, it may eventually converge to the interval [0, 1], in which case the optimal step-size  $\alpha_{*}$  would be 1-competitive.
In high dimensions however, the optimal preconditioner can have a competitive ratio of  $\frac{1}{d}$  even when comparing only against valid preconditioners. This is because the competitive ratio is defined using the P-norm of the gradient, and we need to take the direction of the gradient into account. For example, consider the quadratic function (illustrated in Figure 13)
$$
f (\mathbf {x}) = \frac {1}{2} \langle \mathbf {x}, \mathbf {A x} \rangle
$$
$$
\mathbf {A} = \left[ \begin{array}{c c} 1 & - 1 \\ - 1 & 1 \end{array} \right], \tag {15}
$$
with eigenvalues  $\{2,0\}$  as  $\mathbf{A} = [-1,1]^{\mathrm{T}}[-1,1]$ . The following three preconditioners are all valid:
$$
\mathbf {P} _ {1} = \left[ \begin{array}{c c} 1 & 0 \\ 0 & 0 \end{array} \right],
$$
$$
\mathbf {P} _ {2} = \left[ \begin{array}{c c} 0 & 0 \\ 0 & 1 \end{array} \right], \text {a n d}
$$
$$
\mathbf {P} _ {*} = \left[ \begin{array}{c c} 1 / 2 & 0 \\ 0 & 1 / 2 \end{array} \right].
$$
The preconditioner  $\mathbf{P}_1$  takes the largest possible step-size in the first coordinate and ignores the second, while  $\mathbf{P}_2$  does the opposite. They are not good global preconditioners, as each ignores one coordinate. Yet, they can make much more progress (i.e., the objective value may decrease more) than the optimal preconditioner  $\mathbf{P}_*$  if the gradient is very skewed towards one coordinate. This implies that
<sup>5</sup> How small the set  $S_{t}$  can get is bounded by construction. The cutting plane procedure in Sections 4 and 5 only removes invalid preconditioners. The valid preconditioners contained in the initial set  $S_{0}$  will always be in  $S_{t}$ , along with possibly more preconditioners that have not been deemed invalid over the course of optimization.
$\mathbf{P}_{*}$  may be only  $1 / 2$  -competitive in  $\{\mathbf{P}_1,\mathbf{P}_2\}$  for some  $\mathbf{x}$  since
$$
\text {i f} \nabla f (\mathbf {x}) = \left[ \begin{array}{c} 1 \\ 0 \end{array} \right], \text {t h e n} \| \nabla f (\mathbf {x}) \| _ {\mathbf {P} _ {1}} ^ {2} = 1, \| \nabla f (\mathbf {x}) \| _ {\mathbf {P} _ {2}} ^ {2} = 0, \| \nabla f (\mathbf {x}) \| _ {\mathbf {P} _ {*}} ^ {2} = 1 / 2,
$$
and if  $\nabla f(\mathbf{x}) = \left[ \begin{array}{l}0\\ 1 \end{array} \right]$ , then  $\| \nabla f(\mathbf{x})\|_{\mathbf{P}_1}^2 = 0$ ,  $\| \nabla f(\mathbf{x})\|_{\mathbf{P}_2}^2 = 1$ ,  $\| \nabla f(\mathbf{x})\|_{\mathbf{P}_*}^2 = 1/2$ .
The preconditioner  $\mathbf{P}_{*}$  is still a better choice globally (i.e., for all  $\mathbf{x}$ ) since it ensures optimal worst-case linear rate in preconditioned gradient descent. But there are better preconditioners that depend on the current gradient. We exploit this in the ellipsoid variant of multidimensional backtracking to improve our competitive ratio. We backtrack from the preconditioner that maximizes the local progress guarantees to ensure a  $1 / \sqrt{d}$  competitive ratio, while ensuring volume shrinkage of the set of candidate preconditioners when we call CUT, if the preconditioner fails the Armijo condition.
C Separating hyperplanes
In this section, we prove Propositions 4.2 and 4.3 on existence and strengthening of separating hyperplanes for valid preconditioners.
General idea. Let us start with a summary of the separating hyperplanes used to search for good preconditioners as discussed in Sections 3 and 4. The goal of the separating hyperplanes is to give us ways to shrink the initial set of potential preconditioners  $S$  to narrow in on valid preconditioners using the cutting-plane methods in Section 5. At each iteration we are looking for preconditioners  $\mathbf{P}$  that satisfy the Armijo condition at  $\mathbf{x}$  given by
$$
f (\mathbf {x} - \mathbf {P} \nabla f (\mathbf {x})) \leq f (\mathbf {x}) - \frac {1}{2} \| \nabla f (\mathbf {x}) \| _ {\mathbf {P}} ^ {2}.
$$
If  $\mathbf{P}$  fails the Armijo condition, we conclude that  $\mathbf{P}$  is invalid. To obtain more information, we look at the condition as a function of the (diagonal of the) preconditioner, and define the gap function at  $\mathbf{x}$ ,
$$
h (\mathbf {p}) := f (\mathbf {x} - \operatorname {D i a g} (\mathbf {p}) \nabla f (\mathbf {x})) - f (\mathbf {x}) + \frac {1}{2} \| \nabla f (\mathbf {x}) \| _ {\operatorname {D i a g} (\mathbf {p})} ^ {2}, \quad \forall \mathbf {p} \in \mathbb {R} _ {\geq 0} ^ {d}.
$$
Then,  $h(\mathbf{p}) \leq 0$  if  $\mathbf{P} = \mathrm{Diag}(\mathbf{p})$  satisfies the Armijo condition at  $\mathbf{x}$ , and  $h(\mathbf{p}) > 0$  otherwise. Any preconditioner  $\mathrm{Diag}(\mathbf{q})$  such that  $h(\mathbf{q}) > 0$  is guaranteed to be invalid. We can use the gradient of  $h$  at  $\mathbf{p}$  and convexity to find a half-space such that one side contains only preconditioners with  $h(\mathbf{p}) > 0$ . In this section, we show how to construct such half-space, and strengthen them using the partial order on matrices, which is needed to ensure volume shrinkage of our cutting plane methods.
C.1 Stronger hyperplanes
In the main body we presented the strengthening of separating hyperplanes via truncation (Proposition 4.3) after the result of existence of separating hyperplanes (Proposition 4.2). Here, we prove a more general lemma on strengthening half-spaces of invalid preconditioners first, as it is useful in simplifying the proof of Proposition 4.2. Proposition 4.3 follows directly from the following lemma.
Lemma C.1. Let  $\mathcal{H}_{\mathbf{v},\alpha}$  be the intersection of the non-negative orthant  $\mathbb{R}_{\geq 0}^{d}$  and the half-space defined by the vector  $\mathbf{v} \in \mathbb{R}^d$  and coefficient  $\alpha > 0$ ,
$$
\mathcal {H} _ {\mathbf {v}, \alpha} := \left\{\mathbf {p} \in \mathbb {R} _ {\geq 0} ^ {d}: \langle \mathbf {v}, \mathbf {p} \rangle > \alpha \right\}.
$$
Define  $\mathbf{u} \coloneqq \max \{\mathbf{v}, 0\}$  and let  $\mathcal{H}_{\mathbf{u},\alpha}$  be defined similarly as above, that is,
$$
\mathcal {H} _ {\mathbf {u}, \alpha} := \left\{\mathbf {p} \in \mathbb {R} _ {\geq 0} ^ {d}: \langle \mathbf {u}, \mathbf {p} \rangle > \alpha \right\}.
$$
If  $\mathcal{H}_{\mathbf{v},\alpha}$  only contains diagonals of invalid preconditioners, that is,  $\operatorname{Diag}(\mathbf{p})$  is invalid for any  $\mathbf{p} \in \mathcal{H}_{\mathbf{v}}$ , Then  $\mathcal{H}_{\mathbf{v},\alpha} \subseteq \mathcal{H}_{\mathbf{u},\alpha}$  and  $\mathcal{H}_{\mathbf{u},\alpha}$  only contains diagonals of invalid preconditioners.
Proof. Inclusion  $\mathcal{H}_{\mathbf{v},\alpha} \subseteq \mathcal{H}_{\mathbf{u},\alpha}$ . We have that  $\langle \mathbf{p}, \mathbf{v} \rangle > \alpha$  implies  $\langle \mathbf{p}, \mathbf{u} \rangle > \alpha$  for any  $\mathbf{p} \in \mathbb{R}_{\geq 0}^d$  since
$$
\langle \mathbf {v}, \mathbf {p} \rangle = \sum_ {i: \mathbf {v} [ i ] \geq 0} \mathbf {v} [ i ] \mathbf {p} [ i ] + \sum_ {i: \mathbf {v} [ i ] <   0} \mathbf {v} [ i ] \mathbf {p} [ i ] \leq \sum_ {i: \mathbf {v} [ i ] \geq 0} \mathbf {v} [ i ] \mathbf {p} [ i ] = \sum_ {i: \mathbf {v} [ i ] \geq 0} \mathbf {u} [ i ] \mathbf {p} [ i ] = \langle \mathbf {u}, \mathbf {p} \rangle .
$$
$\mathcal{H}_{\mathbf{u},\alpha}$  only contains invalid diagonals. Let  $\mathbf{p_u} \in \mathcal{H}_{\mathbf{u},\alpha}$ . We can show that  $\mathrm{Diag}(\mathbf{p_u})$  is invalid by finding  $\mathbf{p_v} \in \mathcal{H}_{\mathbf{v},\alpha}$  such that  $\mathrm{Diag}(\mathbf{p_v}) \preceq \mathrm{Diag}(\mathbf{p_u})$ . Since  $\mathrm{Diag}(\mathbf{p_v})$  is invalid by assumption, this would imply that  $\mathrm{Diag}(\mathbf{p_u})$  is also invalid. To find  $\mathbf{p_v}$ , we can truncate the entries of  $\mathbf{p_u}$  as
$$
\mathbf {p} _ {\mathbf {v}} [ i ] := \left\{ \begin{array}{l l} \mathbf {p} _ {\mathbf {u}} [ i ] & \text {i f} \mathbf {v} [ i ] \geq 0 \\ 0 & \text {o t h e r w i s e}, \end{array} \right. \quad \forall i \in \{1, \ldots , d \}.
$$
Then  $\mathbf{p_v} \in \mathcal{H}_{\mathbf{v},\alpha}$  since  $\alpha < \langle \mathbf{u}, \mathbf{p_u} \rangle = \langle \mathbf{u}, \mathbf{p_v} \rangle = \langle \mathbf{v}, \mathbf{p_v} \rangle$ .  $^6$  and  $\operatorname{Diag}(\mathbf{p_u}) \succeq \operatorname{Diag}(\mathbf{p_v})$ , as desired.
One may worry that our original definition of valid preconditioners has a division by 0 if any entry of the preconditioner is 0 as a preconditioner is valid if  $\nabla^2 f(\mathbf{x}) \preceq \mathbf{P}^{-1}$  (Definition 4.1). It is enough to use the convention that  $1/0 = +\infty$ , although this might lead to inconsistencies. In Appendix B.2 we discuss a more general definition without the use of infinities.
C.2 Separating hyperplanes for invalid preconditioners
We are now in position to prove Proposition 4.2.
Proof of Proposition 4.2. Throughout the proof, we shall denote by  $\mathbf{P}$  the matrix  $\operatorname{Diag}(\mathbf{p})$ . If  $f$  is convex, then  $h$  also is since the map  $\mathbf{p} \in \mathbb{R}_{\geq 0}^{d} \mapsto f(\mathbf{x} - \mathbf{P}\nabla f(\mathbf{x}))$  is the composition of an affine transformation and a convex function, and  $\|\nabla f(\mathbf{x})\|_{\mathbf{P}}^{2} = \langle \nabla f(\mathbf{x}), \operatorname{Diag}(\mathbf{p})\nabla f(\mathbf{x}) \rangle$  is linear in  $\mathbf{p}$ . Convexity of  $h$  yields the inequality
$$
h (\mathbf {p}) \geq h (\mathbf {q}) + \left\langle \nabla h (\mathbf {q}), \mathbf {p} - \mathbf {q} \right\rangle , \quad \forall \mathbf {p} \in \mathbb {R} _ {\geq 0} ^ {d}.
$$
This implies that if  $\mathbf{p}$  is such that  $h(\mathbf{q}) + \langle \nabla h(\mathbf{q}),\mathbf{p} - \mathbf{q}\rangle >0$ , then  $h(\mathbf{p}) > 0$ , which implies that  $\operatorname {Diag}(\mathbf{p})$  is an invalid preconditioner. Rearranging we conclude that  $\operatorname {Diag}(\mathbf{p})$  is invalid for all  $\mathbf{p}$  in the set in (5), i.e., in
$$
\left\{\mathbf {p} \in \mathbb {R} _ {\geq 0} ^ {d}: \langle \nabla h (\mathbf {q}), \mathbf {p} \rangle > \langle \nabla h (\mathbf {q}), \mathbf {q} \rangle - h (\mathbf {q}) \right\} \tag {16}
$$
We express the above half-space as
$$
\mathcal {H} _ {>} (\mathbf {v}) = \left\{\mathbf {p}: \langle \mathbf {p}, \mathbf {v} \rangle > 1 \right\} \text {f o r} \mathbf {v} := \frac {\nabla h (\mathbf {q})}{(\langle \nabla h (\mathbf {q}) , \mathbf {q} \rangle - h (\mathbf {q}))}.
$$
Yet, for  $\mathcal{H}_{>}(\mathbf{v})$  to be equivalent to the set in (16) or even to be well-defined, we need to ensure  $\langle \nabla h(\mathbf{q}),\mathbf{q}\rangle -h(\mathbf{q}) > 0$ . To see that this holds, note first that by convexity of  $h$  and that fact that  $h(0) = 0$  we have
$$
h (0) \geq h (\mathbf {q}) + \langle \nabla h (\mathbf {q}), 0 - \mathbf {q} \rangle \Rightarrow \langle \nabla h (\mathbf {q}), \mathbf {q} - 0 \rangle - h (\mathbf {q}) \geq - h (0) = 0
$$