SlowGuess/NeurIPS_2023_OCR huggingface dataset:
$$
\frac {1}{2 L} \left\| \nabla f \left(y _ {N}\right) \right\| ^ {2} \leq \frac {1}{T _ {N}} \left(f \left(y _ {0}\right) - f _ {\star}\right).
$$
Proof outline. By Theorem 1, (C2) holds with  $v_{i} = 1 / T_{N - i}$  for  $0\leq i\leq N$ . We then use (5).
When  $T_{i} = t_{i}^{2}$  for  $0 \leq i \leq N$ , the FSFOM (6) reduces to Nestrov's FGM [37] and its H-dual is, to the best of our knowledge, a new method without a name. If  $t_{i}^{2} = 2T_{i}$  for  $0 \leq i \leq N - 1$  and  $t_{N}^{2} = T_{N}$ , (6) reduces to (OGM) and its H-dual is (OGM-G). If  $t_{i} = i + 1$  for  $0 \leq i \leq N - 1$  and  $t_{N} = \sqrt{N(N + 1) / 2}$ , (6) reduces to (OBL-  $\mathbf{F}_{\flat}$ ) and its H-dual is (OBL-  $\mathbf{G}_{\flat}$ ).
Gradient magnitude rate of (GD). For gradient descent (GD) with stepsize  $h$ , the  $H$  matrix is the identity matrix scaled by  $h$ , and the H-dual is (GD) itself, i.e., (GD) is self-dual. For  $0 < h \leq 1$ , the rate  $f(x_{N}) - f_{\star} \leq \frac{1}{2Nh + 1}\frac{L}{2}\| x_{0} - x_{\star}\|^{2}$ , originally due to [19], can be established from (2) with  $\{u_i\}_{i = 0}^N = \left(\dots ,\frac{(2Nh + 1)(i + 1)}{2N - i},\dots ,2Nh + 1\right)$ . Applying Theorem 1 leads to the following.
Corollary 2. Consider (GD) with  $0 < h \leq 1$  applied to an  $L$ -smooth convex  $f$ . For  $N \geq 1$ ,
$$
\frac {1}{2 L} \left\| \nabla f (x _ {N}) \right\| ^ {2} \leq \min  \left(\frac {f (x _ {0}) - f _ {\star}}{2 N h + 1}, \frac {L \left\| x _ {0} - x _ {\star} \right\| ^ {2}}{2 \left(2 \left\lfloor \frac {N}{2} \right\rfloor h + 1\right)\left(2 \left\lceil \frac {N}{2} \right\rceil h + 1\right)}\right).
$$
To the best of our knowledge, Corollary 2 is the tightest rate on gradient magnitude for (GD) for the general step size  $0 < h < 1$ , and it matches [53, Theorem 3] for  $h = 1$ .
Resolving conjectures of  $\mathcal{A}^{\star}$ -optimality of (OGM-G) and  $(\mathrm{OBL - F_b})$ . The prior work of [44] defines the notion of  $\mathcal{A}^{\star}$ -optimality, a certain restricted sense of optimality of FSFOMs, and shows that (OGM) and  $(\mathrm{OBL - F_b})$  are  $\mathcal{A}^{\star}$ -optimal under a certain set of relaxed inequalities. On the other hand,  $\mathcal{A}^{\star}$ -optimality of (OGM-G) and  $(\mathrm{OBL - G_b})$  are presented as conjectures. Combining Theorem 1 and the  $\mathcal{A}^{\star}$ -optimality of (OGM) and  $(\mathrm{OBL - F_b})$  resolves these conjectures; (OGM-G) and  $(\mathrm{OBL - G_b})$  are  $\mathcal{A}^{\star}$ -optimal.
2.6 Intuition behind energy functions: Lagrangian Formulation
One might ask where the energy functions (2) and (3) came from. In this section, we provide an intuitive explanation of these energy functions using the QCQP and its Lagrangian. Consider an FSFOM (1) given with a lower triangular matrix  $H \in \mathbb{R}^{N \times N}$  with function  $f$ , resulting in the sequence  $\{x_i\}_{i=0}^N$ . To analyze the convergence rate of the function value, we formulate the following optimization problem:
$$
\sup  _ {f} f (x _ {N}) - f _ {\star}
$$
subject to  $f\colon \mathbb{R}^n\to \mathbb{R}$  is  $L$  -smooth convex,  $\| x_0 - x_\star \| ^2\leq R^2$
However, this optimization problem is not solvable, as  $f$  is a functional variable. To address this, [19, 55] demonstrated its equivalence to a QCQP:
subject to  $\llbracket x_i,x_j\rrbracket \leq 0\quad [i,j]\in [-1,\dots ,N]^2,\quad \| x_0 - x_\star \| ^2\leq R^2$
where  $x_{-1} \coloneqq x_{\star}$ . To clarify, the optimization variables are  $\{\nabla f(x_i), f(x_i)\}_{i=0}^N$ ,  $f_{\star}$ ,  $x_0$ , and  $x_{\star}$  since  $\{x_i\}_{i=0}^N$  lies within  $\operatorname{span}\{x_0, \nabla f(x_0), \ldots, \nabla f(x_N)\}$ . We consider a relaxed optimization problem as [26]:
subject to  $\llbracket x_i, x_{i+1}\rrbracket \leq 0$ ,  $i = 0, 1, \ldots, N-1$ ,  $\llbracket x_\star, x_i\rrbracket \leq 0$ ,  $i = 0, 1, \ldots, N$ ,
$$
\left\| x _ {0} - x _ {\star} \right\| ^ {2} \leq R ^ {2}
$$
Now, consider the Lagrangian function and the convex dual.
$$
\mathcal {L} _ {1} \left(f, \left\{a _ {i} \right\} _ {i = 0} ^ {N - 1}, \left\{b _ {i} \right\} _ {i = 0} ^ {N}, \alpha\right) = - f \left(x _ {N}\right) + f _ {\star} + \sum_ {i = 0} ^ {N - 1} a _ {i} \llbracket x _ {i}, x _ {i + 1} \rrbracket + \sum_ {i = 0} ^ {N} b _ {i} \llbracket x _ {\star}, x _ {i} \rrbracket + \alpha \| x _ {0} - x _ {\star} \| ^ {2} - \alpha R ^ {2}
$$
where  $\{a_i\}_{i = 0}^{N - 1},\{b_i\}_{i = 0}^N$  , and  $\alpha$  are dual variables which are nonnegative. Considering  $a_{-1} = 0$  and  $a_N = 1$  , the infimum of  $\mathcal{L}_1$  equals  $-\infty$  unless  $b_{i} = a_{i} - a_{i - 1}$  for  $0\leq i\leq N$  . Therefore, by introducing  $u_{N} = \frac{L}{2\alpha}$  and  $u_{i} = a_{i}u_{N}$  for  $0\leq i\leq N$  , the convex dual problem can be simplified as follows:
$$
\inf_{\substack{\{u_{i}\}_{i = 0}^{N}\\ }} - \frac{LR^{2}}{2u_{N}}
$$
$$
\text{s.t.}\inf_{x_{0},x_{\star},\{\nabla f(x_{i})\}_{i = 0}^{N}} - u_{N}(f(x_{N}) - f_{\star}) + \frac{L}{2}\| x_{0} - x_{\star}\|^{2} + \sum_{i = 0}^{N - 1}u_{i}[[x_{i},x_{i + 1}]] + \sum_{i = 0}^{N}(u_{i} - u_{i - 1})[[x_{\star},x_{i}]]\geq 0
$$
$\{u_i\}_{i = 0}^N$  are nonnegative and nondecreasing.
If the above two constraints hold for  $\{u_i\}_{i=0}^N$ , we have  $f(x_N) - f_\star \leq \frac{L}{2u_N} R^2$ . This understanding motivates the introduction of (2) and (C1). We can perform a similar analysis on the gradient norm minimization problem with the relaxed optimization problem as follows:
$$
\sup _ {f} \frac {1}{2 L} \| \nabla f (y _ {N}) \| ^ {2}
$$
subject to  $\llbracket y_{i},y_{i + 1}\rrbracket \leq 0,\quad i = 0,1,\ldots ,N - 1,\quad \llbracket y_{N},y_{i}\rrbracket \leq 0,\quad i = 0,1,\ldots ,N - 1,$
$$
\llbracket y _ {N}, \star \rrbracket \leq 0, \quad f (y _ {0}) - f _ {\star} \leq R.
$$
Finally, we note that although (2) and (3) both originate from the relaxed optimization problems, they have been commonly employed to achieve the convergence analysis. The function value convergence rate of OGM[26], FGM[37], G-OGM[28], GD[19], OBL- $\mathrm{F}_b$ [44] can be proved by using (2) with appropriate  $\{u_i\}_{i=0}^N$ . The gradient norm convergence rate of OGM-G[29], OBL- $\mathrm{G}_b$ [44], M-OGM-G[60], and [17, Lemma 2.6] can be proved by using (3) with appropriate  $\{v_i\}_{i=0}^N$ . We also note that recent works [2, 25] do not employ (2) to achieve the convergence rate, particularly for gradient descent with varying step sizes.
3 H-duality in continuous time
We now establish a continuous-time analog of the H-duality theorem. As the continuous-time result and, especially, its proof is much simpler than its discrete-time counterpart, the results of this section serve as a vehicle to convey the key ideas more clearly. Let  $T > 0$  be a pre-specified terminal time. Let  $H(t,s)$  be an appropriately integrable $^2$  real-valued kernel with domain  $\{(t,s) | 0 < s < t < T\}$ . We define a Continuous-time Fixed Step First Order Method (C-FSFOM) with  $H$  as
$$
X (0) = x _ {0}, \quad \dot {X} (t) = - \int_ {0} ^ {t} H (t, s) \nabla f (X (s)) d s, \quad \forall t \in (0, T) \tag {7}
$$
for any initial point  $x_0 \in \mathbb{R}^d$  and differentiable  $f$ . Note, the Euler discretization of C-FSFOMs (7) corresponds to FSFOMs (1). The notion of C-FSFOMs has been considered previously in [30].
Given a kernel  $H(t,s)$ , analogously define its anti-transpose as  $H^{A}(t,s) = H(T - s,T - t)$ . We call [C-FSFOM with  $H^{A}$ ] the H-dual of [C-FSFOM with  $H$ ]. In the special case  $H(t,s) = e^{\gamma (s) - \gamma (t)}$  for some function  $\gamma (\cdot)$ , the C-FSFOMs with  $H$  and its H-dual have the form
$$
\ddot {X} (t) + \dot {\gamma} (t) \dot {X} (t) + \nabla f (X (t)) = 0 \quad (\text {C - F S F O M w i t h} H (t, s) = e ^ {\gamma (s) - \gamma (t)})
$$
$$
\ddot {Y} (t) + \dot {\gamma} (T - t) \dot {Y} (t) + \nabla f (Y (t)) = 0 \quad (\text {C - F S F O M w i t h} H ^ {A} (t, s))
$$
Interestingly, friction terms with  $\gamma^\prime$  have time-reversed dependence between the H-duals, and this is why we refer to this phenomenon as time-reversed dissipation.
3.1 Continuous-time H-duality theorem
For the C-FSFOM with  $H$ , define the energy function
$$
\mathcal {U} (t) = \frac {1}{2} \| X (0) - x _ {\star} \| ^ {2} + \int_ {0} ^ {t} u ^ {\prime} (s) [ x _ {\star}, X (s) ] d s \tag {8}
$$
for  $t \in [0, T]$  with differentiable  $u \colon (0, T) \to \mathbb{R}$ . If  $u'(\cdot) \geq 0$ , then  $\{\mathcal{U}(t)\}_{t \in [0, T]}$  is dissipative. Assume we can show
$$
u (T) \left(f (X (T)) - f _ {\star}\right) \leq \mathcal {U} (T) \quad (\forall X (0), x _ {\star}, \{\nabla f (X (s)) \} _ {s \in [ 0, T ]} \in \mathbb {R} ^ {d}). \tag {C3}
$$
Then, the C-FSFOM with  $H$  exhibits the convergence rate
$$
u (T) \left(f (X (T)) - f _ {\star}\right) \leq \mathcal {U} (T) \leq \mathcal {U} (0) = \frac {1}{2} \| X (0) - x _ {\star} \| ^ {2}.
$$
For the C-FSFOM with  $H^{A}$ , define the energy function
$$
\mathcal {V} (t) = v (0) \left(f (Y (0)) - f (Y (T))\right) + \int_ {0} ^ {t} v ^ {\prime} (s) [ Y (T), Y (s) ] d s \tag {9}
$$
for  $t \in [0, T]$  with differentiable  $v \colon (0, T) \to \mathbb{R}$ . If  $v'(\cdot) \geq 0$ , then  $\{\mathcal{V}(t)\}_{t \in [0, T]}$  is dissipative. Assume we can show
$$
\frac {1}{2} \| \nabla f (Y (T)) \| ^ {2} \leq \mathcal {V} (T) \quad (\forall Y (0), \{\nabla f (Y (s)) \} _ {s \in [ 0, T ]} \in \mathbb {R} ^ {d}). \tag {C4}
$$
Then, the C-FSFOM with  $H^{A}$  exhibits the convergence rate
$$
\frac {1}{2} \| \nabla f (Y (T)) \| ^ {2} \leq \mathcal {V} (T) \leq \mathcal {V} (0) = v (0) \left(f (Y (0)) - f (Y (T))\right) \leq v (0) \left(f (Y (0)) - f _ {\star}\right).
$$
Theorem 2 (informal). Consider differentiable functions  $u, v \colon (0,T) \to \mathbb{R}$  related through  $v(t) = \frac{1}{u(T - t)}$  for  $t \in [0,T]$ . Assume certain regularity conditions (specified in Appendix C.2). Then,
$$
[ (\mathrm {C} 3) \text {i s s a t i s f i e d w i t h} u (\cdot) \text {a n d} H ] \Leftrightarrow [ (\mathrm {C} 4) \text {i s s a t i s f i e d w i t h} v (\cdot) \text {a n d} H ^ {A} ].
$$
The formal statement of 2 and its proof are given in Appendix C.2. Loosely speaking, we can consider Theorem 2 as the limit of Theorem 1 with  $N \to \infty$ .
In this paper, we avoid analytical and measure-theoretic details and focus on convergence (rather than existence) results of the continuous-time dynamics.
3.2 Verifying conditions for H-duality theorem
As an illustrative example, consider the case  $H(t,s) = \frac{s^r}{t^r}$  for  $r \geq 3$  which corresponds to an ODE studied in the prior work [50, 51]. For the C-FSFOM with  $H$ , the choice  $u(t) = \frac{t^2}{2(r - 1)}$  for the dissipative energy function  $\{\mathcal{U}(t)\}_{t=0}^{T}$  of (8) leads to
$$
\mathcal {U} (T) - u (T) \left(f (X (T)) - f _ {\star}\right) = \frac {\left\| T \dot {X} (T) + 2 (X (T) - x _ {\star}) \right\| ^ {2} + 2 (r - 3) \| X (T) - x _ {\star} \| ^ {2}}{4 (r - 1)} + \int_ {0} ^ {T} \frac {(r - 3) s}{2 (r - 1)} \left\| \dot {X} (s) \right\| ^ {2} d s.
$$
For the C-FSFOM with  $H^{A}$ , the choice  $v(t) = \frac{1}{u(T - t)} = \frac{2(r - 1)}{(T - t)^{2}}$  for the dissipative energy function  $\{\mathcal{V}(t)\}_{t=0}^{T}$  of (9) leads to
$$
\mathcal {V} (T) - \frac {1}{2} \| \nabla f (Y (T)) \| ^ {2} = \frac {2 (r - 1) (r - 3) \| Y (0) - Y (T) \| ^ {2}}{T ^ {4}} + \int_ {0} ^ {T} \frac {2 (r - 1) (r - 3) \left\| (T - s) \dot {Y} (s) + 2 (Y (s) - Y (T)) \right\| ^ {2}}{(T - s) ^ {5}} d s.
$$
Since the right-hand sides are expressed as sums/integrals of squares, they are nonnegative, so (C3) and (C4) hold. (By Theorem 2, verifying (C3) implies (C4) and vice versa.) The detailed calculations are provided in Appendix C.1.
3.3 Applications of continuous-time H-duality theorem
The C-FSFOM (7) with  $H(t,s) = \frac{Cp^2s^{2p - 1}}{t^{p + 1}}$  recovers
$$
\ddot {X} (t) + \frac {p + 1}{t} \dot {X} (t) + C p ^ {2} t ^ {p - 2} \nabla f (X (t)) = 0,
$$
an ODE considered in [56]. The rate  $f(X(T)) - f_{\star} \leq \frac{1}{2CT^p}\| X(0) - x_{\star}\|^2$  can be established from (8) with  $u(t) = Ct^p$ . The C-FSFOM with  $H^{A}$  can be expressed as the ODE
$$
\ddot {Y} (t) + \frac {2 p - 1}{T - t} \dot {Y} (t) + C p ^ {2} (T - t) ^ {p - 2} \nabla f (Y (t)) = 0. \tag {10}
$$
By Theorem 2, using (9) with  $v(t) = \frac{1}{C(T - t)^p}$  leads to the rate
$$
\frac {1}{2} \| \nabla f (Y (T)) \| ^ {2} \leq \frac {1}{C T ^ {p}} \left(f (Y (0)) - f _ {\star}\right).
$$
Note that the continuous-time models of (OGM) and (OGM-G), considered in [51], are special cases of this setup with  $p = 2$  and  $C = 1/2$ . The detailed derivation and well-definedness of the ODE are presented in Appendix C.3.
4 New method efficiently reducing gradient mapping norm: (SFG)
In this section, we introduce a novel algorithm obtained using the insights of Theorem 1. Consider minimizing  $F(x) \coloneqq f(x) + g(x)$ , where  $f \colon \mathbb{R}^d \to \mathbb{R}$  is  $L$ -smooth convex with  $0 < L < \infty$  and  $g \colon \mathbb{R}^d \to \mathbb{R} \cup \{\infty\}$  is a closed convex proper function. Write  $F_{\star} = \inf_{x \in \mathbb{R}^n} F(x)$  for the minimum value. For  $\alpha > 0$ , define the  $\alpha$ -proximal gradient step as
$$
y ^ {\oplus , \alpha} = \operatorname * {a r g m i n} _ {z \in \mathbb {R} ^ {n}} \left(f (y) + \langle \nabla f (y), z - y \rangle + g (z) + \frac {\alpha L}{2} \left\| z - y \right\| ^ {2}\right) = \operatorname * {P r o x} _ {\frac {g}{\alpha L}} \left(y - \frac {1}{\alpha L} \nabla f (y)\right).
$$
Consider FSFOMs defined by a lower triangular matrix  $H = \{h_{k,i}\}_{0\leq i < k\leq N}$  as follows:
$$
x _ {k + 1} = x _ {k} - \sum_ {i = 0} ^ {k} \alpha h _ {k + 1, i} \left(x _ {i} - x _ {i} ^ {\oplus , \alpha}\right), \quad \forall k = 0, \dots , N - 1.
$$
When  $g = 0$ , this reduces to (1). FISTA [11], FISTA-G [31] and GFPGM [27] are instances of this FSFOM with  $\alpha = 1$ . In this section, we present a new method for efficiently reducing the gradient mapping norm. This method is faster than the prior state-of-the-art FISTA-G [31] by a constant factor of 5.28 while having substantially simpler coefficients.
Theorem 3. Consider the method
$$
y _ {k + 1} = y _ {k} ^ {\oplus , 4} + \frac {(N - k + 1) (2 N - 2 k - 1)}{(N - k + 3) (2 N - 2 k + 1)} \left(y _ {k} ^ {\oplus , 4} - y _ {k - 1} ^ {\oplus , 4}\right) + \frac {(4 N - 4 k - 1) (2 N - 2 k - 1)}{6 (N - k + 3) (2 N - 2 k + 1)} \left(y _ {k} ^ {\oplus , 4} - y _ {k}\right)
$$
$$
y _ {N} = y _ {N - 1} ^ {\oplus , 4} + \frac {3}{1 0} \left(y _ {N - 1} ^ {\oplus , 4} - y _ {N - 2} ^ {\oplus , 4}\right) + \frac {3}{4 0} \left(y _ {N - 1} ^ {\oplus , 4} - y _ {N - 1}\right) \tag {SFG}
$$
for  $k = 0, \dots, N - 2$ , where  $y_{-1}^{\oplus,4} = y_0$ . This method exhibits the rate
$$
\min _ {v \in \partial F (y _ {N} ^ {\oplus , 4})} \| v \| ^ {2} \leq 2 5 L ^ {2} \left\| y _ {N} - y _ {N} ^ {\oplus , 4} \right\| ^ {2} \leq \frac {5 0 L}{(N + 2) (N + 3)} \left(F (y _ {0}) - F _ {\star}\right).
$$
We call this method Super FISTA-  $G$  (SFG), and in Appendix D.3, we present a further general parameterized family (SFG-family). To derive (SFG-family), we start with the parameterized family GFPGM [27], which exhibits an accelerated rate on function values, and expresses it as FSFOMs with  $H$ . We then obtain the FSFOMs with  $H^{A} + C$ , where  $C$  is a lower triangular matrix satisfying certain constraints. We find that the appropriate H-dual for the composite setup is given by this  $H^{A} + C$ , rather than  $H^{A}$ . We provide the proof of Theorem 3 in Appendix D.2.
(SFG) is an instance of (SFG-family) with simple rational coefficients. Among the family, the optimal choice has complicated coefficients, but its rate has a leading coefficient of 46, which is slightly smaller than the 50 of (SFG). We provide the details in Appendix D.4.
In this work, we defined the notion of H-duality and formally established that the H-dual of an optimization method designed to efficiently reduce function values is another method that efficiently reduces gradient magnitude. For optimization algorithms, the notion of equivalence, whether informal or formal [59], is intuitive and standard. For optimization problems, the notion of equivalence is also standard, but the beauty of convex optimization is arguably derived from the elegant duality of optimization problems. In fact, there are many notions of duality for spaces, problems, operators, functions, sets, etc. However, the notion of duality for algorithms is something we, the authors, are unfamiliar with within the context of optimization, applied mathematics, and computer science. In our view, the significance of this work is establishing the first instance of a duality of algorithms. The idea that an optimization algorithm is an abstract mathematical object that we can take the dual of opens the door to many interesting questions. In particular, exploring for what type of algorithms the H-dual or a similar notion of duality makes sense is an interesting direction for future work.
We thank Soonwon Choi, G. Bruno De Luca, and Eva Silverstein for sharing their insights as physicists. We thank Jaewook J. Suh for reviewing the manuscript and providing valuable feedback. We are grateful to Jungbin Kim and Jeongwhan Lee for their valuable discussions at the outset of this work. J.K. and E.K.R. were supported by the National the Samsung Science and Technology Foundation (Project Number SSTF-BA2101-02). C.P. acknowledges support from the Xianhong Wu Fellowship, the Korea Foundation for Advanced Studies, and the Siebel Scholarship. A.O. acknowledges support from the MIT-IBM Watson AI Lab grant 027397-00196, Trustworthy AI grant 029436-00103, and the MIT-DSTA grant 031017-00016. Finally, we also thank anonymous reviewers for giving thoughtful comments.
[1] K. Ahn and S. Sra. From Nesterov's estimate sequence to Riemannian acceleration.  $COLT$ , 2020.
[2] J. M. Altschuler and P. A. Parrilo. Acceleration by stepsize hedging II: Silver stepsize schedule for smooth convex optimization. arXiv:2309.16530, 2023.
[3] A. Y. Aravkin, J. V. Burke, D. Drusvyatskiy, M. P. Friedlander, and K. J. MacPhee. Foundations of gauge and perspective duality. SIAM Journal on Optimization, 28(3):2406-2434, 2018.
[4] H. Attouch and M. Théra. A general duality principle for the sum of two operators. Journal of Convex Analysis, 3:1-24, 01 1996.
[5] J.-F. Aujol and C. Dossal. Optimal rate of convergence of an ODE associated to the fast gradient descent schemes for  $b > 0$ . HAL Archives Ouvertes, 2017.
[6] J.-F. Aujol, C. Dossal, G. Fort, and É. Moulines. Rates of convergence of perturbed FISTA-based algorithms. HAL Archives Ouvertes, 2019.
[7] J.-F. Aujol, C. Dossal, and A. Rondepierre. Optimal convergence rates for Nesterov acceleration. SIAM Journal on Optimization, 29(4):3131-3153, 2019.
[8] A. Auslender and M. Teboulle. Interior gradient and proximal methods for convex and conic optimization. SIAM Journal on Optimization, 16(3):697-725, 2006.
[9] M. Baes. Estimate sequence methods: extensions and approximations. 2009.
[10] N. Bansal and A. Gupta. Potential-function proofs for gradient methods. Theory of Computing, 15(4):1-32, 2019.
[11] A. Beck and M. Teboulle. A fast iterative shrinkage-thresholding algorithm for linear inverse problems. SIAM Journal on Imaging Sciences, 2(1):183-202, 2009.
[12] S. Boyd and L. Vandenberghe. Convex optimization. Cambridge university press, 2004.
[13] R. E. Bruck. On the weak convergence of an ergodic iteration for the solution of variational inequalities for monotone operators in Hilbert space. Journal of Mathematical Analysis and Applications, 61(1):159-164, 1977.
[14] P. L. Combettes and V. R. Wajs. Signal recovery by proximal forward-backward splitting. Multiscale Modeling and Simulation, 4(4):1168-1200, 2005.
[15] G. B. Dantzig, E. Eisenberg, and R. W. Cottle. Symmetric dual nonlinear programs. Pacific Journal of Mathematics, 15(3):809-812, 1965.
[16] I. Daubechies, M. Defrise, and C. De Mol. An iterative thresholding algorithm for linear inverse problems with a sparsity constraint. Communications on Pure and Applied Mathematics, 57(11):1413-1457, 2004.
[17] J. Diakonikolas and P. Wang. Potential function-based framework for making the gradients small in convex and min-max optimization. SIAM Journal on Optimization, 32(3):1668-1697, 2021.
[18] Y. Drori. The exact information-based complexity of smooth convex minimization. Journal of Complexity, 39:1-16, 2017.
[19] Y. Drori and M. Teboulle. Performance of first-order methods for smooth convex minimization: a novel approach. Mathematical Programming, 145(1):451-482, 2014.
[20] W. Fenchel. On conjugate convex functions. Canadian Journal of Mathematics, 1(1):73-77, 1949.
[21] R. M. Freund. Dual gauge programs, with applications to quadratic programming and the minimum-norm problem. Mathematical Programming, 38(1):47-67, 1987.
[22] M. P. Friedlander, I. Macô do, and T. K. Pong. Gauge optimization and duality. SIAM Journal on Optimization, 24(4):1999-2022, 2014.
[23] B. Grimmer. Radial duality part II: Applications and algorithms. arXiv:2104.11185, 2021.
[24] B. Grimmer. Radial duality part I: Foundations. arXiv:2104.11179, 2022.
[25] B. Grimmer. Provably faster gradient descent via long steps. arXiv:2307.06324, 2023.
[26] D. Kim and J. A. Fessler. Optimized first-order methods for smooth convex minimization. Mathematical Programming, 159(1):81-107, 2016.