SlowGuess/NeurIPS_2023_OCR huggingface dataset:
The functions  $f_{t}$  are linear, so they are convex. In order to show that they satisfy Assumptions A3 and A4, it remains to show that they are 1-Lipschitz continuous. Let  $h = (x^{(1)},\ldots ,x^{(m)})$  and
$\tilde{h} = (\tilde{x}^{(1)},\dots ,\tilde{x}^{(m)})$  be arbitrary elements of  $\mathcal{H} = \mathcal{X}^m$  . We have
$$
\begin{array}{l} \left| f _ {t} (h) - f _ {t} (\tilde {h}) \right| \\ \leq \left| \epsilon_ {\lceil \frac {t}{m} \rceil} m ^ {\frac {1 - p}{p}} \left(\left(x ^ {(1)} - \tilde {x} ^ {(1)}\right) + \dots + \left(x ^ {(m)} - \tilde {x} ^ {(m)}\right)\right) \right| \\ \leq m ^ {\frac {1 - p}{p}} \left| \left(x ^ {(1)} - \tilde {x} ^ {(1)}\right) + \dots + \left(x ^ {(m)} - \tilde {x} ^ {(m)}\right) \right| \quad \text {b e c a u s e} \epsilon_ {\lceil \frac {t}{m} \rceil} \in \{- 1, + 1 \} \\ \leq m ^ {\frac {1 - p}{p}} m ^ {1 - \frac {1}{p}} \left(\sum_ {k = 1} ^ {m} \left| x ^ {(k)} - \tilde {x} ^ {(k)} \right| ^ {p}\right) ^ {\frac {1}{p}} \quad \text {b y} \\ = \| h - \tilde {h} \| _ {\mathcal {H}}, \\ \end{array}
$$
where the last equality follows because of our assumption that the problem that follows linear sequence dynamics with the  $p$ -norm.
First we will show that the total expected loss of any algorithm is 0, where the expectation is with respect to the randomness in the choice of  $\{\epsilon_1,\dots ,\epsilon_N\}$ . The total loss in the first block is 0 because  $f_{t} = 0$  for  $t\in [m]$ . For each subsequent block  $n\in \{2,\ldots ,N\}$ , the total loss in block  $n$  depends on the algorithm's choices made before observing  $\epsilon_{n}$ , namely,  $\{x_{(n - 2)m + 2},\ldots ,x_{(n - 1)m + 1}\}$ . Since  $\epsilon_{n}$  is equal to  $\pm 1$  with probability  $\frac{1}{2}$ , the expected loss of any algorithm in a block is equal to 0 and the total expected loss is also equal to 0.
Now we will show that the expected loss of the benchmark is at most
$$
- O \left(\sqrt {T m ^ {\frac {p + 2}{p}}}\right),
$$
where the expectation is with respect to the randomness in the choice of  $\{\epsilon_1,\dots ,\epsilon_N\}$ . We have
$$
\begin{array}{l} \mathbb {E} \left[ \min  _ {x \in \mathcal {X}} \sum_ {t = 1} ^ {T} \tilde {f} _ {t} (x) \right] = \mathbb {E} \left[ \min  _ {x \in \mathcal {X}} \sum_ {n = 2} ^ {N} \sum_ {t = (n - 1) m + 1} ^ {n m} \tilde {f} _ {t} (x) \right] \\ = \mathbb {E} \left[ \min  _ {x \in \mathcal {X}} \sum_ {n = 2} ^ {N} \sum_ {t = (n - 1) m + 1} ^ {n m} \epsilon_ {n} m ^ {\frac {1 - p}{p}} \times x \times \left(m - (t - (n - 1) m - 1)\right) \right]. \\ \end{array}
$$
The first equality follows from first summing over blocks and then summing over the rounds in that block. The second equality follows from the definitions of  $f_{t}$  above and of  $\tilde{f}_{t}$  (Definition 2.1). By the definition of  $\tilde{f}_{t}$ , the history  $h_{t}$  consists of  $m$  copies of  $x$  for  $t \geq m$ . By the definition of  $f_{t}$ , which sums over all past decisions until the first round of the current block, we have that within a block the sum first extends over  $m$  copies of  $x$  (in the first round of the block), then  $m - 1$  copies of  $x$  (in the second round of the block), and so on until the last round of the block. So, we have
$$
\begin{array}{l} \mathbb {E} \left[ \min  _ {x \in \mathcal {X}} \sum_ {t = 1} ^ {T} \tilde {f} _ {t} (x) \right] = \mathbb {E} \left[ \min  _ {x \in \mathcal {X}} \sum_ {n = 2} ^ {N} \sum_ {t = (n - 1) m + 1} ^ {n m} \epsilon_ {n} m ^ {\frac {1 - p}{p}} \times x \times (m - (t - (n - 1) m - 1)) \right] \\ = \mathbb {E} \left[ \min  _ {x \in \mathcal {X}} \sum_ {n = 2} ^ {N} \sum_ {k = 0} ^ {m - 1} \epsilon_ {n} m ^ {\frac {1 - p}{p}} \times x \times (m - k) \right] \\ = m ^ {\frac {1 - p}{p}} \frac {m ^ {2} + m}{2} \mathbb {E} \left[ \min  _ {x \in \mathcal {X}} \sum_ {n = 2} ^ {N} \epsilon_ {n} x \right] \\ = m ^ {\frac {1 - p}{p}} \frac {m ^ {2} + m}{2} \mathbb {E} \left[ \min  _ {x \in \{- 1, 1 \}} \sum_ {n = 2} ^ {N} \epsilon_ {n} x \right] \\ = m ^ {\frac {1 - p}{p}} \frac {m ^ {2} + m}{2} \mathbb {E} \left[ \frac {1}{2} \sum_ {n = 2} ^ {N} \epsilon_ {n} (- 1 + 1) - \frac {1}{2} \left| \sum_ {n = 2} ^ {N} \epsilon_ {n} (- 1 - 1) \right| \right], \\ \end{array}
$$
where the second-last equality follows because the minima of a linear function over an interval is at one of the endpoints and the last equality follows because  $\min \{x,y\} = \frac{1}{2} (x + y) - \frac{1}{2} |x - y|$ . Since  $\epsilon_{n}$  are Rademacher random variables equal to  $\pm 1$  with probability  $\frac{1}{2}$ , we can simplify the above as
$$
\begin{array}{l} \mathbb {E} \left[ \min  _ {x \in \mathcal {X}} \sum_ {t = 1} ^ {T} \tilde {f} _ {t} (x) \right] = m ^ {\frac {1 - p}{p}} \frac {m ^ {2} + m}{2} \mathbb {E} \left[ - \frac {1}{2} \left| \sum_ {n = 2} ^ {N} - 2 \epsilon_ {n} \right| \right] \\ = m ^ {\frac {1 - p}{p}} \frac {m ^ {2} + m}{2} \mathbb {E} \left[ - \left| \sum_ {n = 2} ^ {N} \epsilon_ {n} \right| \right] \\ = - m ^ {\frac {1 - p}{p}} \frac {m ^ {2} + m}{2} \mathbb {E} \left[ \left| \sum_ {n = 2} ^ {N} \epsilon_ {n} \right| \right] \\ \leq - m ^ {\frac {1 - p}{p}} \frac {m ^ {2} + m}{2} \sqrt {N}, \\ \end{array}
$$
where the last inequality follows from Khintchine's inequality. Using the definition  $N = \frac{T}{m}$ , we have
$$
\begin{array}{l} \mathbb {E} \left[ \min  _ {x \in \mathcal {X}} \sum_ {t = 1} ^ {T} \tilde {f} _ {t} (x) \right] \leq - m ^ {\frac {1 - p}{p}} \frac {m ^ {2} + m}{2} \sqrt {\frac {T}{m}} \\ = - \frac {1}{2} \sqrt {T} \left(m ^ {\frac {3}{2} + \frac {1 - p}{p}} + m ^ {\frac {1}{2} + \frac {1 - p}{p}}\right) \\ \leq - O \left(\sqrt {T} m ^ {\frac {3}{2} + \frac {1 - p}{p}}\right) \\ = - O \left(\sqrt {T} m ^ {\frac {p + 2}{2 p}}\right) \\ = - O \left(\sqrt {T m ^ {\frac {p + 2}{p}}}\right). \\ \end{array}
$$
$$
\mathbb {E} _ {\epsilon_ {1}, \ldots , \epsilon_ {N}} \left[ R _ {T} (\mathbf {F T R L}) \right] = \mathbb {E} \left[ \sum_ {t = 1} ^ {T} f _ {t} (h _ {t}) \right] - \mathbb {E} \left[ \min  _ {x \in \mathcal {X}} \sum_ {t = 1} ^ {T} \tilde {f} _ {t} (x) \right] \geq \Omega \left(\sqrt {T m ^ {\frac {p + 2}{p}}}\right).
$$
Now we provide a lower bound for the OCO with  $\rho$ -discounted infinite memory problem. For simplicity, we consider the case when the problem follows linear sequence dynamics with the 2-norm instead of a general  $p$ -norm.
Theorem D.2. Let  $\rho \in [\frac{1}{2}, 1)$ . There exists an instance of the online convex optimization with  $\rho$ -discounted infinite memory problem,  $(\mathcal{X}, \mathcal{H}, A_{\text{infinit}}, \rho, B_{\text{infinit}})$ , that follows linear sequence dynamics with the 2-norm and there exist  $L$ -Lipschitz continuous loss functions  $\{f_t : \mathcal{H} \to \mathbb{R}\}_{t=1}^T$  such that the regret of any algorithm  $\mathcal{A}$  satisfies
$$
R _ {T} (\mathcal {A}) \geq \Omega \left(\sqrt {T L ^ {2} (1 - \rho) ^ {- 2}}\right).
$$
The proof is very similar to that of Theorem D.1 with slight adjustments to account for a  $\rho$ -discounted infinite memory instead of a finite memory of constant size  $m$ .
Proof. Let  $\mathcal{X} = [-1,1]$  and consider an OCO with infinite memory problem with discount factor  $\rho$ ,  $(\mathcal{X},\mathcal{H},A_{\mathrm{infinite}},\rho,B_{\mathrm{infinite}})$ , that follows linear sequence dynamics with the 2-norm. For simplicity, assume that  $T$  is a multiple of  $(1 - \rho)^{-1}$  (otherwise, the same proof works but with slightly more tedious bookkeeping) and that  $L = 1$  (otherwise, multiply the functions  $f_{t}$  defined below by  $L$ ).
Define  $m = (1 - \rho)^{-1}$ . Divide the  $T$  rounds into  $N = \frac{T}{m}$  blocks of  $m$  rounds each. Sample  $N$  independent Rademacher random variables  $\{\epsilon_1, \ldots, \epsilon_N\}$ , where each  $\epsilon_i$  is equal to  $\pm 1$  with
probability  $\frac{1}{2}$ . Recall that  $h_t = (x_t, \rho x_{t-1}, \ldots, \rho^{t-1}x_1, 0, \ldots)$ . Define the loss functions  $\{f_t\}_{t=1}^T$  as follows. If  $t \leq m$ , let  $f_t = 0$ . Otherwise, let
$$
f _ {t} \left(h _ {t}\right) = \epsilon_ {\lceil \frac {t}{m} \rceil} m ^ {- \frac {1}{2}} \sum_ {k = 0} ^ {m - 1} \rho^ {k + t - m \lfloor \frac {t}{m} \rfloor - 1} x _ {m \lfloor \frac {t}{m} \rfloor + 1 - k}.
$$
The functions  $f_{t}$  are linear, so they are convex. In order to show that they satisfy Assumptions A3 and A4, it remains to show that they are 1-Lipschitz continuous. Let  $h = (x^{(1)},\rho x^{(2)},\ldots)$  and  $\tilde{h} = (\tilde{x}^{(1)},\rho \tilde{x}^{(2)},\dots)$  be arbitrary elements of  $\mathcal{H}$ . We have
$$
\begin{array}{l} \left| f _ {t} (h) - f _ {t} (\tilde {h}) \right| \\ \leq \left| \epsilon_ {\lceil \frac {t}{m} \rceil} m ^ {- \frac {1}{2}} \sum_ {k = 1} ^ {m} \rho^ {k - 1} \left(x ^ {(k)} - \tilde {x} ^ {(k)}\right) \right| \\ \leq m ^ {- \frac {1}{2}} \left| \sum_ {k = 1} ^ {m} \rho^ {k - 1} \left(x ^ {(k)} - \tilde {x} ^ {(k)}\right) \right| \quad \text {b e c a u s e} \epsilon_ {\lceil \frac {t}{m} \rceil} \in \{- 1, + 1 \} \\ \leq m ^ {- \frac {1}{2}} m ^ {\frac {1}{2}} \left(\sum_ {k = 1} ^ {m} \rho^ {2 (k - 1)} \left| x ^ {(k)} - \tilde {x} ^ {(k)} \right| ^ {2}\right) ^ {\frac {1}{2}} \quad \text {b y} \\ \leq \| h - \tilde {h} \| _ {\mathcal {H}}, \\ \end{array}
$$
where the last equality follows because the follows linear sequence dynamics with the 2-norm.
$$
- O \left(\sqrt {T (1 - \rho) ^ {- 2}}\right),
$$
$$
\begin{array}{l} \mathbb {E} \left[ \min _ {x \in \mathcal {X}} \sum_ {t = 1} ^ {T} \tilde {f} _ {t} (x) \right] = \mathbb {E} \left[ \min _ {x \in \mathcal {X}} \sum_ {n = 2} ^ {N} \sum_ {t = (n - 1) m + 1} ^ {n m} \tilde {f} _ {t} (x) \right] \\ = \mathbb {E} \left[ \min  _ {x \in \mathcal {X}} \sum_ {n = 2} ^ {N} \sum_ {t = (n - 1) m + 1} ^ {n m} \epsilon_ {n} m ^ {- \frac {1}{2}} \sum_ {k = 0} ^ {m - 1} \rho^ {k + t - (n - 1) m - 1} x \right] \\ = m ^ {- \frac {1}{2}} \mathbb {E} \left[ \min  _ {x \in \mathcal {X}} \sum_ {n = 2} ^ {N} \epsilon_ {n} x \sum_ {t = (n - 1) m + 1} ^ {n m} \rho^ {t - (n - 1) m - 1} \sum_ {k = 0} ^ {m - 1} \rho^ {k} \right] \\ = m ^ {- \frac {1}{2}} \frac {1 - \rho^ {m}}{1 - \rho} \mathbb {E} \left[ \min  _ {x \in \mathcal {X}} \sum_ {n = 2} ^ {N} \epsilon_ {n} x \sum_ {t = (n - 1) m + 1} ^ {n m} \rho^ {t - (n - 1) m - 1} \right] \\ = m ^ {- \frac {1}{2}} \left(\frac {1 - \rho^ {m}}{1 - \rho}\right) ^ {2} \underbrace {\mathbb {E} \left[ \underset {x \in \mathcal {X}} {\min} \sum_ {n = 2} ^ {N} \epsilon_ {n} x \right]} _ {(a)}. \\ \end{array}
$$
The term (a) above can be bounded above by  $-\sqrt{N}$  as in the proof of Theorem D.1 using Khintchine's inequality. Therefore, using that  $N = \frac{T}{m}$  and  $m = (1 - \rho)^{-1}$  we have
$$
\begin{array}{l} \mathbb {E} \left[ \min  _ {x \in \mathcal {X}} \sum_ {t = 1} ^ {T} \tilde {f} _ {t} (x) \right] \leq - m ^ {- \frac {1}{2}} \left(\frac {1 - \rho^ {m}}{1 - \rho}\right) ^ {2} \sqrt {N} \\ \leq - (1 - \rho) ^ {\frac {1}{2}} \left(\frac {1 - \rho^ {m}}{1 - \rho}\right) ^ {2} \sqrt {T (1 - \rho)} \\ = - \sqrt {T} \frac {\left(1 - \rho^ {m}\right) ^ {2}}{1 - \rho} \\ = - \sqrt {T (1 - \rho) ^ {- 2}} (1 - \rho^ {m}) ^ {2} \\ \leq - O \left(\sqrt {T (1 - \rho) ^ {- 2}}\right), \\ \end{array}
$$
where the last inequality follows from the assumption that  $\rho \in [\frac{1}{2}, 1)$  and the following argument:
$$
\begin{array}{l} \rho^ {m} = (1 - (1 - \rho)) ^ {m} = (1 - (1 - \rho)) ^ {\frac {1}{1 - \rho}} \leq \frac {1}{e} \\ \Rightarrow (1 - \rho^ {m}) \geq 1 - \frac {1}{e} \\ \Rightarrow (1 - \rho^ {m}) ^ {2} \geq \left(1 - \frac {1}{e}\right) ^ {2} \\ \Rightarrow - (1 - \rho^ {m}) ^ {2} \leq - \left(1 - \frac {1}{e}\right) ^ {2} \\ \end{array}
$$
E Online Linear Control
E.1 Formulation as OCO with Unbounded Memory
Now we formulate the online linear control problem in our framework by defining the decision space  $\mathcal{X}$ , the history space  $\mathcal{H}$ , and the linear operators  $A: \mathcal{H} \to \mathcal{H}$  and  $B: \mathcal{W} \to \mathcal{H}$ . Then, we define the functions  $f_t: \mathcal{H} \to \mathbb{R}$  in terms of  $c_t$  and finally, prove an upper bound on the policy regret. For notational convenience, let  $(M^{[s]})$  and  $(Y_k)$  denote the sequences  $(M^{[1]}, M^{[2]}, \ldots)$  and  $(Y_0, Y_1, \ldots)$  respectively.
Recall that we fix  $K \in \mathcal{K}$  to be an arbitrary  $(\kappa, \rho)$ -strongly stable linear controller and consider the disturbance-action controller policy class  $\mathcal{M}_K$  (Definition 4.1). For the rest of this paper let  $\widetilde{F} = F - GK$ . The first step is a change of variables with respect to the control inputs from linear controllers to DACs and the second is a corresponding change of variables for the state. Define the decision space  $\mathcal{X}$  as
$$
\mathcal {X} = \left\{M = \left(M ^ {[ s ]}\right): M ^ {[ s ]} \in \mathbb {R} ^ {d \times d}, \| M ^ {[ s ]} \| _ {2} \leq \kappa^ {4} \rho^ {s} \right\} \tag {10}
$$
$$
\| M \| _ {\mathcal {X}} = \sqrt {\sum_ {s = 1} ^ {\infty} \rho^ {- s} \| M ^ {[ s ]} \| _ {F} ^ {2}}. \tag {11}
$$
Define the history space  $\mathcal{H}$  to be the set consisting of sequences  $h = (Y_k)$ , where  $Y_0 \in \mathcal{X}$  and  $Y_k = \widetilde{F}^{k-1}GX_k$  for  $X_k \in \mathcal{X}, k \geq 1$  with
$$
\| h \| _ {\mathcal {H}} = \sqrt {\sum_ {k = 0} ^ {\infty} \xi_ {k} ^ {2} \| Y _ {k} \| _ {\mathcal {X}} ^ {2}}, \tag {12}
$$
where the weights  $(\xi_{k})$  are nonnegative real numbers defined as
$$
\xi = (1, 1, 1, \rho^ {- \frac {1}{2}}, \rho^ {- 1}, \rho^ {- \frac {3}{2}}, \dots). \tag {13}
$$
Define the linear operators  $A:\mathcal{H}\to \mathcal{H}$  and  $B:\mathcal{W}\rightarrow \mathcal{H}$  as
$$
A \left(\left(Y _ {0}, Y _ {1}, \dots\right)\right) = \left(0, G Y _ {0}, \widetilde {F} Y _ {1}, \widetilde {F} Y _ {2}, \dots\right) \quad \text {a n d} \quad B (M) = \left(M, 0, 0, \dots\right).
$$
Note that the problem follows linear sequence dynamics with the  $\xi$ -weighted 2-norm (Definition 2.3), where  $\xi$  is defined above in Eq. (13). The weights in the weighted norms on  $\mathcal{X}$  and  $\mathcal{H}$  increase exponentially. However, the norms  $\| M^{[s]}\| _F^2$  and  $\| \widetilde{F}^{k - 1}G\| _F^2$  decrease exponentially as well: by definition of  $M^{[s]}$  in Eq. (11) and the assumption on  $\widetilde{F} = F - GK$  for  $K\in \mathcal{K}$ . Leveraging this exponential decrease in  $\| M^{[s]}\| _F^2$  and  $\| \widetilde{F}^{k - 1}G\| _F^2$  to define exponentially increasing weights turns out to be crucial for deriving our regret bounds that are stronger than existing results. Furthermore, the choice to have  $\xi_p = 1$  for  $p\in \{1,2\}$  in addition to  $p = 0$  (as required by Definition 2.3) might seem like a small detail, but this also turns out to be crucial for avoiding unnecessary factors of  $\rho^{-1}$  in the regret bounds.
Recall that the loss functions in the online linear control problem are  $c_{t}(s_{t},u_{t})$ , where  $s_t$  and  $u_{t}$  are the state and control at round  $t$ . Now we will show how to construct the functions  $f_{t}:\mathcal{H}\to \mathbb{R}$  that correspond to  $c_{t}(s_{t},u_{t})$ . By definition, given a sequence of decisions  $(M_0,\dots ,M_t)$ , the history at the end of round  $t$  is given by
$$
h _ {t} = \left(M _ {t}, G M _ {t - 1}, \widetilde {F} G M _ {t - 2}, \dots , \widetilde {F} ^ {t - 1} G M _ {0}, 0, \dots\right).
$$
A simple inductive argument shows that the state and control in round  $t$  can be written as
$$
s _ {t} = \widetilde {F} ^ {t} s _ {0} + \sum_ {k = 0} ^ {t - 1} \sum_ {s = 1} ^ {k + 1} \widetilde {F} ^ {t - k - 1} G M _ {k} ^ {[ s ]} w _ {k - s} + w _ {t - 1}, \tag {14}
$$
$$
u _ {t} = - K s _ {t} + \sum_ {s = 1} ^ {t + 1} M _ {t} ^ {[ s ]} w _ {t - s}. \tag {15}
$$
Define the functions  $f_{t}:\mathcal{H}\to \mathbb{R}$  by  $f_{t}(h) = c_{t}(s,u)$ , where  $s$  and  $u$  are the state and control determined by the history as above. Note that  $f_{t}$  is parameterized by the past disturbances. Since the state and control are linear functions of the history and  $c_{t}$  is convex, this implies that  $f_{t}$  is convex.
With the above formulation and the fact that the class of disturbance-action controllers is a superset of the class of  $(\kappa, \rho)$ -strongly-stable linear controllers, we have that the policy regret for the online linear control problem is at most
$$
\sum_ {t = 0} ^ {T - 1} f _ {t} \left(h _ {t}\right) - \min  _ {M \in \mathcal {X}} \sum_ {t = 0} ^ {T - 1} \tilde {f} _ {t} (M).
$$
This completes the specification of the online convex optimization with unbounded memory problem,  $(\mathcal{X},\mathcal{H},A,B)$ , corresponding to the online linear control problem. Using Algorithm 1 and Theorem 3.1 we can upper bound the above by
$$
O \left(\sqrt {\frac {D}{\alpha} T L \tilde {L} H _ {2}}\right),
$$
where  $L$  is the Lipschitz constant of  $f_{t}$ ,  $\tilde{L}$  is the Lipschitz constant of  $\tilde{f}_{t}$ ,  $H_{2}$  is the 2-effective memory capacity, and  $D = \max_{x,\tilde{x} \in \mathcal{X}} |R(x) - R(\tilde{x})|$  for an  $\alpha$ -strongly-convex regularizer  $R: \mathcal{X} \to \mathbb{R}$ . In the next subsection we bound these quantities in terms of the problem parameters of the online linear control problem. We use  $O(\cdot)$  to hide absolute constants.
E.2 Regret Analysis
We use the following standard facts about matrix norms.
Lemma E.1. Let  $M, N \in \mathbb{R}^{d \times d}$ . Then,
1.  $\| M\| _2\leq \| M\| _F\leq \sqrt{d}\| M\| _2$
2.  $\| MN\| _F\leq \| M\| _2\| N\| _F$
Proof. Part 1 can be found in, for example, Golub and Loan [1996, Section 2.3.2]. Letting  $N_{j}$  denote the  $j$ -th column of  $N$ , part 2 follows from
$$
\| M N \| _ {F} ^ {2} = \sum_ {j = 1} ^ {d} \| M N _ {j} \| _ {2} ^ {2} \leq \| M \| _ {2} ^ {2} \sum_ {j = 1} ^ {d} \| N _ {j} \| _ {2} ^ {2} = \| M \| _ {2} ^ {2} \| N \| _ {F} ^ {2}.
$$
Lemma E.2. For  $s \geq 2$ , the operator norm  $\|A^s\|$  is bounded above as
$$
\left\| A ^ {s} \right\| \leq O \left(\kappa^ {4} \rho^ {\frac {s}{2}}\right).
$$
Proof. Recall the definition of  $\mathcal{H}$  and  $\| \cdot \|_{\mathcal{H}}$  (Eq. (12)). Let
$$
\left(Y _ {0}, Y _ {1}, \dots\right) = \left(Y _ {0}, G X _ {1}, \widetilde {F} G X _ {2}, \widetilde {F} ^ {2} G X _ {3}, \dots\right)
$$
be an element of  $\mathcal{H}$  with unit norm, i.e.,
$$
\sqrt {\sum_ {k = 0} ^ {\infty} \xi_ {k} ^ {2} \| Y _ {k} \| _ {\mathcal {X}} ^ {2}} = 1,
$$
where the weights  $(\xi_{k})$  are defined in Eq. (13). Note that  $\xi_p = 1$  for  $p = 0,1$  and  $\xi_p^2 = \rho^{-p + 2}$  for  $p = 2,3,\ldots$ . From the definition of the operator  $A$  and for  $s\geq 2$ , we have
$$
A ^ {s} \left(\left(Y _ {0}, Y _ {1}, \dots\right)\right) = (0, \dots , 0, \widetilde {F} ^ {s - 1} G Y _ {0}, \widetilde {F} ^ {s} G X _ {1}, \widetilde {F} ^ {s + 1} G X _ {2}, \dots).
$$
Now we bound  $\| A^s\|$  as follows. By definition of  $A^s$  and  $\| \cdot \|_{\mathcal{H}}$  (Eq. (12)), and part 2 of Lemma E.1, we have
$$
\begin{array}{l} \| A ^ {s} ((Y _ {0}, Y _ {1}, \dots)) \| = \sqrt {\rho^ {- s + 2} \| \widetilde {F} ^ {s - 1} G Y _ {0} \| _ {\mathcal {X}} ^ {2} + \sum_ {k = 1} ^ {\infty} \rho^ {- s - k + 2} \| \widetilde {F} ^ {s + k - 1} G X _ {k} \| _ {\mathcal {X}} ^ {2}} \\ \leq \sqrt {\rho^ {- s + 2} \| \widetilde {F} ^ {s - 1} G \| _ {2} ^ {2} \| Y _ {0} \| _ {\mathcal {X}} ^ {2} + \sum_ {k = 1} ^ {\infty} \rho^ {- s - k + 2} \| \widetilde {F} ^ {s - 1} \| _ {2} ^ {2} \| \widetilde {F} \| _ {2} ^ {2} \| \widetilde {F} ^ {k - 1} G X _ {k} \| _ {\mathcal {X}} ^ {2}} \\ \leq \rho^ {- \frac {s}{2}} \| \widetilde {F} ^ {s - 1} \| _ {2} \sqrt {\rho^ {2} \| G \| _ {2} ^ {2} \| Y _ {0} \| _ {\mathcal {X}} ^ {2} + \sum_ {k = 1} ^ {\infty} \rho^ {- k + 2} \| \widetilde {F} \| _ {2} ^ {2} \| \widetilde {F} ^ {k - 1} G X _ {k} \| _ {\mathcal {X}} ^ {2}} \\ = \rho^ {- \frac {s}{2}} \| \widetilde {F} ^ {s - 1} \| _ {2} \sqrt {\rho^ {2} \| G \| _ {2} ^ {2} \| Y _ {0} \| _ {\mathcal {X}} ^ {2} + \sum_ {k = 1} ^ {\infty} \rho^ {- k + 2} \| \widetilde {F} \| _ {2} ^ {2} \| Y _ {k} \| _ {\mathcal {X}} ^ {2}}. \\ \end{array}
$$
Using our assumptions that  $\| G\| _2\leq \kappa$  and  $\| \widetilde{F}\| _2\leq \kappa^2\rho$ , we have
$$
\begin{array}{l} \| A ^ {s} ((Y _ {0}, Y _ {1}, \dots)) \| \leq \rho^ {- \frac {s}{2}} \| \widetilde {F} ^ {s - 1} \| _ {2} \sqrt {\rho^ {2} \kappa^ {2} \| Y _ {0} \| _ {\mathcal {X}} ^ {2} + \sum_ {k = 1} ^ {\infty} \rho^ {- k + 2} \kappa^ {4} \rho^ {2} \| Y _ {k} \| _ {\mathcal {X}} ^ {2}} \\ \leq \rho^ {- \frac {s}{2}} \rho \kappa^ {2} \| \widetilde {F} ^ {s - 1} \| _ {2} \sqrt {\| Y _ {0} \| _ {\mathcal {X}} ^ {2} + \sum_ {k = 1} ^ {\infty} \rho^ {- k + 2} \| Y _ {k} \| _ {\mathcal {X}} ^ {2}} \\ \leq \rho^ {- \frac {s}{2}} \rho \kappa^ {2} \kappa^ {2} \rho^ {s - 1} \sqrt {\| Y _ {0} \| _ {\mathcal {X}} ^ {2} + \sum_ {k = 1} ^ {\infty} \rho^ {- k + 2} \| Y _ {k} \| _ {\mathcal {X}} ^ {2}} \\ = \kappa^ {4} \rho^ {\frac {8}{2}} \sqrt {\| Y _ {0} \| _ {\mathcal {X}} ^ {2} + \sum_ {k = 1} ^ {\infty} \rho^ {- k + 2} \| Y _ {k} \| _ {\mathcal {X}} ^ {2}}. \\ \end{array}
$$
Using  $\rho^{-1 + 2} = \rho < 1$  for  $k = 1$  in the above sum, the definition of  $(\xi_k)$ , and our assumption that  $(Y_0,Y_1\dots)$  has unit norm, we have
$$
\| A ^ {s} ((Y _ {0}, Y _ {1}, \dots)) \| \leq \kappa^ {4} \rho^ {\frac {s}{2}} \sqrt {\xi_ {0} ^ {2} \| Y _ {0} \| _ {\mathcal {X}} ^ {2} + \sum_ {k = 1} ^ {\infty} \xi_ {k} ^ {2} \| Y _ {k} \| _ {\mathcal {X}} ^ {2}} = \kappa^ {4} \rho^ {\frac {s}{2}}.
$$
Lemma E.3. The 2-effective memory capacity is bounded above as
$$
H _ {2} \leq O \left(\kappa^ {4} (1 - \rho) ^ {- \frac {3}{2}}\right).
$$
Proof. Using Lemma E.2 to bound  $\| A^k\|$  for  $k\geq 2$ , we have
$$
H _ {2} = \sqrt {\sum_ {k = 0} ^ {\infty} k ^ {2} \| A ^ {k} \| ^ {2}} \leq O \left(\sqrt {\sum_ {k = 2} ^ {\infty} k ^ {2} \kappa^ {8} \rho^ {k}}\right) \leq O \left(\kappa^ {4} (1 - \rho) ^ {- \frac {3}{2}}\right).
$$
Lemma E.4. Suppose  $R: \mathcal{X} \to \mathbb{R}$  is defined by  $R(M) = \frac{1}{2}\| M\|_{\mathcal{X}}^2$ . Then, it is 1-strongly-convex and  $D = \max_{M,\widetilde{M} \in \mathcal{X}} |R(M) - R(\widetilde{M})| \leq d\kappa^8 (1 - \rho)^{-1}$ .
Proof. Note that  $R$  is 1-strongly-convex by definition. Using part 1 of Lemma E.1 and the definition of  $\mathcal{X}$  (Eq. (10)), we have for all  $M, \widetilde{M} \in \mathcal{X}$ ,
$$
\begin{array}{l} D = \max  _ {M, \widetilde {M} \in \mathcal {X}} | R (M) - R (\widetilde {M}) | \\ = \max _ {M, \widetilde {M} \in \mathcal {X}} \left| \frac {1}{2} \| M \| _ {\mathcal {X}} ^ {2} - \frac {1}{2} \| \widetilde {M} \| _ {\mathcal {X}} ^ {2} \right| \\ \leq \max  _ {M \in \mathcal {X}} \| M \| _ {\mathcal {X}} ^ {2} \\ = \max  _ {M \in \mathcal {X}} \sum_ {s = 1} ^ {\infty} \rho^ {- s} \| M ^ {[ s ]} \| _ {F} ^ {2} \quad \text {b y} (11) \\ \leq \max  _ {M \in \mathcal {X}} \sum_ {s = 1} ^ {\infty} \rho^ {- s} d \| M ^ {[ s ]} \| _ {2} ^ {2} \quad \text {b y} \\ \leq \sum_ {s = 1} ^ {\infty} \rho^ {- s} d \kappa^ {8} \rho^ {2 s} (10) \\ \leq d \kappa^ {8} (1 - \rho) ^ {- 1}. \\ \end{array}
$$
Lemma E.5. We can bound the norm of the state and control at time  $t$  as
$$
\max  \left\{\| s _ {t} \| _ {2}, \| u _ {t} \| _ {2} \right\} \leq D _ {\mathcal {X}} = O \left(W \kappa^ {8} (1 - \rho) ^ {- 2}\right).
$$
Proof. We can bound the norm of  $s_t$  and  $u_t$  using Eqs. (14) and (15) as
$$
\begin{array}{l} \| s _ {t} \| _ {2} \leq \left\| \widetilde {F} ^ {t} s _ {0} + \sum_ {k = 0} ^ {t - 1} \sum_ {s = 1} ^ {k + 1} \widetilde {F} ^ {t - k - 1} G M _ {k} ^ {[ s ]} w _ {k - s} + w _ {t - 1} \right\| _ {2} \\ \leq \kappa^ {2} \rho^ {t} + W + \sum_ {k = 0} ^ {t - 1} \sum_ {s = 1} ^ {k + 1} \kappa^ {2} \rho^ {t - k - 1} \kappa \kappa^ {4} \rho^ {s} W \\ \leq \kappa^ {2} + W + W \kappa^ {7} (1 - \rho) ^ {- 2} \\ \leq O \left(W \kappa^ {7} (1 - \rho) ^ {- 2}\right). \\ \end{array}
$$
$$
\begin{array}{l} \left\| u _ {t} \right\| _ {2} \leq \left\| K s _ {t} + \sum_ {s = 1} ^ {t + 1} M _ {t} ^ {[ s ]} w _ {t - s} \right\| _ {2} \\ \leq O \left(W \kappa^ {8} (1 - \rho) ^ {- 2}\right) + \sum_ {s = 1} ^ {t + 1} W \kappa^ {4} \rho^ {s} \\ \leq O \left(W \kappa^ {8} (1 - \rho) ^ {- 2}\right). \\ \end{array}
$$
Above, we used the assumptions that  $\kappa, W \geq 1$ . This completes the proof.