SlowGuess/NeurIPS_2023_OCR huggingface dataset:
1: Initialize Epoch length  $E$ , Classifiers  $\mathbf{w}_{1:K}^{0} = (0, \dots, 0)$ , margin parameter  $\gamma > 0$ , learning rate  $\eta > 0$   
2: for  $\tau = 1, 2, \dots, T / E$  do (% iterate over epochs)  
3:  $(\widehat{\Theta}_{\tau, i})_{1 \leq i \leq K}$ ,  $\widehat{g}_{\tau} \gets \text{ERMORACLE}(\bar{\mathbf{x}}_{1: \tau E}, \mathbf{y}_{1: \tau E})$   
4:  $\widehat{g}_{\tau} \gets \text{Reorder}(\widehat{g}_{\tau}, (\widehat{\Theta}_{\tau, i})_{1 \leq i \leq K}, (\widehat{\Theta}_{\tau - 1, i})_{1 \leq i \leq K})$   
5:  $\mathbf{w}_{1:K}^{\tau E} \gets \text{OGD}(\mathbf{w}_{1:K}^{(\tau - 1)E}, \bar{\mathbf{x}}_{(\tau - 1)E: \tau E}; \widehat{g}_{\tau}, \gamma, \eta)$   
6: Let  $\tilde{g}_{\tau}(\bar{\mathbf{x}}) = \arg \max_{1 \leq i \leq K} \langle \mathbf{w}_i^{\tau E}, \bar{\mathbf{x}} \rangle$  denote classifier induced by  $\mathbf{w}_{1:K}^{\tau E}$ .  
7: for  $t = \tau E, \dots, (\tau + 1)E - 1$  do  
8: Receive  $\bar{\mathbf{x}}_t$ , and predict  $\hat{\mathbf{y}}_t = \widehat{\Theta}_{\tau, \tilde{g}_{\tau}(\bar{\mathbf{x}}_t)} \cdot \bar{\mathbf{x}}_t$   
9: Receive  $\mathbf{y}_t$
Finally, we assume that the true affine parameters are well-separated:
Assumption 5. There is some  $\Delta_{\mathrm{sep}} > 0$  such that for all  $1 \leq i < j \leq K$ , it holds that  $\| \Theta_i^\star - \Theta_j^\star \|_{\mathrm{F}} \geq \Delta_{\mathrm{sep}}$ .
Note that Assumption 5 is, in a sense, generic, as explained in Remark 4. We defer the formal setting of one-step prediction and simulation regret to Appendix G.1 and section 3.1.
3 Algorithm and Guarantees
We propose Algorithm 1, an oracle-efficient protocol with provably sublinear regret and polynomial dependence on all problem parameters. Algorithm 1 runs in epochs: at the beginning of epoch  $\tau$ , the learner calls ERMORACLE on the past data to produce a linear classifier  $\widehat{g}_{\tau}$  and estimates of the regression matrices in each mode,  $\widehat{\Theta}_{\tau,i}$ . A major challenge is identifying the same affine regions between epochs. To this end, the learner permutes the labels of  $\widehat{g}_{\tau}$  to preserve consistency in labels across epochs, as explained informally in Section 4.2 and in greater detail in Appendix E.
The learner then runs online gradient descent (Algorithm 2) on the hinge loss to produce a modified classifier  $\widetilde{g}_{\tau}$ ; finally, throughout the epoch, the learner uses  $\widehat{\Theta}_{\tau,i}$  and  $\widetilde{g}_{\tau}$  to predict  $\widehat{\mathbf{y}}_t$  before repeating the process in the next epoch. The reason for using a secondary algorithm to transform  $\widehat{g}_{\tau}$  into  $\widetilde{g}_{\tau}$  is to enforce stability of the predictor across epochs, which is necessary to bound the regret. Again, we suppose that Assumption 1 holds throughout.
Theorem 3 (Regret Bound). Suppose we are in the setting of online PWA regression (2.1) and Assumptions 2-5 hold. Then, if the parameters  $E$ ,  $\gamma$ ,  $\eta$  are set as in (F.6), it holds that with probability at least  $1 - \delta$ ,  $\mathrm{Reg}_T \leq \mathrm{poly}(\overline{\mathrm{par}}, \log(1/\delta)) \cdot T^{35/36} + \varepsilon_{\mathrm{orac}} \cdot T + K^2 \cdot \sum_{t=1}^{T} ||\pmb{\delta}_t||$ , for  $\overline{\mathrm{par}} := \max \{d, m, \sigma_{\mathrm{dir}}^{-1}, \Delta_{\mathrm{sep}}^{-1}, B, R, K, \nu\}$ , where the polynomial dependence is given in (F.7).
Remark 1. While Theorem 3 requires that we are correctly setting the parameters of the algorithm, aggregation algorithms allow us to tune these parameters in an online fashion (see, e.g. [30]).
Remark 2. The misspecification errors  $\delta_t$  can be indicators corresponding to the  $\bar{\mathbf{x}}_t$  being in small, rarely visited regions; thus, we could run an underspecified ERMORACLE, with  $K' \ll K$  modes and recover a regret bound depending only on frequently visited modes.
When  $\varepsilon_{\mathrm{crp}} = \varepsilon_{\mathrm{orac}} = 0$ , we obtain exactly the polynomial, no-regret rates promised in the introduction. Along the way to proving our regret bound, we establish that ERMORACLE correctly recovers the linear parameters within frequently visited modes.
Theorem 4 (Parameter Recovery). Suppose that Assumptions 2-4 hold and let  $\left\{\widehat{\Theta}_i|i\in [K]\right\}$ ,  $\widehat{g}$  denote the output of ERMORACLE  $(\bar{\mathbf{x}}_{1:T},\mathbf{y}_{1:T})$  and  $I_{ij}(\widehat{g})$  denote the set of times  $t$  such that  $\widehat{g} (\bar{\mathbf{x}}_t) = i$  and  $g_{\star}(\bar{\mathbf{x}}_t) = j$ . Then with probability at least  $1 - \delta$ , for all  $1\leq i,j\leq K$  such that  $|I_{ij}(\widehat{g})|\geq$  poly  $\left(\overline{\mathrm{par}},\log \left(\frac{1}{\delta}\right)\right)\cdot T^{1 - \Omega (1)}$  it holds that  $\| \widehat{\Theta}_i - \Theta_j^*\| _F^2$  is bounded by  $\frac{\sqrt{T}}{|I_{ij}(\widehat{g})|}\cdot \mathrm{poly}\left(\overline{\mathrm{par}},\log \left(\frac{1}{\delta}\right)\right)$ .
Above we have given the result with the assumption that  $\varepsilon_{\mathrm{crp}} = \varepsilon_{\mathrm{orac}} = 0$  for the sake of presentation; in the formal statement, the general case is stated. Theorem 4 implies that directional smoothness
is sufficient to guarantee parameter recovery at a parametric rate, in marked distinction to most adversarial learning regimes. Before providing more detail on the algorithm and its analysis in Section 4, we discuss an application to  $H$ -step prediction. An application to one-step prediction with learner-provided controls can be found in Appendix G.1
3.1 Guarantees for Simulation Regret
We now describe an application of our results to  $H$ -step prediction; more formal description is detailed in Appendix G.3. The learning process occurs in episodes  $t = 1,2,\dots,T$ , consisting of steps  $h = 1,2,\dots,H$ . In each episode, an external planner provides the learner a policy  $\pi_t$ . For simplicity, we assume that  $\pi_t = (\mathbf{u}_{t,1},\ldots ,\mathbf{u}_{t,H})$  is open loop (state-independent) and stochastic, minimicking situations such as model predictive control; extensions are mentioned in Appendix G.6. With each episode, the true dynamics are given by
$$
\mathbf {z} _ {t, h + 1} = \mathbf {A} _ {i _ {t, h}} ^ {\star} \mathbf {z} _ {t} + \mathbf {B} _ {i _ {t, h}} ^ {\star} \mathbf {u} _ {t, h} + \mathbf {m} _ {i _ {t, h}} ^ {\star} + \mathbf {e} _ {t, h}, \quad i _ {t, h} = g ^ {\star} \left(\mathbf {z} _ {t, h}, \mathbf {u} _ {t, h}\right). \tag {3.1}
$$
We assume that the process noise  $\mathbf{e}_{t,h}$  are independently drawn from a known distribution  $\mathcal{D}$ , and that  $\mathbf{z}_{t,1}$  is sampled from an arbitrary  $\sigma_{\mathrm{dir}}$ -smooth distribution. At the start of episode  $t$ , the learner then produces estimates  $\hat{\mathbf{A}}_t, \hat{\mathbf{B}}_t, \hat{g}_t$  and simulates the plan  $\pi_t$  using the plug in estimate of the dynamics, i.e.,  $\hat{\mathbf{z}}_{t,h+1} = \hat{\mathbf{A}}_{t,\hat{i}_{t,h}}\hat{\mathbf{z}}_{t,h} + \hat{\mathbf{B}}_{t,\hat{i}_{t,h}}\mathbf{u}_{t,h} + \hat{\mathbf{m}}_{t,\hat{i}_{t,h}}\hat{\mathbf{e}}_{t,h}, \hat{i}_{t,h} = \widehat{g}_t(\hat{\mathbf{z}}_{t,h}, \hat{\mathbf{u}}_{t,h})$ , where  $\hat{\mathbf{e}}_{t,h} \stackrel{\mathrm{i.i.d.}}{\sim} \mathcal{D}$ , and  $\hat{\mathbf{u}}_{t,h}$  are an i.i.d. draw from the stochastic, open-loop policy  $\pi_t$ . Because the simulated and true processes use a different noise sequence, we consider the following notion of error, which measures the distance between the two distributions induced by the dynamics, as opposed to the regret, which measure the distance between the realizations thereof. We choose the Wasserstein metric  $\mathcal{W}_2$ , as it upper bounds the difference in expected value of any Lipschitz reward function between the true and simulated trajectories (see Appendix G.3 for a formal definition and more explanation).
Definition 5 (Simulation Regret). Let  $\mathcal{W}_2(\cdot, \cdot)$  denote the  $L_2$ -Wasserstein distance, define the concatenations  $\mathbf{x}_{t,h} = (\mathbf{z}_{t,h}, \mathbf{u}_{t,h})$  and  $\hat{\mathbf{x}}_{t,h} = (\hat{\mathbf{z}}_{t,h}, \hat{\mathbf{u}}_{t,h})$ , and let  $\bar{\mathcal{F}}_t$  be the filtration generated by  $\{\mathbf{x}_{s,h}\}_{1 \leq s \leq t, 1 \leq h \leq H}$ . We define the  $T$ -episode,  $H$ -step simulation regret as  $\mathrm{SimReg}_{T,H} := \sum_{t=1}^{T} \mathcal{W}_2(\mathbf{x}_{t,1:H}, \hat{\mathbf{x}}_{t,1:H} \mid \bar{\mathcal{F}}_{t-1})^2$ .
Our goal is to achieve  $\mathrm{SimReg}_{T,H} \lesssim \mathrm{poly}(H) \cdot T^{1 - \Omega(1)}$ , but polynomial-in- $H$  dependence may be challenging for arbitrary open-loop policies and unstable dynamics of the pieces. Thus, in the interest of simplicity, we decouple the problems of linear stability from the challenge of error compounding due to discontinuity of the dynamics by adopting the following strong assumption.
Assumption 6. There exists a known, positive definite Lyapunov matrix  $\mathbf{P} \in \mathbb{R}^{d_z \times d_z}$  that satisfies  $(\mathbf{A}_i^{\star})^\top \mathbf{P}(\mathbf{A}^{\star})_i \preceq \mathbf{P}$  for all modes  $i \in [K]$ .
Using a minor modification of Algorithm 1, detailed in Algorithm 4, we show that we can get vanishing simulation regret, summarized in the following result:
Theorem 6. Suppose that we are in the setting of (3.1) and that Assumptions 2-5 and 6 hold. If we run a variant of Algorithm 1 (see Algorithm 4 in Appendix G.4), then with probability at least  $1 - \delta$ , it holds that  $\mathrm{SimReg}_{T,H} \leq \mathrm{poly}(\overline{\mathrm{par}}, H) \cdot T^{35/36}$ .
The proof of Theorem 6 with the exact polynomial dependence on the parameters can be found in Appendix G.3 and rests on a lemma showing that under Assumption 6, simulation regret at a given time  $t$  can be bounded as  $\text{poly}(H)$  multiplied by the maximum one-step prediction error for times  $t, t + 1, \ldots, t + H$ , which is bounded in Theorem 3. We provide an exact recovery guarantee in the case  $H = 1$  and without requiring Assumption 6 in Appendix G.1.
In this section, we present a sketch of the proof of Theorem 15. There are two primary sources of regret: that which is due to poorly estimating the linear parameters within a mode and that which is due to misclassification of the mode. We analyze each source separately, beginning with the parameter recovery result of Theorem 4.
Algorithm 2 Online Gradient Descent (Single Epoch)
1: Initialize Data  $\bar{\mathbf{x}}_{1:E}$ , classifiers  $\mathbf{w}_{1:K}^{0}$ , margin parameter  $\gamma > 0$ , learning rate  $\eta > 0$ , classifier  $\widehat{g}: \mathcal{X} \to [K]$
2: for  $s = 1,2,\ldots ,E$  do % defined in (4.1)
3: Receive  $\bar{\mathbf{x}}_i$  and update  $\mathbf{w}_{1:K}^s\gets \Pi_{(\mathcal{B}^{d - 1})\times K}\left(\mathbf{w}_{1:K}^{s - 1} - \eta \nabla \widetilde{\ell}_{\gamma ,i,\widehat{g}}(\mathbf{w}^{s - 1})\right)$
4: Return  $\mathbf{w}_{1:K}^{m}$
4.1 Proof Sketch of Theorem 4
We break the proof into three parts: first, we show that the regret of  $\widehat{g}$  and the  $\widehat{\Theta}_i$ , when restricted to times  $t \in I_{ij}(\widehat{g})$ , is small; second, we relate the prediction error  $\mathcal{Q}_{ij}(\widehat{g}) = \sum_{t \in I_{ij}(\widehat{g})} \|( \widehat{\Theta}_i - \Theta_j^\star) \bar{\mathbf{x}}_t \|^2$  to the regret on  $I_{ij}(\widehat{g})$ ; third, demonstrate that  $\Sigma_{ij}(\widehat{g}) = \sum_{t \in I_{ij}(\widehat{g})} \bar{\mathbf{x}}_t \bar{\mathbf{x}}_t^T$  has minimal eigenvalue bounded below by some constant with high probability. If all three of these claims hold, then, noting that  $\| \widehat{\Theta}_i - \Theta_j^\star \|_{\mathrm{F}}^2 \leq \| \Sigma_{ij}(\widehat{g})^{-1} \|_{\mathrm{op}} \cdot \sum_{t \in I_{ij}(\widehat{g})} \| (\widehat{\Theta}_i - \Theta_j^\star) \bar{\mathbf{x}}_t \|^2$ ,
we conclude. Each of these claims requires significant technical effort in its own right. We introduce disagreement covers, a stronger analogue of  $\varepsilon$ -nets adapted to our problem and allowing us to turn statements proven for a single  $g \in \mathcal{G}$  into ones uniform over  $\mathcal{G}$ . The first step is to bound the size of disagreement covers for the class  $\mathcal{G}$  of interest; we then prove each of the three claims for some fixed  $g \in \mathcal{G}$  and lift the result to apply to the data-dependent  $\widehat{g}$ . This is done in Appendix D.1.
We now turn to our three claims. For the first, proved in Lemma D.15, we introduce the excess risk  $\mathcal{R}_{ij}(g,\Theta) = \sum_{t\in I_{ij}(g)}||\Theta_i\bar{\mathbf{x}}_t - \mathbf{y}_t||^2 -||\mathbf{e}_t + \delta_t||^2$ , for each pair  $(i,j)$ , and note that the cumulative empirical excess risk  $\sum_{i,j}\mathcal{R}_{ij}(\widehat{g},\widehat{\Theta})$  of predicting using  $\widehat{g}$  and  $\widehat{\Theta}$  returned by the ERM oracle is bounded by  $\varepsilon_{\mathrm{orac}}$  because  $\mathcal{R}_{ij}(g_\star,\Theta_\star) = 0$  due to Equation (2.1). Thus, showing that  $\mathcal{R}_{ij}(\widehat{g},\widehat{\Theta})$  is small can be done by showing that for no  $i',j'$  is  $\mathcal{R}_{i'j'}(\widehat{g},\widehat{\Theta}) \ll 0$ . This can be accomplished by a concentration argument for a single  $g,\Theta_0$ , coupled with a covering argument using our notion of disagreement covers to boost the result to one uniform in  $\mathcal{G}$ .
For the second claim, i.e., that  $\mathcal{Q}_{ij}(\widehat{g})$  is controlled by  $\mathcal{R}_{ij}(\widehat{g},\widehat{\Theta})$ , we provide a proof in Lemma D.14. For a fixed  $g\in \mathcal{G}$ , we decompose  $\mathcal{R}_{ij}(g,\widehat{\Theta})$  into  $\mathcal{Q}_{ij}(g)$  and an error term. We then use a generalization of the self-normalized martingale inequality from Abbasi-Yadkori et al. [2] to control the error in terms of  $\mathcal{Q}_{ij}(g)$ , providing a self-bounded inequality. Finally, we rearrange this inequality and apply a union bound on a disagreement cover to boost the result to one uniform in  $\mathcal{G}$  and  $\Theta$ .
To prove the third claim, that  $\Sigma_{ij}(\widehat{g})$  has singular values uniformly bounded from below, we split our argument into two parts. First, in Appendix D.2, we assume the following sliced-small ball condition on the data, i.e., for some  $\zeta, \rho > 0$ , it holds that  $\mathbb{P}(\langle \bar{\mathbf{x}}_t, \mathbf{w} \rangle^2 \geq c \cdot \zeta^2 | \mathcal{F}_t, t \in I_{ij}(\widehat{g})) \geq c \cdot \rho$ . Given the above condition, we establish a high probability lower bound on  $\Sigma_{ij}(g)$  for fixed  $g$  using a self-normalized martingale inequality, using analysis that may be of independent interest. We then again apply a union bound on a disagreement cover to lift this result to be uniform in  $\mathcal{G}$ . Finally, in Appendix D.3, we provide bounds on  $\zeta, \rho$  using directional smoothness and Markov's inequality.
4.2 Mode Prediction
In this section, we address the second source of error, mode misclassification. The primary challenge is that directional smoothness, as opposed to independent data, is insufficiently strong to guarantee that  $\widehat{g}_{\tau}$  generalizes well to unseen data in epoch  $\tau + 1$ . We take inspiration from online learning and stabilize our algorithm across epochs by modifying the classifier  $\widehat{g}_{\tau}$ . There are two challenges in ensuring good performance of our online classifier. First, the dynamics described in (2.1) are only identifiable up to a permutation of the labels. Thus, in order to enforce stability across epochs, we need to enforce consistency of labelling across epochs. This task is made more difficult by the fact that different modes may be combined or split up across epochs due to the black box nature of ERMORACLE. We solve this by appealing to a subroutine, Reorder, described in Algorithm 3 in Appendix E, which combines modes that are sufficiently large and have sufficiently similar parameters and then relabels the modes so that similar nominal clusters persist across epochs.
The second challenge is enforcing stability of our online classifier. We do this by introducing a surrogate loss  $\widetilde{\ell}_{\gamma, t, \widehat{g}}$ , the multi-class hinge loss with parameter  $\gamma > 0^2$ . Formally, let  $\mathbf{w}_{1:K} = (\mathbf{w}_1, \ldots, \mathbf{w}_K)$  for  $\mathbf{w}_i \in \mathcal{B}^{d-1}$ , and for  $g \in \mathcal{G}$ , define the  $(1/\gamma)$ -Lipschitz and convex surrogate loss:
$$
\widetilde {\ell} _ {\gamma , t, g} \left(\mathbf {w} _ {1: K}\right) = \max  \left(0, \max  _ {j \neq g \left(\bar {\mathbf {x}} _ {t}\right)} \left(1 - \frac {\left\langle \mathbf {w} _ {g \left(\bar {\mathbf {x}} _ {t}\right)} - \mathbf {w} _ {j} , \bar {\mathbf {x}} _ {t} \right\rangle}{\gamma}\right)\right). \tag {4.1}
$$
In OGD (see Algorithm 2), we modify the output of ERMORACLE to construct a classifier  $\widetilde{g}_{\tau}$  by running lazy online gradient descent on this surrogate loss, using the reordered labels given by the output of ERMORACLE evaluated on the previous epoch. By invoking OGD, we show that our mode classifier is sufficiently stable to ensure low regret, as stated informally in the following result.
Theorem 7 (Mode Prediction, Informal Statement of Theorem 13). Suppose that Assumptions 2-5 hold and let  $\mathcal{E}_t^{\mathrm{mode}}$  be the event that the learner, invoking Algorithm 1 with correctly tuned parameters, misclassifies the mode of  $\bar{\mathbf{x}}_t$ . Then, with probability at least  $1 - \delta$ , it holds that  $\sum_{t=1}^{T} \mathbb{I}\{\mathcal{E}_t^{\mathrm{mode}}\} \leq \text{poly}(\overline{\text{par}}, \log\left(\frac{1}{\delta}\right)) \cdot T^{1-\Omega(1)}$ .
Proof Sketch. We begin by addressing the lack of identifiability of the modes. For the purposes of analysis, for each epoch  $\tau$ , we introduce the function  $\pi_{\tau} : [K] \to [K]$  such that  $\pi_{\tau}(i) = \operatorname*{argmin}_{1 \leq j \leq K} \| \widehat{\Theta}_{\tau, i} - \Theta_{j}^{\star} \|_{\mathrm{F}}$ . In words,  $\pi_{\tau}(i)$  is the mode  $j$  according to the ground truth whose parameters are closest to those estimated by ERMORACLE. We let  $\mathcal{E}_t^{\mathrm{mode}}$  denote the event of misclassifying the mode, i.e. the event that  $\pi_{\tau}(\widetilde{g}_{\tau}(\bar{\mathbf{x}}_t)) \neq g_{\star}(\bar{\mathbf{x}}_t)$ . We can then decompose the misclassification as  $\mathcal{E}_t^{\mathrm{mode}} \subset \{\pi_{\tau + 1}(\widetilde{g}_{\tau}(\bar{\mathbf{x}}_t)) \neq g_{\star}(\bar{\mathbf{x}}_t)\} \cup \{\pi_{\tau}(\widetilde{g}_{\tau}(\bar{\mathbf{x}}_t)) \neq \pi_{\tau + 1}(\widetilde{g}_{\tau}(\bar{\mathbf{x}}_t))\}$ . We call the first event look-ahead classification error and the second permutation disagreement error. To bound the first of these, we further decompose the look-ahead classification error event:  $\{\pi_{\tau + 1}(\widetilde{g}_{\tau}(\bar{\mathbf{x}}_t)) \neq g_{\star}(\bar{\mathbf{x}}_t)\} \subset \{\pi_{\tau + 1}(\widehat{g}_{\tau + 1}(\bar{\mathbf{x}}_t)) \neq g_{\star}(\bar{\mathbf{x}}_t)\} \cup \{\widetilde{g}_{\tau}(\bar{\mathbf{x}}_t) \neq \widehat{g}_{\tau + 1}(\bar{\mathbf{x}}_t)\}$ . Controlling the first of these events is proved as a corollary of the bound on the permutation disagreement error and we defer its discussion for the sake of simplicity. To bound the probability of the second event, upper bound the indicator loss by  $\widetilde{\ell}_{\gamma, t, \widehat{g}_{\tau + 1}}$  and apply standard online learning techniques to show that regret of OGD on the  $(1/\gamma)$ -Lipschitz and convex surrogate loss is small. We then use smoothness to show that the optimal comparator with respect to the surrogate loss does not experience large regret with respect to the indicator loss, which in turn has a comparator experiencing zero loss due to the well-specified nature of  $g_{\star}$ . Thus, putting everything together, this provides bound on the look-ahead classification error, with full details presented in Appendix E.1.
Bounding the permutation disagreement error is what necessitates the gap assumption. We show that if there are sufficiently many data points that  $\widehat{g}_{\tau}$  assigns to a given mode, with the threshold defined in terms of  $\Delta_{\mathrm{sep}}$  and other parameters of the problem, then with high probability the cluster becomes stable in the sense that  $\widehat{\Theta}_{\tau,i} \approx \widehat{\Theta}_{\tau+1,i}$ . This result is proved using the fact that if  $|I_{i,j}(\widehat{g}_{\tau})|$  is large enough, then the parameter recovery results from the previous section tell us that  $\|\widehat{\Theta}_i - \widehat{\Theta}_j\|_{\mathrm{F}}$  is small. Using the triangle inequality and the fact that  $\|\Theta_j^* - \Theta_{j'}^*\|_{\mathrm{F}} > \Delta_{\mathrm{sep}}$  for  $j' \neq j$  ensures that  $j$  is unique in satisfying  $\|\widehat{\Theta}_{\tau,i} - \Theta_j^*\|_{\mathrm{F}} \ll 1$ . A similar argument applies to epoch  $\tau + 1$ , and thus we can identify  $\widehat{\Theta}_{\tau,i}$  with some  $\widehat{\Theta}_{\tau+1,i'}$  by these matrices being sufficiently close in Frobenius norm. Algorithm 3 takes advantage of exactly this property and permutes the labels across epochs in order to maintain consistency and control the permutation disagreement error; full details can be found in Appendix E.2. Combining this argument with the bound on the look-ahead classification error suffices to control the online classification component of the regret.
We have given an efficient online learning algorithm for prediction in piecewise affine (PWA) systems. Our results are the first foray into the study of such systems, and a number of outstanding questions remain: Does directional smoothness facilitate low regret for planning and control, in addition to for simulation and prediction? If the systems are piecewise-affine but forced to be continuous on their boundaries, is there an oracle efficient algorithm which suffers low regret without assuming directional
2We use the hinge loss because it is convex, Lipschitz, and its relationship to the indicator loss is particularly convenient under directional smoothness. Other convex, Lipschitz surrogates provide similar guarantees.
smoothness? Can learning in piecewise affine systems be obtained under partial observation of the system state? Can the dependence of regret on horizon be improved? We hope that our work initiates investigation of the above as the field continues to expand its understanding beyond the linear regime.
AB acknowledges support from the National Science Foundation Graduate Research Fellowship under Grant No. 1122374 as well as support from ONR under grant N00014-20-1-2336 and DOE under grant DE-SC0022199. MS acknowledges support from Amazon.com Services LLC grant; PO# 2D-06310236. We also acknowledge Zak Mhammedi, Terry H.J. Suh and Tao Pang for their helpful comments.