SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Verónica Álvarez $^{1}$  Santiago Mazuelas $^{1,2}$  Jose A. Lozano $^{1,3}$
<sup>1</sup>Basque Center of Applied Mathematics (BCAM)
$^{2}$ IKERBASQUE-Basque Foundation for Science  $^{3}$ University of the Basque Country UPV/EHU
{valvarez, smazuelas, jlozano}@bcamath.org
For a sequence of classification tasks that arrive over time, it is common that tasks are evolving in the sense that consecutive tasks often have a higher similarity. The incremental learning of a growing sequence of tasks holds promise to enable accurate classification even with few samples per task by leveraging information from all the tasks in the sequence (forward and backward learning). However, existing techniques developed for continual learning and concept drift adaptation are either designed for tasks with time-independent similarities or only aim to learn the last task in the sequence. This paper presents incremental minimax risk classifiers (IMRCs) that effectively exploit forward and backward learning and account for evolving tasks. In addition, we analytically characterize the performance improvement provided by forward and backward learning in terms of the tasks' expected quadratic change and the number of tasks. The experimental evaluation shows that IMRCs can result in a significant performance improvement, especially for reduced sample sizes.
In practical scenarios, it is often of interest to incrementally learn a growing sequence of classification problems (tasks) that arrive over time. In such a sequence, it is common that tasks are evolving in the sense that consecutive tasks often have a higher similarity. Examples of evolving tasks are the classification of portraits from different time periods [1] and the classification of spam emails over time [2]; in these problems, the similarity between consecutive tasks (portraits of consecutive time periods and emails from consecutive years) is significantly higher (see Figure 1). The incremental learning of a growing sequence of tasks holds promise to significantly improve performance by leveraging information from different tasks. Specifically, at each time step, information from preceding tasks can be used to improve the performance of the last task (forward learning) and, reciprocally, the information from the last task can be used to improve the performance of the preceding tasks (backward learning) [3-5]. Such transfer of information can enable accurate classification even in cases with reduced sample sizes, thus significantly increasing the effective sample size (ESS) of each task. However, exploiting the benefits of forward and backward learning is challenging due to the continuous arrival of samples from tasks characterized by different underlying distributions [6-8].
Techniques developed for concept drift adaptation (aka learning in a dynamic scenario) [9-16] are designed for evolving tasks but only aim to learn the last task in the sequence. In particular, methods based on learning rates learn the last task by slightly updating the classification rule for the preceding task [12, 13]; and methods based on sliding windows learn the last task by using a set of stored samples from the most recent preceding tasks [14, 15]. In concept drift adaptation, at each time
Figure 1: Tasks that arrive over time are characterized by different underlying distributions and consecutive tasks often have a higher similarity. The black line represents the evolution of the underlying distributions that characterize different tasks. IMRCs minimize the worst-case error probability over uncertainty sets  $\mathcal{U}_i$  that can include the underlying distribution  $p_i$ . A single task uncertainty set (blue hexagons) can be obtained leveraging information only from the corresponding task, while a forward uncertainty set (green hexagons) can be obtained leveraging information from preceding tasks. Then, the proposed methodology obtains forward and backward uncertainty sets (red hexagons) leveraging information from all the tasks in the sequence.
step only the last task is considered of interest, and is learned leveraging information from the most recent preceding tasks since they are the most similar to the last task.
Techniques developed for continual learning (aka lifelong learning) [4-7, 17-20] aim to learn the whole sequence of tasks but existing methods are designed for situations where tasks' similarities do not depend on the time steps when tasks are observed. In particular, methods based on dynamic architectures learn shared parameters using samples from all the tasks together with task-specific parameters using samples from the corresponding task [6, 20]; and methods based on replay learn parameters using a pool of stored samples from all the preceding tasks together with the samples from the last task [5, 7]. Existing methods for continual learning are not designed for evolving tasks and consider scenarios in which the order of the tasks in the sequence is not relevant. In the current literature of continual learning, only [21] considers scenarios with evolving tasks but focus on the theoretical analysis of transferring information from the preceding tasks.
This paper presents incremental minimax risk classifiers (IMRCs) that determine classification rules minimizing worst-case error probabilities over uncertainty sets that can contain the sequence of evolving underlying distributions. The proposed techniques can effectively exploit forward and backward learning by obtaining uncertainty sets that get smaller using information from all the tasks (see Figure 1). Specifically, the main contributions presented in the paper are as follows.
- We propose forward learning techniques that recursively use the information from preceding tasks to reduce the uncertainty set of the last task.
- We propose forward and backward learning techniques that use the information from the last task to further reduce the sequence of uncertainty sets obtained with forward learning.
- We analytically characterize the increase in ESS provided by the presented methods in terms of the expected quadratic change between consecutive tasks and the number of tasks.
- We numerically quantify the performance improvement provided by IMRCs in comparison with existing techniques using multiple datasets, sample sizes, and number of tasks.
Notations Calligraphic letters represent sets;  $\| \cdot \| _1$  and  $\| \cdot \|_{\infty}$  denote the 1-norm and the infinity norm of its argument, respectively;  $\preceq$  and  $\succeq$  denote vector inequalities;  $\mathbb{I}\{\cdot \}$  denotes the indicator function; and  $\mathbb{E}_{\mathrm{p}}\{\cdot \}$  and  $\mathbb{Var}_{\mathrm{p}}\{\cdot \}$  denote the expectation and the variance of its argument with respect to distribution p. For a vector v,  $v^{(i)}$  and  $\mathbf{v}^{\mathrm{T}}$  denote its i-th component and transpose. Non-linear operators acting on vectors denote component-wise operations. For instance,  $|\mathbf{v}|$  and  $\mathbf{v}^2$  denote the vector formed by the absolute value and the square of each component, respectively. For the reader's convenience, we also provide in Table 2 in Appendix A a list with the main notions used in the paper and their corresponding notations.
This section first formulates the problem of incrementally learning a growing sequence of tasks, and describes evolving tasks in comparison with the time-independent assumption common in continual learning. Then, we briefly describe MRCs that determine classification rules by minimizing the worst-case error probability over an uncertainty set.
In the following, we denote by  $\mathcal{X}$  the set of instances or attributes,  $\mathcal{V}$  the set of labels or classes,  $\Delta(\mathcal{X} \times \mathcal{V})$  the set of probability distributions over  $\mathcal{X} \times \mathcal{V}$ , and  $\mathrm{T}(\mathcal{X}, \mathcal{V})$  the set of classification rules. A classification task is characterized by an underlying distribution  $\mathrm{p}^* \in \Delta(\mathcal{X} \times \mathcal{V})$ , and standard supervised classification methods use a sample set  $D = \{(x_i, y_i)\}_{i=1}^n$  formed by  $n$  i.i.d samples from distribution  $\mathrm{p}^*$  to find a classification rule  $\mathrm{h} \in \mathrm{T}(\mathcal{X}, \mathcal{V})$  with small expected loss  $\ell(\mathrm{h}, \mathrm{p}^*)$ .
In the addressed settings, sample sets  $D_{1}, D_{2}, \ldots$  arrive over time steps 1, 2, ... corresponding with different classification tasks characterized by underlying distributions  $\mathrm{p}_1, \mathrm{p}_2, \ldots$ . Incremental learning aims to continually learn over time the growing sequence of tasks exploiting information acquired from all the tasks. At each time step  $k$ , learning methods obtain classification rules  $\mathrm{h}_1, \mathrm{h}_2, \ldots, \mathrm{h}_k$  for the current sequence of  $k$  tasks using the new sample set  $D_k$  and certain information retained from step  $k - 1$ . The performance of learning methods is assessed at each time step  $k$  by quantifying the performance in the current  $k$  tasks. For instance, overall performance can be assessed by the averaged error  $\frac{1}{k} \sum_{j=1}^{k} \ell(\mathrm{h}_j, \mathrm{p}_j)$  with  $\ell(\mathrm{h}_j, \mathrm{p}_j)$  the expected loss of the classification rule  $\mathrm{h}_j$  for distribution  $\mathrm{p}_j$ .
Evolving tasks are considered by methods for concept drift adaptation, but existing continual learning methods are designed for scenarios where tasks' similarities do not depend on the time steps when tasks are observed. These scenarios are usually mathematically modeled assuming that the tasks' distributions  $\mathrm{p}_1, \mathrm{p}_2, \ldots$  are independent and identically distributed (i.i.d.) [18, 19]. In this paper, we develop techniques designed for evolving tasks that can be mathematically modeled assuming that the changes between consecutive distributions  $\mathrm{p}_2 - \mathrm{p}_1, \mathrm{p}_3 - \mathrm{p}_2, \ldots$  are independent and zero-mean (evolving task assumption). The assumption in this paper is not stronger than the usual i.i.d. assumption and can better describe evolving tasks; their main difference is considering independent changes between consecutive distributions instead of independent distributions. Note that with independent distributions the difference between the distributions of the  $i$ -th and the  $(i + t)$ -th tasks has zero mean and variance  $\mathbb{Var}\{\mathrm{p}_{i + t} - \mathrm{p}_i\} = \mathbb{Var}\{\mathrm{p}_{i + 1} - \mathrm{p}_i\} = 2\mathbb{Var}\{\mathrm{p}_1\}$  that does not depend on  $t$ . On the other hand, with independent changes between consecutive distributions, the difference between the distributions of the  $i$ -th and the  $(i + t)$ -th tasks has zero mean and variance  $\mathbb{Var}\{\mathrm{p}_{i + t} - \mathrm{p}_i\} = \sum_{j = 1}^{t}\mathbb{Var}\{\mathrm{p}_{i + j} - \mathrm{p}_{i + j - 1}\}$  that increases with  $t$ . Appendix H further describes how the assumption used in the paper is more appropriate for evolving tasks than the conventional i.i.d. assumption using real datasets.
2.2 Minimax risk classifiers
The methods presented in the paper are based on robust risk minimization [22, 23] instead of empirical risk minimization since training samples corresponding to different tasks follow different distributions. In particular, we utilize MRCs [24, 25] that learn classification rules by minimizing the worst-case expected loss against an uncertainty set given by constraints on the expectation of a feature mapping  $\Phi : \mathcal{X} \times \mathcal{Y} \to \mathbb{R}^m$  as
$$
\mathcal {U} = \left\{\mathrm {p} \in \Delta (\mathcal {X} \times \mathcal {Y}): | \mathbb {E} _ {\mathrm {p}} \left\{\Phi (x, y) \right\} - \boldsymbol {\tau} | \preceq \boldsymbol {\lambda} \right\} \tag {1}
$$
where  $\tau$  denotes a mean vector of expectation estimates and  $\lambda$  denotes a confidence vector. Feature mappings are vector-valued functions over  $\mathcal{X} \times \mathcal{Y}$ , e.g., one-hot encodings of labels [26] with instances represented by values from the last layers in a neural network [8, 27] or by random Fourier features (RFF) [28, 29].
Given the uncertainty set  $\mathcal{U}$ , MRC rules are solutions of the optimization problem
$$
R (\mathcal {U}) = \min  _ {\mathrm {h} \in \mathrm {T} (\mathcal {X}, \mathcal {Y})} \max  _ {\mathrm {p} \in \mathcal {U}} \ell (\mathrm {h}, \mathrm {p}) \tag {2}
$$
where  $R(\mathcal{U})$  denotes the minimax risk and  $\ell(\mathrm{h},\mathrm{p})$  denotes the expected loss of classification rule  $\mathrm{h}$  for distribution  $\mathrm{p}$ . In the following, we utilize the 0-1-loss so that  $\ell(\mathrm{h},\mathrm{p}) = \mathbb{E}_{\mathrm{p}}\{\mathbb{I}\{\mathrm{h}(x)\neq y\} \}$  and the expected loss with respect to the underlying distribution becomes the error probability of the classification rule. Deterministic MRCs assign each instance  $x\in \mathcal{X}$  with the label  $\mathrm{h}(x)\in \arg \max_{y\in \mathcal{Y}}\Phi (x,y)^{\mathrm{T}}\boldsymbol{\mu}^{*}$  where the parameter  $\boldsymbol{\mu}^{*}$  is the solution of the convex optimization problem
$$
\min  _ {\boldsymbol {\mu}} 1 - \boldsymbol {\tau} ^ {\mathrm {T}} \boldsymbol {\mu} + \max  _ {x \in \mathcal {X}, \mathcal {C} \subseteq \mathcal {Y}} \frac {\sum_ {y \in \mathcal {C}} \Phi (x , y) ^ {\mathrm {T}} \boldsymbol {\mu} - 1}{| \mathcal {C} |} + \boldsymbol {\lambda} ^ {\mathrm {T}} | \boldsymbol {\mu} | \tag {3}
$$
given by the Lagrange dual of (2) [24, 25].
The baseline approach of single-task learning obtains a classification rule  $\mathrm{h}_j$  for each  $j$ -th task leveraging information only from the sample set  $D_j = \{(x_{j,i},y_{j,i})\}_{i = 1}^{n_j}$  given by  $n_j$  samples from distribution  $\mathfrak{p}_j$ . In that case, IMRCs coincide with MRCs for standard supervised classification that obtain the mean and confidence vectors as
$$
\boldsymbol {\tau} _ {j} = \frac {1}{n _ {j}} \sum_ {i = 1} ^ {n _ {j}} \Phi (x _ {j, i}, y _ {j, i}), \quad \boldsymbol {\lambda} _ {j} = \sqrt {\boldsymbol {s} _ {j}}, \quad \boldsymbol {s} _ {j} = \frac {\boldsymbol {\sigma} _ {j} ^ {2}}{n _ {j}} \tag {4}
$$
with  $\sigma_{j}^{2}$  an estimate of  $\mathrm{Var}_{\mathrm{p}_j}\{\Phi (x,y)\}$ , e.g., the sample variance of the  $n_j$  samples. The vector  $s_j$  describes the mean squared errors (MSEs) of the mean vector components and directly gives the confidence vector  $\lambda_{j}$  as shown in (4).
MRCs provide bounds for the minimax risk  $R(\mathcal{U}_j)$  in terms of the smallest minimax risk as described in [24, 25, 30]. The smallest minimax risk is that corresponding to the ideal case of knowing mean vectors exactly, that is, the minimax risk corresponding with the uncertainty set  $\mathcal{U}_j^\infty = \{\mathrm{p} \in \Delta(\mathcal{X} \times \mathcal{Y}) : \mathbb{E}_{\mathrm{p}}\{\Phi(x,y)\} = \tau_j^\infty\}$  given by the expectation  $\tau_j^\infty = \mathbb{E}_{\mathrm{p}_j}\{\Phi(x,y)\}$ . In the baseline approach of single-task learning, if  $R_j^\infty$  and  $\mu_j^\infty$  denote the smallest minimax risk and the MRC parameter corresponding with  $\mathcal{U}_j^\infty$ , with probability at least  $1 - \delta$  we have that
$$
R \left(\mathcal {U} _ {j}\right) \leq R _ {j} ^ {\infty} + \frac {M (\kappa + 1) \sqrt {2 \log (2 m / \delta)}}{\sqrt {n _ {j}}} \| \boldsymbol {\mu} _ {j} ^ {\infty} \| _ {1} \tag {5}
$$
where  $M$  and  $\kappa$  are such that  $\| \Phi (x,y)\|_{\infty}\leq M$  for any  $(x,y)\in \mathcal{X}\times \mathcal{Y}$  and  $subG(\Phi_j)\preceq \kappa \sigma_j$ ,  $subG(\cdot)$  denotes the sub-Gaussian parameter of the argument components, and  $\Phi_j$  denotes the random variable given by the feature mapping of samples from the  $j$ -th task. Inequality (5) is obtained using the bounds in [25] together with the Chernoff bound [31] for sub-Gaussian variables.
In the following sections, we describe techniques that obtain the mean and MSE vectors using forward and backward learning. Once such vectors are obtained, IMRC methods obtain the classifier parameter  $\mu_{j}$  for each  $j$ -th task solving the convex optimization problem in (3) that can be efficiently addressed using conventional methods [32, 33].
3 Forward learning with performance guarantees
This section presents the recursions that allow to obtain mean and MSE vectors for each task retaining information from preceding tasks. In addition, it characterizes the increase in ESS provided by forward learning in terms of the tasks' expected quadratic change and the number of tasks.
3.1 Forward learning
Let  $\tau_{j}^{\rightarrow}$  and  $s_j^\rightarrow$  denote the mean and MSE vectors for forward learning corresponding to the  $j$ -th task. The following recursions allow to obtain  $\tau_{j}^{\rightarrow}$  and  $s_j^\rightarrow$  for each  $j$ -th task using the vectors for the preceding task  $\tau_{j-1}^{\rightarrow}, s_{j-1}^{\rightarrow}$  as
$$
\boldsymbol {\tau} _ {j} ^ {\rightarrow} = \boldsymbol {\tau} _ {j} + \frac {\boldsymbol {s} _ {j}}{\boldsymbol {s} _ {j - 1} ^ {\rightarrow} + \boldsymbol {s} _ {j} + \boldsymbol {d} _ {j} ^ {2}} \left(\boldsymbol {\tau} _ {j - 1} ^ {\rightarrow} - \boldsymbol {\tau} _ {j}\right), \quad \boldsymbol {s} _ {j} ^ {\rightarrow} = \left(\frac {1}{\boldsymbol {s} _ {j}} + \frac {1}{\boldsymbol {s} _ {j - 1} ^ {\rightarrow} + \boldsymbol {d} _ {j} ^ {2}}\right) ^ {- 1} \tag {6}
$$
with  $\tau_{j}$  and  $s_j$  given by (4) and  $\tau_1^{\rightarrow} = \tau_1$  and  $s_1^{\rightarrow} = s_1$ .
The vector  $\pmb{d}_i^2$  assesses the expected quadratic change between consecutive tasks described by  $\pmb{w}_i = \pmb{\tau}_i^\infty - \pmb{\tau}_{i-1}^\infty$ . Taking  $\pmb{d}_i^2 = \mathbb{E}\{\pmb{w}_i^2\}$  and  $\pmb{\sigma}_i^2 = \mathrm{Var}_{\mathrm{pi}}\{\Phi(x,y)\}$  for any  $i$ , the first recursion in (6) provides the unbiased linear estimator of the mean vector  $\pmb{\tau}_j^\infty$  based on  $D_1, D_2, \ldots, D_j$  that has the minimum MSE, while the second recursion in (6) provides its MSE (see Appendix B for a detailed derivation). Vectors  $\pmb{\sigma}_i^2$  and  $\pmb{d}_i^2$  can be estimated online using the sample sets. In particular,  $\pmb{\sigma}_i^2$  can be estimated as the sample variance, while  $\pmb{d}_i^2$  can be estimated using sample averages as
$$
\boldsymbol {d} _ {i} ^ {2} = \sum_ {l = 1} ^ {W} \frac {\left(\boldsymbol {\tau} _ {i _ {l}} - \boldsymbol {\tau} _ {i _ {l - 1}}\right) ^ {2}}{W} \text {w h e r e} i _ {0}, i _ {1}, \dots , i _ {W} \text {a r e t h e c l o s e s t i n d e x e s t s o} i \in \{1, 2, \dots , k \}. \tag {7}
$$
Recursions in (6) obtain mean and MSE vectors for the  $j$ -th task by using information from preceding tasks and the  $j$ -th sample set  $D_{j}$ . Specifically, the first recursion in (6) obtains the mean vector  $\tau_{j}^{\rightarrow}$  by adding a correction to the sample average  $\tau_{j}$ . This correction is proportional to the difference between  $\tau_{j}$  and  $\tau_{j-1}^{\rightarrow}$  with a proportionality constant that depends on the MSE vectors  $s_{j}, s_{j-1}^{\rightarrow}$  and the expected quadratic change  $d_{j}^{2}$ . In particular, if  $s_{j} \ll s_{j-1}^{\rightarrow} + d_{j}^{2}$ , the mean vector is given by the sample average as in single-task learning, and if  $s_{j} \gg s_{j-1}^{\rightarrow} + d_{j}^{2}$ , the mean vector is given by that of the preceding task. Note that for forward learning, at each step  $k$ , only the vectors for the last task  $\tau_{k}^{\rightarrow}$  and  $s_{k}^{\rightarrow}$  need to be obtained from those of the  $(k-1)$ -th task, the vectors for the remaining tasks stay the same as at step  $k-1$  (see also Fig. 2 and Alg. 1 below).
3.2 Performance guarantees and effective sample sizes with forward learning
The following result provides bounds for the minimax risk for each task using forward learning.
Theorem 1. Let  $\mathcal{U}_j^{\rightarrow}$  be the uncertainty set given by (1) using the mean and confidence vectors  $\pmb{\tau}_{j}^{\rightarrow}$  and  $\lambda_j^\rightarrow = \sqrt{s_j^\rightarrow}$  provided by (6), and let  $\kappa$  be such that  $subG(\Phi_j)\preceq \kappa \sigma_j$  and  $subG(w_{j})\preceq \kappa d_{j}$  for  $j = 1,2,\ldots ,k$ . Then, under the evolving task assumption of Section 2, we have with probability at least  $1 - \delta$  that
$$
R \left(\mathcal {U} _ {j} ^ {\rightarrow}\right) \leq R _ {j} ^ {\infty} + \frac {M (\kappa + 1) \sqrt {2 \log (2 m / \delta)}}{\sqrt {n _ {j} ^ {\rightarrow}}} \left\| \boldsymbol {\mu} _ {j} ^ {\infty} \right\| _ {1} \text {f o r a n y} j \in \{1, 2, \dots , k \} \tag {8}
$$
with  $n_1^{\vec{\cdot}} = n_1$  and  $n_j^{\vec{\cdot}}\geq n_j + n_{j - 1}^{\vec{\cdot}}\frac{\|\pmb{\sigma}_j^2\|_{\infty}}{\|\pmb{\sigma}_j^2\|_{\infty} + n_{j - 1}^{\vec{\cdot}}\|\pmb{d}_j^2\|_{\infty}}$  for  $j\geq 2$
The value  $n_j^{\rightarrow}$  in (8) is the ESS of the proposed IMRC method with forward learning since the bound in (8) coincides with that of single-task learning in (5) if the sample size for the  $j$ -th task is  $n_j^{\rightarrow}$ . The ESS of each task is obtained by adding a fraction of the ESS for the preceding task to the sample size. In particular, if  $d_j^2$  is large, the ESS is given by the sample size, while if  $d_j^2$  is small, the ESS is given by the sum of the sample size and the ESS of the preceding task.
The bound in (8) shows that recursions in (6) do not need to use very accurate values for  $\sigma_{j}$  and  $d_{j}$ . Specifically, the coefficient  $\kappa$  in (8) can be taken to be small as long as the values used for  $\sigma_{j}$  and  $d_{j}$  are not much lower than the sub-Gaussian parameters of  $\Phi_{j}$  and  $\boldsymbol{w}_{j}$ , respectively. In particular,  $\kappa$  is smaller than the maximum of  $M / \min_{j,i} \{\sigma_{j}^{(i)}\}$  and  $2M / \min_{j,i} \{d_{j}^{(i)}\}$  due to the bound for the sub-Gaussian parameter of bounded random variables (see e.g., Section 2.1.2 in [31]).
Theorem 1 shows the increase in ESS in terms of the ESS of the preceding task. The following result allows to directly quantify the ESS in terms of the tasks' expected quadratic change and the number of tasks.
Theorem 2. If  $\| d_j^2\|_\infty \leq d^2$ ,  $M \leq 1$ , and  $n_j \geq n$  for  $j = 1, 2, \ldots, k$ , then for any  $j \in \{1, 2, \ldots, k\}$  we have that the ESS in (8) can be taken so that it satisfies
$$
n _ {j} ^ {\rightarrow} \geq n \left(1 + \frac {(1 + \alpha) ^ {2 j - 1} - 1 - \alpha}{\alpha (1 + \alpha) ^ {2 j - 1} + \alpha}\right) \text {w i t h} \alpha = \frac {2}{\sqrt {1 + \frac {4}{n d ^ {2}}} - 1}. \tag {9}
$$
In particular, if  $nd^2 < 1 / j^2$ , we have that  $n_j^\prec \geq n\left(1 + \frac{j - 1}{3}\right)$ .
Proof. See Appendix D.
The above theorem characterizes the increase in ESS provided by forward learning in terms of the tasks' expected quadratic change. Such increase grows monotonically with the number of preceding tasks  $j$  as shown in (9) and becomes proportional to  $j$  when the expected quadratic change is smaller than  $1 / (j^2 n)$ . Figure 3 below further illustrates the increase in ESS with respect to the sample size  $(n_j \stackrel{\rightarrow}{/} n)$  due forward learning in comparison with forward and backward learning.
4 Forward and backward learning with performance guarantees
This section presents the recursions that allow to obtain mean and MSE vectors for each task leveraging information from all the tasks. In addition, it characterizes the increase in ESS provided by forward and backward learning in terms of the tasks' expected quadratic change and the number of tasks.
Backward learning is more challenging than forward learning since the new task provides additional information for preceding tasks at each time step, while the information from preceding tasks is always the same. The techniques proposed below for backward learning effectively increase the ESS over time by carefully accounting for the new information at each step.
4.1 Forward and backward learning