SlowGuess/NeurIPS_2023_OCR huggingface dataset:
At each step  $k$ , the proposed techniques learn to classify each  $j$ -th task leveraging information obtained from the  $j$  preceding tasks (tasks  $\{1, 2, \ldots, j\}$ ) and from the  $k - j$  succeeding tasks (tasks  $\{j + 1, j + 2, \ldots, k\}$ ). From preceding tasks, we obtain the forward mean and MSE vectors  $\tau_{j}^{\rightarrow}, s_{j}^{\rightarrow}$  using recursions in (6), while from succeeding tasks, we obtain the backward mean and MSE vectors  $\tau_{j}^{\rightarrow k}, s_{j}^{\rightarrow k}$  using recursions in (6) in retrodiction. Specifically, vectors  $\tau_{j}^{\rightarrow k}$  and  $s_{j}^{\rightarrow k}$  are obtained using the same recursion as for  $\tau_{j}^{\rightarrow}$  and  $s_{j}^{\rightarrow}$  in (6) with  $s_{j + 1}^{\rightarrow k}, d_{j + 1}^{2}$ , and  $\tau_{j + 1}^{\rightarrow k}$  instead of  $s_{j - 1}^{\rightarrow}, d_{j}^{2}$ , and  $\tau_{j - 1}^{\rightarrow}$ .
Let  $\tau_{j}^{\overline{=k}}$  and  $s_j^{\overline{=k}}$  denote the mean and MSE vectors for forward and backward learning corresponding to the  $j$ -th task for  $j \in \{1, 2, \dots, k\}$ . The following recursions allow to obtain the mean and MSE vectors  $\tau_{j}^{\overline{=k}}$  and  $s_j^{\overline{=k}}$  for each  $j$ -th task using those vectors for forward learning  $\tau_{j}^{\rightarrow}$ ,  $s_j^{\rightarrow}$  and backward learning  $\tau_{j+1}^{\overline{k}}$ ,  $s_{j+1}^{\overline{k}}$  as
$$
\boldsymbol {\tau} _ {j} ^ {\rightleftharpoons k} = \boldsymbol {\tau} _ {j} ^ {\rightleftharpoons} + \frac {\boldsymbol {s} _ {j} ^ {\rightleftharpoons}}{\boldsymbol {s} _ {j} ^ {\rightleftharpoons} + \boldsymbol {s} _ {j + 1} ^ {\leftarrow k} + \boldsymbol {d} _ {j + 1} ^ {2}} \left(\boldsymbol {\tau} _ {j + 1} ^ {\leftarrow k} - \boldsymbol {\tau} _ {j} ^ {\rightleftharpoons}\right), \quad \boldsymbol {s} _ {j} ^ {\rightleftharpoons k} = \left(\frac {1}{\boldsymbol {s} _ {j} ^ {\rightleftharpoons}} + \frac {1}{\boldsymbol {s} _ {j + 1} ^ {\leftarrow k} + \boldsymbol {d} _ {j + 1} ^ {2}}\right) ^ {- 1} \tag {10}
$$
with  $\tau_{k}^{-k} = \tau_{k}, s_{k}^{-k} = s_{k}$  and  $\tau_{k}^{\overrightarrow{= k}} = \tau_{k}^{\rightarrow}$ ,  $s_{k}^{\overrightarrow{= k}} = s_{k}^{\rightarrow}$ . Analogously to the case of forward learning in Section 3.1, taking  $d_{i}^{2} = \mathbb{E}\{\pmb{w}_{i}^{2}\}$  and  $\sigma_{i}^{2} = \mathbb{Var}_{\mathrm{pi}}\{\Phi(x,y)\}$  for any  $i$ , the first recursion in (10) provides the unbiased linear estimator of the mean vector  $\tau_{j}^{\infty}$  based on  $D_{1}, D_{2}, \ldots, D_{j}$  and  $D_{j+1}, D_{j+2}, \ldots, D_{k}$  that has the minimum MSE, while the second recursion in (10) provides its MSE (see Appendix B for a detailed derivation).
Recursions in (10) obtain at step  $k$  the mean and MSE vectors for the  $j$ -th task by retaining information from preceding tasks and acquiring information from the new task. Specifically, the first recursion in (10) obtains the mean vector  $\tau_{j}^{\overleftarrow{k}}$  by adding a correction to the mean vector of the corresponding task  $\tau_{j}^{\rightarrow}$  obtained for forward learning. This correction is proportional to the difference between  $\tau_{j}^{\overleftarrow{k}}$  and  $\tau_{j+1}^{\overleftarrow{k}}$  with a proportionality constant that depends on the MSE vectors  $s_{j}^{\overleftarrow{k}}$ ,  $s_{j+1}^{\overleftarrow{k}}$  and the expected quadratic change  $d_{j+1}^{2}$ . In particular, if  $s_{j}^{\overleftarrow{k}} \ll s_{j+1}^{\overleftarrow{k}} + d_{j+1}^{2}$ , the mean vector is given by that of the corresponding task for forward learning, and if  $s_{j}^{\overleftarrow{k}} \gg s_{j+1}^{\overleftarrow{k}} + d_{j+1}^{2}$ , the mean vector is given by that of the next task for backward learning.
This section describes the implementation of the proposed IMRCs with forward and backward learning and its computational and memory complexities.
Figure 2 depicts the flow diagram for the proposed IMRC methodology. The presented techniques carefully avoid the repeated usage of the same information from the sequence
of tasks. At each step  $k$ , the IMRC method obtains forward mean vector  $\tau_{k}^{\rightarrow}$  for the  $k$ -th task leveraging information from preceding tasks using the forward mean vector  $\tau_{k-1}^{\rightarrow}$  and the sample average  $\tau_{k}$ . Reciprocally, backward mean vectors  $\tau_{j+1}^{\rightarrow k}$  for each  $j$ -th task are obtained leveraging information from the  $k$ -th task through the sample average  $\tau_{k}$ .
Then, the forward and backward mean vectors  $\tau_{j}^{\overrightarrow{k}}$  are obtained from the forward mean vectors  $\tau_{j}^{\overrightarrow{-}}$  and the backward mean vectors  $\tau_{j+1}^{\overrightarrow{k}}$ . In particular,  $\tau_{j}^{\overrightarrow{-}}$  provides the information from the preceding tasks  $1, 2, \ldots, j$ , while  $\tau_{j+1}^{\overrightarrow{k}}$  provides the information from the succeeding tasks  $j+1, j+2, \ldots, k$ . At each step  $k$ , the IMRC method obtains forward and backward mean vectors  $\tau_{j}^{\overrightarrow{k}}$  for  $j = k-b, k-b+1, \ldots, k$  with  $b$  the number of backward steps. In particular, if  $b = 0$ , IMRC carries out only forward learning. Note that, at each step  $k$ , the proposed IMRC methods only need to retain the forward mean vectors  $\tau_{j}^{\overrightarrow{-}}$  and sample averages  $\tau_{j}$  for  $j = k-b, k-b+1, \ldots, k$ .
Figure 2: Diagram for IMRC methodology.
Algorithm 1 IMRC at step  $k$
Input:  $D_{k}$  from new task, and  $\pmb{\tau}_{j},\pmb{s}_{j},\pmb{\tau}_{j}^{\rightarrow},\pmb{s}_{j}^{\rightarrow}$  for  $k - b\leq j\leq k - 1$  from previous  $b - 1$  steps   
Output:  $\mu_{j}$  for  $k - b\leq j\leq k,\tau_k,s_k,\tau_k^{\rightarrow},s_k^{\rightarrow}$    
Obtain sample average and MSE vectors  $\pmb{\tau}_{k}^{-k} = \pmb{\tau}_{k},\pmb{s}_{k}^{-k} = \pmb{s}_{k}$  using the sample set  $D_{k}$ $\triangleright$  Single-task Estimate the tasks' expected quadratic change  $\pmb{d}_k^2$  using (7)   
Obtain the forward mean and MSE vectors  $\pmb{\tau}_{k}^{\rightarrow},\pmb{s}_{k}^{\rightarrow}$  using (6)  $\triangleright$  Forward Take  $\lambda_{k}^{\rightarrow} = \sqrt{\pmb{s}_{k}^{\rightarrow}}$  and obtain classifier parameter  $\pmb{\mu}_{k}$  solving the optimization problem (3)   
for  $j = k - 1,k - 2,\dots ,k - b$  do Estimate the tasks' expected quadratic change  $\pmb{d}_j^2$  using (7)   
Obtain backward mean and MSE vectors  $\pmb{\tau}_{j + 1}^{-k},\pmb{s}_{j + 1}^{-k}$  using (6) in retrodiction  $\triangleright$  Backward Obtain mean and MSE vectors  $\pmb{\tau}_{j}^{\overrightarrow{k}},\pmb{s}_{j}^{\overrightarrow{k}}$  using (10)  $\triangleright$  Forward and backward Take  $\lambda_{j}^{\overrightarrow{k}} = \sqrt{\pmb{s}_{j}^{\overrightarrow{k}}}.$  and obtain classifier parameters  $\pmb{\mu}_{j}$  solving the optimization problem (3)
Algorithm 1 details the implementation of the proposed IMRCs at each step. For  $k$  steps, IMRCs have computational complexity  $\mathcal{O}((b + 1)Kmk)$  and memory complexity  $\mathcal{O}((b + k)m)$  where  $K$  is the number of iterations used for the convex optimization problem (3),  $m$  is the length of the feature vector, and  $b$  is the number of backward steps. The complexity of forward and backward learning increases proportionally to the number of backward steps that can be taken to be rather small, as shown in the following. Even more efficient implementations can be obtained using Rauch-Tung-Striebel recursions (see e.g., Section 7.2 in [34]) that can be used to obtain  $\tau_{j}^{\overline{=} k}$  from  $\tau_{j+1}^{\overline{=} k}$ , as shown in Appendix E.
4.3 Performance guarantees and effective sample sizes with forward and backward learning
The following result provides bounds for the minimax risk for each task using forward and backward learning
Theorem 3. Let  $\mathcal{U}_j^{\overline{\equiv} k}$  be the uncertainty set given by (1) using the mean and confidence vector  $\tau_{j}^{\overline{\equiv} k}$  and  $\lambda_j^{\overline{\equiv} k} = \sqrt{s_j^{\overline{\equiv} k}}$  provided by (10), and let  $\kappa$  and  $n_j^{\rightarrow}$  be as in Theorem 1. Then, under the evolving task assumption of Section 2, we have with probability at least  $1 - \delta$  that
$$
R \left(\mathcal {U} _ {j} ^ {\rightleftharpoons k}\right) \leq R _ {j} ^ {\infty} + \frac {M (\kappa + 1) \sqrt {2 \log (2 m / \delta)}}{\sqrt {n _ {j} ^ {\rightleftharpoons k}}} \left\| \boldsymbol {\mu} _ {j} ^ {\infty} \right\| _ {1} \text {f o r a n y} j \in \{1, 2, \dots , k \} \tag {11}
$$
with  $n_{k}^{\overleftarrow{= k}} = n_{k}^{\rightarrow}$  and  $n_{j}^{\overleftarrow{= k}}\geq n_{j}^{\rightarrow} + n_{j + 1}^{-k}\frac{\|\pmb{\sigma}_{j}^{2}\|_{\infty}}{\|\pmb{\sigma}_{j}^{2}\|_{\infty} + n_{j + 1}^{-k}\|\pmb{d}_{j + 1}^{2}\|_{\infty}}$  for  $j\leq k - 1$  , where the backward ESSs satisfy  $n_k^{-k} = n_k$  and  $n_j^{-k}\geq n_j + n_{j + 1}^{-k}\frac{\|\pmb{\sigma}_j^2\|_\infty}{\|\pmb{\sigma}_j^2\|_\infty + n_{j + 1}^{-k}\|\pmb{d}_{j + 1}^2\|_\infty}$
Proof. See Appendix F.
Theorem 3 shows that the methods proposed can increase ESS of each task by leveraging information from all the tasks. In particular, the bounds for forward and backward learning provided by inequality (11) are significantly lower than those for forward learning in Theorem 1. The ESS of each task is obtained by adding a fraction of the ESS for the next task to the ESS of the corresponding task using forward learning. In particular, if  $d_j^2$  is large, the ESS is given by that with forward learning, while if  $d_j^2$  is small, the ESS is given by the sum of the ESS using forward learning and the ESS of the next task.
Theorem 3 shows the increase in ESS in terms of the ESS with forward learning and the ESS of the next task. The following result allows to directly quantify the ESS in terms of the tasks' expected quadratic change and the number of tasks.
Theorem 4. If  $\| d_j^2\|_\infty \leq d^2$ ,  $M \leq 1$ , and  $n_j \geq n$  for  $j = 1, 2, \ldots, k$ , then for any  $j \in \{1, 2, \ldots, k\}$  we have that the ESS in (11) can be taken so that it satisfies
$$
n _ {j} ^ {\overline {{\overline {{\alpha}}}}} \geq n \left(1 + \frac {(1 + \alpha) ^ {2 j - 1} - 1 - \alpha}{\alpha (1 + \alpha) ^ {2 j - 1} + \alpha} + \frac {(1 + \alpha) ^ {2 (k - j) + 1} - 1 - \alpha}{\alpha (1 + \alpha) ^ {2 (k - j) + 1} + \alpha}\right) \text {w i t h} \alpha = \frac {2}{\sqrt {1 + \frac {4}{n d ^ {2}}} - 1}. \tag {12}
$$
In particular, for  $j \geq 2$ , we have that
$$
n _ {j} ^ {\rightleftharpoons k} \geq n _ {j} ^ {\leftarrow} + n \frac {j (k - j)}{j + 2 (k - j)} \geq n \left(1 + \frac {j - 1}{3} + \frac {j (k - j)}{j + 2 (k - j)}\right) \quad \text {i f} \quad n d ^ {2} <   \frac {1}{j ^ {2}}
$$
$$
n _ {j} ^ {\rightleftharpoons k} \geq n _ {j} ^ {\rightharpoonup} + \frac {1}{5} \sqrt {\frac {n}{d ^ {2}}} \qquad \geq n \left(1 + \frac {2}{5 \sqrt {n d ^ {2}}}\right) \qquad \mathrm {i f} \frac {1}{j ^ {2}} \leq n d ^ {2} <   1
$$
$$
n _ {j} ^ {\overleftarrow {=} k} \geq n _ {j} ^ {\rightarrow} + \frac {1}{3 d ^ {2}} \quad \geq n \left(1 + \frac {2}{3 n d ^ {2}}\right) \quad \text {i f} \quad n d ^ {2} \geq 1.
$$
Proof. See Appendix G.
The above theorem characterizes the increase in ESS provided by forward and backward learning. This increase grows monotonically with the number of preceding tasks  $j$  and with the number of succeeding tasks  $k - j$  as shown in (12).
In addition, it becomes proportional to the total number of tasks  $k$  when the expected quadratic change is smaller than  $1 / (j^2 n)$  and  $j \geq k / 2$ . Figure 3 further illustrates the increase in ESS with respect to the sample size  $(n_j^{= k} / n)$  due to forward and backward learning in comparison with forward learning. As the figure shows, the increase in ESS can be classified into three regimes depending on the sample size  $n$  and the expected quadratic change  $d^2$ . The ESS is only marginally larger than the sample size for sizeable values of  $nd^2$  (large samples sizes or drastic changes in the distribution); the ESS quickly increases when  $nd^2$  becomes small (reduced sample sizes and moderate changes in the distribution); and the ESS becomes proportional to the total number of tasks  $k$  if  $nd^2$  is rather small (very small sample sizes and very slow changes in the distribution).
Figure 3: ESS increase provided by forward and backward learning.
Table 1: Classification error, standard deviation, and running time of the proposed IMRC method in comparison with existing techniques using  $n = 10$  samples per task.
Method	GEM [5]	MER [17]	ELLA [4]	EWC [6]	Condor [9]	DriftSurf [10]	AUE [16]	IMRC
Yearbook [1]	.18 ± .03	.16 ± .03	.45 ± .01	.47 ± .05	.14 ± .01	.33 ± .02	.33 ± .02	.13 ± .04
I. noise [35]	.39 ± .08	.17 ± .03	.48 ± .05	.47 ± .04	.16 ± .02	.48 ± .02	.48 ± .02	.15 ± .03
DomainNet [36]	.69 ± .05	.38 ± .04	.67 ± .05	.75 ± .04	.45 ± .04	.32 ± .02	.33 ± .02	.34 ± .06
UTKFaces [37]	.12 ± .00	.17 ± .09	.19 ± .12	.12 ± .00	.23 ± .00	.12 ± .00	.12 ± .00	.10 ± .01
R. MNIST [38]	.36 ± .06	.37 ± .09	.48 ± .05	.48 ± .01	.45 ± .02	.48 ± .02	.48 ± .02	.36 ± .01
CLEAR [39]	.57 ± .10	.10 ± .03	.61 ± .06	.65 ± .03	.35 ± .00	.33 ± .00	.33 ± .01	.09 ± .03
P. Supply [10]	.46 ± .01	.47 ± .01	.41 ± .07	.47 ± .00	.36 ± .01	.45 ± .02	.45 ± .02	.30 ± .01
Usenet1 [40]	.48 ± .02	.48 ± .02	.39 ± .05	.48 ± .02	.48 ± .02	.40 ± .01	.40 ± .01	.32 ± .03
Usenet2 [40]	.36 ± .01	.33 ± .01	.39 ± .03	.48 ± .02	.42 ± .01	.35 ± .03	.39 ± .03	.33 ± .02
German [41]	.38 ± .16	.38 ± .10	.33 ± .04	.35 ± .11	.41 ± .02	.36 ± .00	.35 ± .02	.34 ± .01
Spam [2]	.25 ± .05	.13 ± .02	.28 ± .07	.33 ± .11	.22 ± .02	.24 ± .03	.25 ± .03	.13 ± .01
Covert. [10]	.09 ± .01	.08 ± .00	.13 ± .01	.09 ± .00	.09 ± .00	.10 ± .00	.11 ± .00	.08 ± .00
Ave. running time 0.238 0.250 0.055 0.313 0.184 0.128 0.044 0.275
This section first compares the classification performance of IMRCs with existing techniques using multiple datasets, then we show the performance improvement of the presented IMRCs due to forward and backward learning. In Appendix H, we provide additional implementation details and numerical results. The implementation of the proposed IMRCs is available on the web https://github.com/MachineLearningBCAM/IMRCs-for-incremental-learning-NeurIPS-2023.
The proposed method is evaluated using 12 public datasets composed by evolving tasks [10, 1, 35-41, 2]. Six datasets are formed by images with characteristics/quality/realism that change over time; while the rest are formed by tabular data that are segmented by time (see further details in Appendix H). For instance, each task in the "Yearbook" dataset corresponds to portrait's photographs from one year from 1905 to 2013 and the goal is to predict males and females; and each task in the "CLEAR" dataset corresponds to images with a natural temporal evolution of visual concepts per year from 2004 to 2014 and the goal is to predict if an image is soccer, hockey, or racing. For the image datasets, instances are represented by pixel values in "Rotated MNIST" dataset, and by the last layer of the ResNet18 pre-trained network [42] in the remaining datasets; while for the tabular datasets, instances are represented by 200 RFFs [28, 29] with scaling parameter  $\sigma^2 = 10$ .
The proposed IMRC method is compared with 7 state-of-the-art-techniques: 2 techniques of continual learning based on experience replay [5, 17], a technique of continual learning based on regularization [6], a technique of continual learning based on dynamic architectures [4], 2 techniques of concept drift adaptation based on weight factors [9, 16], and a technique of concept drift adaptation based on sliding windows [10]. The hyper-parameters in all methods are set to the default values provided by the authors. IMRCs are implemented using  $b = 3$  backward steps and the expected quadratic change  $d_{j}^{2}$  is estimated using  $W = 2$  in (7). In Appendix H, among other additional results, we study the change in classification error and processing time achieved by varying the number  $b$  of backward steps.
In the first set of numerical results, we compare the performance of the proposed IMRCs with existing techniques using  $n = 10$  samples per task. These numerical results are obtained computing the average classification error over all the time steps and tasks in 50 random instantiations of data samples. As can be observed in Table 1, IMRCs can significantly improve performance in evolving tasks with respect to existing methods. Certain techniques of continual learning or concept drift adaptation achieve top performance in specific datasets, but the performance improvement of the methods presented are realized over general datasets with evolving tasks. The results for  $n = 10$  samples per task in Table 1 are complemented with results for  $n = 50, 100,$  and 150 samples per task in Tables 4, 5, and 6 in Appendix H. These results show that the improved performance of IMRCs compared to the state-of-the-art techniques is similar for different sample sizes. In addition, Table 1 shows that the average running time per task of IMRC methods is similar to other state-of-the-art methods.
(a) Classification error per number of tasks in "Yearbook" dataset.
(b) Classification error per sample size in "Yearbook" dataset.
Figure 4: Forward and backward learning can sharply boost performance and ESS as tasks arrive.
In the second set of numerical results, we analyze the contribution of forward and backward learning to the final performance of IMRCs. In particular, we show the relationship among classification error, number of tasks, and sample size for single-task, forward, and forward and backward learning. These numerical results are obtained averaging, for each number of tasks and sample size, the classification errors achieved with 10 random instantiations of data samples in "Yearbook" dataset (see Appendix H for further details). Figure 4a shows the classification error of IMRC method divided by the classification error of single-task learning for different number of tasks with  $n = 10$  and  $n = 100$  sample sizes. Such figure shows that forward and backward learning can significantly improve the performance of IMRCs as tasks arrive. In addition, Figure 4b shows the classification error of IMRC method for different sample sizes with  $k = 10$  and  $k = 100$  tasks. Such figure shows that IMRCs with forward and backward learning for  $k = 100$  tasks and  $n = 10$  samples achieve significantly better results than single-task learning using  $n = 100$  samples. In particular, the experiments show that the methods proposed can effectively exploit backward learning that results in enhanced classification error in all the experimental results.
The paper proposes IMRCs that effectively exploit forward and backward learning and account for evolving tasks by using a sequence of uncertainty sets that can contain the true underlying distributions. In addition, the paper analytically characterizes the increase in ESS achieved by the proposed techniques in terms of the tasks' expected quadratic change and number of tasks. The numerical results assess the performance improvement of IMRCs with respect to existing methods using multiple datasets, sample sizes, and number of tasks. The proposed methodology for incremental learning with evolving tasks can lead to techniques that further approach the humans' ability to learn from few examples and to continuously improve on tasks that arrive over time.
Funding in direct support of this work has been provided by projects PID2022-137063NB-I00, PID2022-137442NB-I00, CNS2022-135203, and CEX2021-001142-S funded by MCIN/AEI/10.13039/501100011033 and the European Union "NextGenerationEU"/PRTR, BCAM Severo Ochoa accreditation CEX2021-001142-S / MICIN / AEI/ 10.13039/501100011033 funded by the Ministry of Science and Innovation, and programes ELKARTEK, IT1504-22, and BERC-2022-2025 funded by the Basque Government.
[39] Zhiqiu Lin, Jia Shi, Deepak Pathak, and Deva Ramanan. The CLEAR benchmark: Continual learning on real-world imagery. In Thirty-fifth Conference on Neural Information Processing Systems Datasets and Benchmarks Track (Round 2), 2021.
[40] Ioannis Katakis, Grigorios Tsoumakas, and Ioannis Vlahavas. An ensemble of classifiers for coping with recurring contexts in data streams, 2008.
[41] Dheeru Dua and Casey Graff. UCI machine learning repository, 2017.
[42] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In IEEE Conference on Computer Vision and Pattern Recognition, 2016.
A Main notations used in the paper.
In this section we summarize the notations used in the paper. Table 2 shows the notation used for the mean vector, the confidence vector, the MSE vector, the ESS, the uncertainty set, and the minimax risk using single task, forward, backward, and forward and backward learning.
Table 2: Main notations used in the paper
Single task	Forward	Backward	Forward and Backward
Mean vector	τj	τj←	τj-k	τj=k
Confidence vector	λj	λj←	λj-k	λj=k
MSE vector	sj	sj←	sj-k	sj=k
ESS	nj	nj←	nj-k	nj=k
Uncertainty set	Uj	Uj←	Uj-k	Uj=k
Minimax risk	R(Uj)	R(Uj←)	R(Uj-k)	R(Uj=k)
B Derivation of recursions in (6) for forward learning and recursions in (10) for forward and backward learning
This section shows how recursions in (6) and recursions in (10) are obtained using those for filtering and smoothing in linear dynamical systems.
The mean vectors evolve over time steps through the linear dynamical system
$$
\boldsymbol {\tau} _ {j} ^ {\infty} = \boldsymbol {\tau} _ {j - 1} ^ {\infty} + \boldsymbol {w} _ {j} \tag {13}
$$
where vectors  $\boldsymbol{w}_j$  for  $j \in \{2, 3, \dots, k\}$  are independent and zero-mean because  $\mathrm{p}_j - \mathrm{p}_{j-1}$  are independent and zero-mean. In addition, each state variable  $\tau_j^\infty$  is observed at each step  $j$  through  $\tau_j$  that is the sample average of i.i.d. samples from  $\mathrm{p}_j$ , so that we have
$$
\boldsymbol {\tau} _ {j} = \boldsymbol {\tau} _ {j} ^ {\infty} + \boldsymbol {v} _ {j} \tag {14}
$$
where vectors  $\pmb{v}_j$  for  $j\in \{1,2,\dots,k\}$  are independent and zero mean, and independent of  $\pmb{w}_j$  for  $j\in \{1,2,\dots,k\}$ . Therefore, equations (13) and (14) above describe a linear dynamical system (state-space model with white noise processes) [34]. For such systems, the Kalman filter recursions provide the unbiased linear estimator with minimum MSE based on samples corresponding to preceding steps  $D_{1},D_{2},\ldots ,D_{j}$ , and fixed-lag smoother recursions provide the unbiased linear estimator with minimum MSE based on samples corresponding to preceding and succeeding steps  $D_{1},D_{2},\ldots ,D_{k}$  [34]. Then, equations in (6) and equations in (10) are obtained after some algebra from the Kalman filter recursions and fixed-lag smoother recursions, respectively.
Proof. To obtain bound in (8) we first prove that the mean vector estimate and the MSE vector given by (6) satisfy
$$
\mathbb {P} \left\{\left| \tau_ {j} ^ {\infty (i)} - \tau_ {j} ^ {\rightarrow (i)} \right| \leq \kappa \sqrt {2 s _ {j} ^ {\rightarrow (i)} \log \left(\frac {2 m}{\delta}\right)} \right\} \geq (1 - \delta) \tag {15}
$$
for any component  $i = 1,2,\ldots ,m$ . Then, we prove that  $\| \sqrt{\pmb{s}_j^{\rightarrow}} \|_{\infty}\leq M / \sqrt{n_j^{\rightarrow}}$  for  $j\in \{1,2,\dots ,k\}$ , where the ESSs satisfy  $n_1^{\rightarrow} = n_1$  and  $n_j^\rightarrow \geq n_j + n_{j - 1}^{\rightarrow}\frac{\|\pmb{\sigma}_j^2\|_{\infty}}{\|\pmb{\sigma}_j^2\|_{\infty} + n_{j - 1}^{\rightarrow}\|\pmb{d}_j^2\|_{\infty}}$  for  $j\geq 2$ .
To obtain inequality (15), we prove by induction that each component  $i = 1,2,\ldots ,m$  of the error in the mean vector estimate  $z_{j}^{\rightarrow (i)} = \tau_{j}^{\infty (i)} - \tau_{j}^{\rightarrow (i)}$  is sub-Gaussian with parameter  $\eta_j^{\rightarrow (i)}\leq \kappa \sqrt{s_j^{\rightarrow(i)}}$  Firstly, for  $j = 1$  , we have that
$$
z _ {1} ^ {\rightharpoonup (i)} = \tau_ {1} ^ {\infty (i)} - \tau_ {1} ^ {\rightharpoonup (i)} = \tau_ {1} ^ {\infty (i)} - \tau_ {1} ^ {(i)}.
$$