SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Jianwei Tang
tangjw7@mail2.sysu.edu.cn
Jiangxin Sun
sunjx5@mail2.sysu.edu.cn
Xiaotong Lin
linxt29@mail2.sysu.edu.cn
Lifang Zhang
Dongguan University of Technology
2017028@dgut.edu.cn
Wei-Shi Zheng
Jian-Fang Hu*
hujf5@mail.sysu.edu.cn
Human Motion Prediction (HMP) aims to predict future poses at different moments according to past motion sequences. Previous approaches have treated the prediction of various moments equally, resulting in two main limitations: the learning of short-term predictions is hindered by the focus on long-term predictions, and the incorporation of prior information from past predictions into subsequent predictions is limited. In this paper, we introduce a novel multi-stage training framework called Temporal Continual Learning (TCL) to address the above challenges. To better preserve prior information, we introduce the Prior Compensation Factor (PCF). We incorporate it into the model training to compensate for the lost prior information. Furthermore, we derive a more reasonable optimization objective through theoretical derivation. It is important to note that our TCL framework can be easily integrated with different HMP backbone models and adapted to various datasets and applications. Extensive experiments on four HMP benchmark datasets demonstrate the effectiveness and flexibility of TCL. The code is available at https://github.com/hyqlat/TCL.
Human Motion Prediction (HMP) aims to predict future poses at varied temporal moments based on the observed motion sequences. The accurate prediction of human motion plays a vital role in many applications, such as autonomous driving, human-robot interaction, and security monitoring, enabling the anticipation and mitigation of risks. This task is challenging due to its requirement for predicting multiple moments, including short-term predictions for the "near-future" and long-term predictions for the "far-future".
Previous approaches address this task by autoregressively forecasting using recurrent neural networks (RNNs) and transformer architectures [1, 4, 5, 9-12, 14, 26, 37, 40, 42], or parallelly generating all frames with graph convolution networks (GCNs) [2, 6, 7, 23, 25, 30, 31, 28]. These methods employ
*Jian-Fang Hu is the corresponding author. Hu is also with Guangdong Province Key Laboratory of Information Security Technology, Guangzhou, China and Key Laboratory of Machine Intelligence and Advanced Computing, Ministry of Education, China.
Figure 1: Illustration of three different prediction settings. "short+long" and "short only" represent optimizing prediction for the entire sequence and prediction only for the first 5 frames, respectively. "short then short+long" denotes training the entire sequence after pretraining for the first 5 frames.
(a) Short-term prediction result
(b) Long-term prediction result
Figure 2: Preliminary experiment results of three different prediction settings (lower values indicate better performance). (a) shows the short-term prediction results (predicting the 2-nd frame), while (b) illustrates the long-term prediction results(predicting the 25-th frame).
the one-stage training strategy to directly train a model that can predict both the short-term prediction and the long-term prediction. However, the long-term motion prediction is more challenging since the future motion can vary greatly (i.e., the prediction space is large), which would increase the uncertainty and ambiguity of future prediction. As the prediction length increases, the fitting of the high-uncertainty long-term prediction will gradually dominate the learning process of the prediction model, which hinders the learning of short-term predictions and further limits the full potential of leveraging the prior knowledge learned from short-term inputs to facilitate long-term predictions.
This motivates us to exploit proper training strategies to better learn and utilize the prior knowledge. We further investigate this by conducting preliminary experiments on the following settings "short+long", "short only", and "short then short+long", which are illustrated in Figure 1. The results are summarized in Figure 2. We observe that "short then short+long" outperforms "short+long" on long-term prediction, which implies that the knowledge learned in short-term prediction can serve as a prior to facilitating the learning of "far-future" prediction. This is intuitive and thus we can use a progressive learning approach, where the model is trained to predict increasing numbers of frames over multiple training stages, e.g., starting with 5 frames in the first stage and 10 frames in the second stage, and so on. However, we also observe that "short then short+long" performs worse than "short only" by a considerable margin for short-term prediction, which demonstrates the joint learning of short-term and long-term prediction results in knowledge forgetting for short-term prediction.
To overcome these problems, we introduce the Prior Compensation Factor (PCF) into the multi-stage training method to obtain a sequential continuous learning framework, which is named Temporal Continual Learning (TCL). It is a multi-stage training framework that alleviates constraint posed by long-term prediction on short-term prediction and effectively utilize prior information from short-term prediction. Specifically, we divide the future sequence into segments and divide the training process
into multiple stages accordingly. We incrementally increase the number of prediction segments, allowing us to leverage the prior knowledge acquired from earlier stages for predicting the subsequent ones. Upon completion of each stage's training, the learned prior knowledge is saved in the model parameters. However, with the changing of optimization objective when switching stages, the prior knowledge gradually fades away. In order to overcome this forgetting problem of prior knowledge, we further introduce the PCF, which is designed as a learnable variable. Then, we derive a more reasonable optimization objective for this regression problem through theoretical derivation.
The proposed training framework is flexible and can be easily integrated with various HMP backbones or adapted to different datasets. To validate the effectiveness and flexibility of our framework, we conduct experiments on four popular HMP benchmark datasets by integrating TCL with several HMP backbones. Our contributions can be summarized as follows: 1) We identify certain limitations in existing HMP models and propose a novel multi-stage training strategy called Temporal Continual Learning to obtain a more accurate motion prediction model. 2) We introduce a Prior Compensation Factor to tackle the forgetting problem of prior knowledge, which can be learned jointly with the prediction model parameters. 3) We obtain an easily optimized and more reasonable objective function through theoretical derivation.
Autoregressive prediction approaches for HMP. Motivated by natural language processing, many researchers have adopted sequence-to-sequence models to exploit the temporal information of pose sequences in HMP, which includes RNNs, Long Short-Term Memory Networks (LSTMs) [15] and Transformer [43]. For instance, ERD [8] combined LSTMs with an encoder-decoder to model the temporal aspect, while Jain et al.[18] proposed Structural-RNN to capture spatiotemporal features of human motion. Martinez et al.[32] applied a sequence-to-sequence architecture for modeling the human motion structure. Aksan et al.[1] used Transformer to autoregressively predict future poses. Sun et al.[40] designed a query-read process to retrieve some motion dynamics from the memory bank. Lucas et al.[27] proposed a GPT-like[35] autoregressive method to generate human poses. Tang et al.[42] combined attention mechanism and LSTM to complete the human motion prediction task. However, autoregressive methods are difficult to train and suffer from error accumulation problem.
Parallel prediction approaches for HMP. Some researchers employed parallel prediction methods to address HMP problem [2, 6, 7, 23, 25, 30, 31, 28]. The works of [21-23] used GCN to encode feature or to decode it, which associates different joints' information. Mao et al.[31] viewed a pose as a fully connected graph and used GCN to extract hidden information between any pair of joints. Martinez et al.[33] devised a transformer-based network to predict human poses. Sofianos et al.[39] proposed a method to extract spatiotemporal features using GCNs. And Ma et al.[28] tried to achieve better prediction results using a progressive manner. Xu et al.[45] used multi-level spatial-temporal anchors to make diverse predictions.
Continual learning. Although Deep Neural Networks (DNNs) have demonstrated impressive performance on specific tasks, their limitations in handling diverse tasks hinder their broader applicability. Therefore, some researchers introduced the concept of Continual Learning (CL)[36] to DNNs to ensure that models retain the knowledge of previous tasks while learning new tasks. Kirkpatrick et al.[20] proposed the Elastic Weight Consolidation (EWC) method to overcome the catastrophic forgetting problem and improve the performance of multi-task problems. Shin et al.[38] introduced a method that addresses catastrophic forgetting in sequential learning scenarios by using a generative model to replay data from past tasks during the training of new tasks. It is important to note that the traditional CL approaches do not account for temporal correlation and are unable to leverage data from previous tasks.
The problem of human motion prediction involves predicting future motion sequences by utilizing previously observed motion sequences. Formally, let  $\mathbf{X}_{1:T_h} = [\mathbf{X}_1,\mathbf{X}_2,\dots ,\mathbf{X}_{T_h}]\in \mathbb{R}^{J\times D\times T_h}$  denotes the observed motion sequence of length  $T_{h}$  where  $\mathbf{X}_i$  indicates motion of time  $i$ , and  $\mathbf{X}_{T_h + 1:T_h + T_p} = [\mathbf{X}_{T_h + 1},\mathbf{X}_{T_h + 2},\dots ,\mathbf{X}_{T_h + T_p}]\in \mathbb{R}^{J\times D\times T_p}$  represents the motion sequence of
length  $T_{p}$  that needs to be predicted. Note that  $J$  is the number of joints for each pose, and  $D$  is the dimension of coordinates. It can be regarded as a composite task consisting of multiple sequential prediction tasks, which involves predicting the future poses at varied moments conditioned on the motion sequences observed in the past.
To accomplish this multiple sequential prediction task, we first model the HMP problem as solving the following optimization problem:
$$
\boldsymbol {\theta} ^ {*} = \underset {\boldsymbol {\theta}} {\arg \max } P \left(\mathbf {X} _ {T _ {h} + 1}, \mathbf {X} _ {T _ {h} + 2}, \dots , \mathbf {X} _ {T _ {h} + T _ {p}} \mid \mathbf {X} _ {1}, \mathbf {X} _ {2}, \dots , \mathbf {X} _ {T _ {h}}; \boldsymbol {\theta}\right). \tag {1}
$$
Thus our target is to find the optimal model that maximizes Equation (1). To achieve this, we propose a framework called Temporal Continual Learning. Specifically, we partition the entire prediction interval into several smaller segments and perform multi-stage training. We claim that this enables the utilization of prior information from previous segments as knowledge for predicting the subsequent segments. Further, as the optimization objective changes in each training stage, we find that the prior knowledge, i.e., information learned in previous training stages, will be forgotten to a certain degree. To mitigate this problem, we introduce a Prior Compensation Factor and accordingly derive a more reasonable optimization objective at each stage.
3.1 Multi-stage Training Process.
We initially decouple the future sequence into  $K$  segments with time boundaries  $T_{Z_1}, T_{Z_2}, \dots, T_{Z_K}$ , where  $T_{Z_K} = T_h + T_p$ . And we denote the prediction of segment  $k$  as task  $Z_k$ , which can be expressed as follows:
- Task  $Z_{1}$  :  $\mathbf{X}_{1:T_{h}}\rightarrow \mathbf{X}_{T_{h} + 1:T_{Z_{1}}}$
Task  $Z_{2}$  ..  $\mathbf{X}_{1:T_h}\to \mathbf{X}_{T_{Z_1} + 1:T_{Z_2}}$
- Task  $Z_{K}$  :  $\mathbf{X}_{1:T_{h}}\rightarrow \mathbf{X}_{T_{Z_{K - 1}} + 1:T_{Z_{K}}}$
To be specific, the target of task  $Z_{1}$  is to predict  $\mathbf{X}_{T_h + 1:T_{Z_1}}$  conditioned on  $\mathbf{X}_{1:T_h}$ , and task  $Z_{k}$  aims to predict  $\mathbf{X}_{T_{Z_{k - 1}} + 1:T_{Z_k}}$  with  $\mathbf{X}_{1:T_h}$  as condition. Therefore, by leveraging bayesian formulation, optimization problem  $P(\mathbf{X}_{T_h + 1},\mathbf{X}_{T_h + 2},\dots ,\mathbf{X}_{T_h + T_p}|\mathbf{X}_1,\mathbf{X}_2,\dots ,\mathbf{X}_{T_h};\pmb {\theta})$  can be formulated as:
$$
P \left(Z _ {1} Z _ {2} \dots Z _ {K}; \boldsymbol {\theta}\right) = P \left(Z _ {K} \mid Z _ {1} Z _ {2} \dots Z _ {K - 1}; \boldsymbol {\theta}\right) P \left(Z _ {K - 1} \mid Z _ {1} Z _ {2} \dots Z _ {K - 2}; \boldsymbol {\theta}\right) \dots P \left(Z _ {1}; \boldsymbol {\theta}\right), \tag {2}
$$
where  $\theta$  is model parameters to be learned. In the following, we denote  $Z_{1}Z_{2}\dots Z_{k}$  as  $Z_{1:k}$ . Our target is to maximize Equation (2) which means finding optimal model to accomplish all tasks.
For the purpose of transferring the prior knowledge in preceding tasks to their subsequent prediction task, we progressively increase the number of tasks in temporal order and train them successively. More precisely, our training is decomposed into  $K$  stages. In each stage  $S_{k}$ , we leverage the optimal model parameters  $\theta_{k-1}^{*}$  trained in the previous stage  $S_{k-1}$  to initialize the parameters  $\theta$ , and then update it based on the prediction tasks  $Z_{1}, Z_{2}, \dots, Z_{k}$  (maximizing  $P(Z_{1:k}; \theta)$ ). Since the model is trained to optimize the prediction tasks  $Z_{1:k-1}$  in training stage  $S_{k-1}$ , the knowledge of tasks  $Z_{1:k-1}$  can be implicitly involved in its well-trained parameters  $\theta_{k-1}^{*}$ . Initializing  $\theta_{k}$  as  $\theta_{k-1}^{*}$  can exploit the prior knowledge learned in previous tasks to assist the prediction of the next task.
3.2 Definition of Prior Compensation Factor.
With training different optimization objectives stage by stage, the prior knowledge provided by previous tasks can be effectively exploited to predict the subsequent task. However, the change of the optimization objective in different training stages could also bring about the knowledge forgetting problem. To mitigate the problem, we introduce  $\alpha_{Z_{1:k-1} \to Z_k}$  to estimate the extent of forgotten knowledge when utilizing prior knowledge from tasks  $Z_{1:k-1}$  to predict task  $Z_k$ , which we refer to as the "Prior Compensation Factor".
$$
\alpha_ {Z _ {1: k - 1} \rightarrow Z _ {k}} = P \left(Z _ {k} \mid Z _ {1: k - 1}; \boldsymbol {\theta}\right) - P \left(Z _ {k} \mid \hat {Z} _ {1: k - 1}; \boldsymbol {\theta}\right). \tag {3}
$$
Here,  $\hat{Z}_{1:k-1}$  is regarded as the prior knowledge that is reserved and can be still provided for predicting task  $Z_k$ . So  $\hat{Z}_{1:k-1}$  initially represents the prior knowledge reserved in  $\pmb{\theta}_{k-1}^*$  in every stage
$S_{k}$  and would get somewhat corrupted gradually during training.  $P(Z_{k}|Z_{1:k - 1};\pmb{\theta})$  indicates the most ideal case, where the current prediction task  $Z_{k}$  can fully leverage the prior information provided by previous prediction tasks  $Z_{1:k - 1}$ . Consequently, the loss of the prior knowledge is non-negative, implying that  $0 \leq \alpha_{Z_{1:k - 1} \to Z_k} \leq 1 - P(Z_k|\hat{Z}_{1:k - 1};\pmb{\theta})$ . Specifically, we can observe that  $\alpha = 0$  when  $P(Z_{k}|Z_{1:k - 1};\pmb{\theta}) = P(Z_{k}|\hat{Z}_{1:k - 1};\pmb{\theta})$ , which implies that all the prior knowledge of previous tasks is completely exploited although the  $\pmb{\theta}$  changes. By substituting Equation (3) into Equation (2) and taking the negative logarithm, we can obtain:
$$
- \log P \left(Z _ {1: k}; \boldsymbol {\theta}\right) = - \log P \left(Z _ {1}; \boldsymbol {\theta}\right) - \sum_ {i = 2} ^ {k} \log \left(P \left(Z _ {i} \mid \hat {Z} _ {1: i - 1}; \boldsymbol {\theta}\right) + \alpha_ {Z _ {1: i - 1} \rightarrow Z _ {i}}\right). \tag {4}
$$
Our objective is to minimize  $-\log P(Z_{1:k};\pmb{\theta})$  with respect to the model parameter  $\pmb{\theta}$  and prior compensation factors  $\{\alpha_{Z_{1:i-1} \to Z_i}, i = 2,3,\dots,k\}$ .
3.3 Optimization Objective
Optimizing Equation (4) directly is challenging due to the presence of the PCF  $\alpha$  and  $P(Z_{k}|\hat{Z}_{1:k - 1};\pmb{\theta})$  inside the logarithm. By applying Lemma 3.1 (details provided in the appendix), we can obtain an upper bound for Equation (4), which can be expressed as:
$$
\begin{array}{l} U B = - \log P \left(Z _ {1}; \boldsymbol {\theta}\right) + \sum_ {i = 2} ^ {k} \left(\left(1 - \alpha_ {Z _ {1: i - 1} \rightarrow Z _ {i}}\right)\left(- \log P \left(Z _ {i} \mid \hat {Z} _ {1: i - 1}; \boldsymbol {\theta}\right)\right)\right. \tag {5} \\ + \left(1 - \alpha_ {Z _ {1: i - 1} \rightarrow Z _ {i}}\right) \log \left(1 - \alpha_ {Z _ {1: i - 1} \rightarrow Z _ {i}}\right) + \log \left(1 + \alpha_ {Z _ {1: i - 1} \rightarrow Z _ {i}}\right)). \\ \end{array}
$$
Hence, we can turn to minimize the upper bound of Equation (4). It appears that in the optimization objective,  $\alpha_{Z_{1:i-1} \to Z_i}$  serves as a factor to control the weights of different tasks, which mitigates the loss of prior information and thus compensates for the lost prior knowledge. The Lemma 3.2 indicates that the largest difference between  $-\log P(Z_{1:k}; \pmb{\theta})$  and the upper bound  $UB$  would not exceed  $\log(3/2) * (k-1)$  when  $P(Z_i | \hat{Z}_{1:i-1}; \pmb{\theta}) \geq 1/2, i \in \{2, 3, \dots, k\}$ .
Lemma 3.1. For  $0 \leq a \leq 1 - b$  and  $0 < b \leq 1$ , the inequality  $-\log (a + b) \leq (1 - a)(-\log b) + (1 - a)\log (1 - a) + \log (1 + a)$  holds. The equality holds if and only if  $a = 0$ .
Lemma 3.2. The absolute difference between the target objective (Equation (4)) and the upper bound (Equation (5)) is not larger than  $\log(3/2) * (k-1)$  when  $P(Z_i | \hat{Z}_{1:i-1}; \boldsymbol{\theta}) \geq 1/2, i \in \{2,3,\dots,k\}$ . This bound is achieved when  $P(Z_k | \hat{Z}_{1:k-1}; \boldsymbol{\theta}) = 1/2$  and  $\alpha_{Z_{1:i-1} \to Z_i} = 1/2, i \in \{2,3,\dots,k\}$ .
Due to the space limitation, we present the proofs of Lemma 3.1 and 3.2 in the appendix.
An intuitive explanation. Figure 3 illustrates the comparison between the term  $-\log (P(Z_k|\hat{Z}_{1:k - 1};\pmb {\theta}) + \alpha_{Z_{1:k - 1}\to Z_k})$  in the actual optimization objective, the term  $-\log P(Z_k|\hat{Z}_{1:k - 1};\pmb {\theta})$  in the naive optimization objective merely leveraging multi-stage training strategy, and the corresponding approximate term in our optimization objective. It can be observed that our approximation method has a smaller difference from the actual optimization objective compared to the naive method. This is important as a more reasonable objective function can improve the accuracy of the optimization process. When the prior compensation factor  $\alpha_{Z_{1:k - 1}\to Z_k}$  is zero, which means  $P(Z_{k}|Z_{1:k - 1};\pmb {\theta}) = P(Z_{k}|\hat{Z}_{1:k - 1};\pmb {\theta})$ , the prior prediction information from previous tasks  $Z_{1:k - 1}$  is not lost. As the value of  $\alpha$  increases, the discrepancy between the actual objective  $P(Z_{k}|Z_{1:k - 1};\pmb{\theta})$  and the objective of the naive method  $P(Z_{k}|\hat{Z}_{1:k - 1};\pmb{\theta})$  becomes more evident, indicating a greater degree of forgetting prior knowledge. In contrast, our approach narrows this gap by effectively mitigating the loss of prior information obtained from tasks  $Z_{1:k - 1}$ .
3.4 Optimization Strategy
We train the model in a multi-stage manner, in which an initial stage and  $K - 1$  TCL stages are involved. The initial stage  $S_{1}$  aims to forecast the motion in the foremost segment, while each TCL stage  $S_{k}, k \in \{2,3,\dots ,K\}$  performs prediction for segments  $Z_{1:k}$  and simultaneously trains
Figure 3: A toy example illustrating the optimization objectives. Our approximate optimization objective is closer to the actual objective compared to the naive optimization method.
$\alpha_{Z_{1:k-1} \to Z_k}$ . Once the stage  $S_k$  is completely trained, we then estimate the factors  $\hat{\alpha}_{Z_{1:k-1} \to Z_k}$  which would be used in the optimization of the following stages. This process is repeated until the final stage  $S_K$  is reached.
Learning of initial stage  $S_{1}$ . Following the implementations of previous methods [19], we can train the initial stage  $Z_{1}$  with Mean Squared Error (MSE) loss:
$$
\mathcal {L} _ {1} = \sum_ {i = T _ {h} + 1} ^ {T _ {Z _ {1}}} \left\| \mathbf {X} _ {i} - \hat {\mathbf {X}} _ {i} \right\| ^ {2} \tag {6}
$$
where  $\mathbf{X}_i$  and  $\tilde{\mathbf{X}}_i$  represent the ground truth and predicted motion of the  $i$ -th frames respectively.
Temporal Continual Learning at stage  $S_{k}$ . In stage  $S_{k}$  ( $k \geq 2$ ), we need to update the model parameters  $\theta$  corresponding to tasks  $Z_{1:k}$  and the PCF  $\alpha_{Z_{1:k-1} \to Z_k}$ . According to Equation (5), the loss function in this stage can be calculated as follows:
$$
\begin{array}{l} \mathcal {L} _ {k} = \left(1 - \alpha_ {Z _ {1: k - 1} \rightarrow Z _ {k}}\right) \sum_ {i = T _ {Z _ {k - 1}} + 1} ^ {T _ {Z _ {k}}} \left\| \mathbf {X} _ {i} - \hat {\mathbf {X}} _ {i} \right\| ^ {2} + \left(1 - \alpha_ {Z _ {1: k - 1} \rightarrow Z _ {k}}\right) \log \left(1 - \alpha_ {Z _ {1: k - 1} \rightarrow Z _ {k}}\right) \\ + \log (1 + \alpha_ {Z _ {1: k - 1} \to Z _ {k}}) + \sum_ {j = 2} ^ {k - 1} (1 - \hat {\alpha} _ {Z _ {1: j - 1} \to Z _ {j}}) \sum_ {i = T _ {Z _ {j - 1}} + 1} ^ {T _ {Z _ {j}}} \left\| \mathbf {X} _ {i} - \hat {\mathbf {X}} _ {i} \right\| ^ {2} + \mathcal {L} _ {1} \\ \end{array}
$$
where the parameters  $\hat{\alpha}_{Z_1\to Z_2},\dots ,\hat{\alpha}_{Z_{1:k - 2}\to Z_{k - 1}}$  are determined in the learning of previous stages. Once the model parameters for stage  $S_{k}$  are determined, we then calculate  $\hat{\alpha}_{Z_{1:k - 1}\to Z_k}$  as:
$$
\hat {\alpha} _ {Z _ {1: k - 1} \rightarrow Z _ {k}} = \frac {1}{M} \sum_ {m = 1} ^ {M} \hat {\alpha} _ {Z _ {1: k - 1} \rightarrow Z _ {k}} \tag {8}
$$
where  $M$  represents number of samples and  $\hat{\alpha}_{Z_{1:k-1} \to Z_k}^m$  is PCF estimated for the  $m$ -th sample.
We continue the TCL training process by predicting stage  $S_{k + 1}$  and updating the model parameter  $\theta$  as well as PCF  $\alpha_{Z_{1:k}\to Z_{k + 1}}$ . We repeat this TCL process until we reach the final stage  $S_K$ . In practice, we require the backbone model to output an extra dimension and pass it through an MLP head to obtain  $\alpha$ . The algorithm flow is summarized in Algorithm 1.
Algorithm 1 Training procedure of proposed TCL framework.
Require: observed frames  $\mathbf{X}_{1:T_h}$ , ground truth future frames  $\mathbf{X}_{T_h + 1:T_h + T_p}$ , model parameters  $\theta$ , stage number  $K$ , training epoch for  $k$ -th stage  $E_k$ , learning rate  $\lambda$ , training sample number  $M$ .  
for  $i = 1$  to  $E_1$  do  
 $\hat{\mathbf{X}}_{T_h + 1:T_{Z_1}} = \mathbf{f}_\theta (\mathbf{X}_{1:T_h})$ $\theta \leftarrow \theta - \lambda * \nabla_\theta \mathcal{L}_1(\mathbf{X}_{T_h + 1:T_{Z_1}}, \hat{\mathbf{X}}_{T_h + 1:T_{Z_1}})$   
end for  
 $\mathbf{A} = \emptyset$   
for  $k = 2$  to  $K$  do  
for  $i = 1$  to  $E_k$  do  
 $\hat{\mathbf{X}}_{T_h + 1:T_{Z_k}}, \alpha_{Z_{1:k-1}} \rightarrow Z_k = \mathbf{f}_\theta (\mathbf{X}_{1:T_h})$ $\theta \leftarrow \theta - \lambda * \nabla_\theta \mathcal{L}_k(\mathbf{X}_{T_h + 1:T_{Z_k}}, \hat{\mathbf{X}}_{T_h + 1:T_{Z_k}}, \alpha_{Z_{1:k-1}} \rightarrow Z_k, \mathbf{A})$   
end for  
 $\hat{\alpha}_{Z_{1:k-1}} = 0$   
for  $m = 1$  to  $M$  do  
 $\alpha_{Z_{1:k-1}}^m = \mathbf{f}_\theta (\mathbf{X}_{1:T_h}^m)$ $\hat{\alpha}_{Z_{1:k-1}} = \hat{\alpha}_{Z_{1:k-1}} \rightarrow Z_k + \alpha_{Z_{1:k-1}}^m \rightarrow Z_k$   
end for  
 $\hat{\alpha}_{Z_{1:k-1}} = \frac{1}{M} \hat{\alpha}_{Z_{1:k-1}} \rightarrow Z_k$ $\mathbf{A} = \mathbf{A} \cup \{\hat{\alpha}_{Z_{1:k-1}} \rightarrow Z_k\}$   
end for