SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Xiangchen Song $^{1*}$  Weiran Yao $^{2}$  Yewen Fan $^{1}$  Xinshuai Dong $^{1}$  Guangyi Chen $^{1,3}$
Juan Carlos Niebles² Eric Xing<sup>1,3</sup> Kun Zhang<sup>1,3</sup>
$^{1}$ Carnegie Mellon University
2Salesforce Research
<sup>3</sup>Mohamed bin Zayed University of Artificial Intelligence
In unsupervised causal representation learning for sequential data with time-delayed latent causal influences, strong identifiability results for the disentanglement of causally-related latent variables have been established in stationary settings by leveraging temporal structure. However, in nonstationary setting, existing work only partially addressed the problem by either utilizing observed auxiliary variables (e.g., class labels and/or domain indexes) as side-information or assuming simplified latent causal dynamics. Both constrain the method to a limited range of scenarios. In this study, we further explored the Markov Assumption under time-delayed causally related process in nonstationary setting and showed that under mild conditions, the independent latent components can be recovered from their nonlinear mixture up to a permutation and a component-wise transformation, without the observation of auxiliary variables. We then introduce NCTRL, a principled estimation framework, to reconstruct time-delayed latent causal variables and identify their relations from measured sequential data only. Empirical evaluations demonstrated the reliable identification of time-delayed latent causal influences, with our methodology substantially outperforming existing baselines that fail to exploit the nonstationarity adequately and then, consequently, cannot distinguish distribution shifts.
Causal reasoning for time-series data is a long-lasting yet fundamental task [1-3]. The majority of the studies focus on the temporal causal discovery among observed variables [4-6]. However, in many real-world scenarios, the observed data (e.g., image pixels in videos) instead of having direct causal edges, are generated by some causally related latent temporal processes or confounders. Learning causal relations has practical use cases, which benefit a lot of downstream tasks. However, estimating latent causal structures among those unobserved variables purely from observations without appropriate class of assumptions is an extremely challenging task (i.e. the latent variables are generally not identifiable) [7, 8].
Under the topic of unsupervised representation learning via nonlinear Independent Component Analysis (ICA), some strong identifiability results of the latent variables have been established [9-14] by introducing side information such as class labels and domain indices. Specifically focusing on time-series data, history information is also widely used as the side information for the identifiability of latent processes [15-18]. However, existing studies mainly focused on and derived identifiability results in stationary settings [10, 16] (Fig 1 (a)) or nonstationary settings with explicitly observed domain indices [12, 17, 18] (Fig 1 (b)).
*Part of the work was done while interning at Salesforce Research
(c)
(d)
Figure 1: Graphic models for three different settings in causally related time-delayed time series data with a visual illustration. (a) is a stationary setting in which the transition function  $\mathbf{z}_{t+1} = f_z(\mathbf{z}_t)$  stays universally the same. (b) is the setting widely explored in existing work, in which the transition function  $f_z$  changes according to different domains (denoted as  $c_t$ ), and all those domain indices are observed. (c) capture the unobserved domain indices by introducing a Markov chain on  $c_t$ . (d) is a more general form to model the time series data in this work. It allows nonstationary settings and it doesn't require the domain indices to be observed.
One can immediately tell the infeasibility of those two scenarios that general time-series data is usually nonstationary and the side information (class labels and domain indices) is usually unobserved. That is particularly true when considering real-world data such as video or signal sequences. It doesn't make any sense to assume that there exists a stationary transition function that is applied to the whole video clip. Take a very simple video clip of a mouse² [19] as an example, it is fairly clear that such a simple motion example can be divided into at least two phases (1) active phase in which the mouse is moving and (2) inactive phase in which the mouse is laying down. Instead of using a complex transition function to describe the whole video clip, a more reasonable assumption is that the same transition function is shared within the same phase, but across different phases, the transition functions are different, in other words, the transition function can be expressed as a function of the domain index. Also, it is worth mentioning that if such domain or phase indices is latent or unobserved, then we cannot directly utilize the existing framework to learn the latent causal dynamics. That is again a more realistic case that in general, the domain indices within a video are not accessible without expensive human annotation.
Recently, HMNLICA [14] attempted to resolve the problem by introducing Markov Assumption on the nonstationary discrete domain variable, they assumed the domain indices follow a first-order Markov Chain and estimated the domain information purely from observed data. However, HMNLICA assumes temporally mutually independent sources in the data-generating process (conditioning on domain indices), i.e. they don't allow latent variables to have time-delayed causal relations in between (Fig 1 (c)). Such an assumption imposed a huge negative impact on the usability of those methods. Considering the video of the little mouse example, the  $\mathbf{x}_t\mathbf{s}$  are the observed video frames,  $\mathbf{z}_t\mathbf{s}$  can be the independent motion dynamics or causal process such as position, velocity, (angular) momentum, etc, and  $c_{t}s$  are the phases or actions such as standing up (active) and laying down (inactive). To accommodate for such general sequential data, time-delayed temporal dependence should be considered in the latent  $\mathbf{z}_t$  space (Fig 1 (d)), otherwise, it is impossible to model a complex video data's temporal relation purely from discrete, domain indices. Also to make sure that the latent independent components can be recovered, temporally conditional independence should also be enforced, i.e. Each dimension of  $\mathbf{z}_t$  is conditionally independent given the history  $\mathbf{z}_{\mathrm{history}}$ . To this end, a natural question is:
How can we establish identifiability of nonlinear ICA for general sequential data with nonstationary causally-related process without observing auxiliary variable?
To answer this question, we first formulate the latent nonstationary states as a discrete Markov process and further explore the Markov Assumption [20] which is introduced for identifiability of nonlinear ICA in HMNLICA [14] and provided stronger identifiability result corresponding to the conditional emission distribution (i.e. the transition function of different domains) and the transition matrix of
$^{2}$ https://dattalab.github.io/moseq2-website/images/sample-extraction.gif.
the Markov process. Specifically, we generalized the identifiability of Hidden Markov Models in [20] to accommodate time-delayed causally-related non-parametric transitions in latent space (Thm. 1). Then we utilize the linear independence (Thm. 2) to further establish the identifiability of  $\mathbf{z}_t$ .
- To our best knowledge, this is the first identifiability result that can handle the nonstationary time-delayed causally-related latent temporal processes without the auxiliary variable. We formulate the problem, especially the nonstationary states into the Markov process, establish identifiability purely from observed data, and then show strong identifiability of latent independent components.
- We present NCTRL, Nonstationary Causal Temporal Representation Learning, a principled framework to recover time-delayed latent causal variables and identify their relations from measured sequential data under unobserved different distribution shifts.
- Experiments on both synthetic and real-world datasets demonstrate the effectiveness of the proposed method in recovering the latent variables.
2.1 Time Series Generative Model
Assume we observe  $n$ -dimensional time-series data at discrete time steps,  $\mathbf{X} = \{\mathbf{x}_1,\mathbf{x}_2,\dots ,\mathbf{x}_T\}$ , where each  $\mathbf{x}_t\in \mathcal{X}$  is generated from time-delayed causally related hidden components  $\mathbf{z}_t\in \mathbb{R}^n$  by the invertible mixing function:
$$
\mathbf {x} _ {t} = \mathbf {g} \left(\mathbf {z} _ {t}\right). \tag {1}
$$
In addition to latent components  $\mathbf{z}_t$ , there is an extra hidden variable  $c_t$  which is discrete with cardinality  $|c_t| = C$ , it follows a first-order Markov process controlled by a  $C\times C$  transition matrix  $\mathbf{A}$ , in which the  $i,j$ -th entry  $A_{i,j}$  is the probability to transit from state  $i$  to  $j$ .
$$
c _ {1}, c _ {2}, \dots , c _ {t} \sim \operatorname {M a r k o v C h a i n} (\mathbf {A}) \tag {2}
$$
For  $i \in \{1, \dots, n\}$ ,  $z_{it}$ , as the  $i$ -th component of  $\mathbf{z}_t$ , is generated by (some) components of history information  $\mathbf{z}_{t-1}$ , discrete nonstationary indicator  $c_t$ , and noise  $\epsilon_{it}$ .
$$
z _ {i t} = f _ {i} \left(\left\{z _ {j, t - 1} \mid z _ {j, t - \tau} \in \mathbf {P a} (z _ {i t}) \right\}, c _ {t}, \epsilon_ {i t}\right) \quad w i t h \quad \epsilon_ {i t} \sim p _ {\epsilon_ {i}} \tag {3}
$$
where  $\mathbf{Pa}(z_{it})$  is the set of latent factors that directly cause  $z_{it}$ , which can be any subset of  $\mathbf{z}_{\mathrm{Hx}} = \{\mathbf{z}_{t-1}, \mathbf{z}_{t-2}, \ldots, \mathbf{z}_{t-L}\}$  up to history information maximum lag  $L$ . The components of  $\mathbf{z}_t$  are mutually independent conditional on  $\mathbf{z}_{\mathrm{Hx}}$  and  $c_t$ .
2.2 Identifiability of Latent Causal Processes and Time-Delayed Latent Causal Relations
We define the identifiability of time-delayed latent causal processes in the representation function space in Definition 1. Furthermore, if the estimated latent processes can be identified at least up to permutation and component-wise invertible nonlinearities, the latent causal relations are also immediately identifiable because conditional independence relations fully characterize time-delayed causal relations in a time-delayed causally sufficient system, in which there are no latent causal confounders in the (latent) causal processes. Note that invertible component-wise transformations on latent causal processes do not change their conditional independence relations.
Definition 1 (Identifiable Latent Causal Processes). Formally let  $\mathbf{X} = \{\mathbf{x}_1, \mathbf{x}_2, \dots, \mathbf{x}_T\}$  be a sequence of observed variables generated by the true temporally causal latent processes specified by  $(f_i, p(\epsilon_i), \mathbf{A}, \mathbf{g})$  given in Eqs. (1), (2), and (3). A learned generative model  $(\hat{f}_i, \hat{p}(\epsilon_i), \hat{\mathbf{A}}, \hat{\mathbf{g}})$  is observationally equivalent to  $(f_i, p(\epsilon_i), \mathbf{A}, \mathbf{g})$  if the model distribution  $p_{\hat{g}, \hat{p}_{\epsilon}, \hat{\mathbf{A}}, \hat{\mathbf{g}}}(\{\mathbf{x}_1, \mathbf{x}_2, \dots, \mathbf{x}_T\})$  matches the data distribution  $p_{f_i, p_{\epsilon}, \mathbf{A}, \mathbf{g}}(\{\mathbf{x}_1, \mathbf{x}_2, \dots, \mathbf{x}_T\})$  everywhere. We say latent causal processes are identifiable if observational equivalence can lead to identifiability of the latent variables up to permutation  $\pi$  and component-wise invertible transformation  $T$ :
$$
\begin{array}{l} p _ {\hat {f} _ {i}, \hat {p} _ {\epsilon_ {i}}, \hat {\mathbf {A}}, \hat {\mathbf {g}}} (\{\mathbf {x} _ {1}, \mathbf {x} _ {2}, \dots , \mathbf {x} _ {T} \}) = p _ {f _ {i}, p _ {\epsilon_ {i}}, \mathbf {A}, \mathbf {g}} (\{\mathbf {x} _ {1}, \mathbf {x} _ {2}, \dots , \mathbf {x} _ {T} \}) \tag {4} \\ \Rightarrow \hat {\mathbf {g}} ^ {- 1} (\mathbf {x} _ {t}) = T \circ \pi \circ \mathbf {g} ^ {- 1} (\mathbf {x} _ {t}), \quad \forall \mathbf {x} _ {t} \in \mathcal {X}, \\ \end{array}
$$
where  $\mathcal{X}$  is the observation space.
In this section, we showed that under mild conditions, the latent variable  $\mathbf{z}_t$  is identifiable up to permutation and a component-wise transformation. The theoretical results can be divided into two parts (1) identifiability of the nonstationarity and (2) identifiability of the independent components. As introduced above, the major challenge comes from the unobserved domain indices or nonstationary indicators ( $c_t$  in our graphic models). We first establish the identifiability of the different conditional distributions from the observed data and then show that the latent variables  $\mathbf{z}$  are identifiable. The complete proofs can be found in Appendix A.
3.1 Identifiability of Nonstationary Hidden States
Gassiat et al.[20] showed that the conditional emission distributions in Hidden Markov Models and the transition matrix are identifiable up to label swapping. We first generalize it to the autoregressive setting to accommodate for the time-delayed causal relation, i.e. we showed the identifiability of conditional emission distributions  $p(\mathbf{x}_t|\mathbf{x}_{t - 1},c)$ .
Theorem 1. (identifiability of the nonstationarity with Markov Assumptions) Suppose the observed data is generated following the nonlinear ICA framework as defined in Eqs. (1), (2) and (3). Suppose the following assumptions (Markov Assumptions) hold:
i For the Markov process, the number of latent states,  $C$ , is known.
ii The transition matrix  $\mathbf{A}$  is full rank.
Use  $\mu_1, \ldots, \mu_C \in \mathbb{R}^n$  to denote nonparametric probability distributions of the  $C$  emission distributions  $\mu_c = p(\mathbf{x}_t | \mathbf{x}_{t-1}, c)$ . Then the parameters  $\mathbf{A}$  and  $M = (\mu_1, \ldots, \mu_C)$  are identifiable given the distribution,  $\mathbb{P}_{\mathbf{A}, M}^{(4)}$ , of at least 4 consecutive observations  $\mathbf{x}_t, \mathbf{x}_{t+1}, \mathbf{x}_{t+2}, \mathbf{x}_{t+3}$ , up to label swapping of the hidden states, that is:
If  $\widetilde{\mathbf{A}}$  is a  $C\times C$  transition matrix and if  $\widetilde{\pi} (c)$  is a stationary distribution of  $\widetilde{\mathbf{A}}$  with  $\widetilde{\pi} (c) > 0$ $\forall c\in \{1,\ldots ,C\}$ , and if  $\tilde{M} = (\tilde{\mu}_1,\dots ,\tilde{\mu}_C)$  are  $C$  probability distributions on  $\mathbb{R}^n$  that verify the equality of the distribution functions  $\mathbb{P}_{\widetilde{\mathbf{A}},\tilde{M}}^{(4)} = \mathbb{P}_{\mathbf{A},M}^{(4)}$ , then there exists a permutation  $\sigma$  of the set  $\{1,\ldots ,C\}$  such that for all  $k,l = 1,\ldots ,C$  we have  $\tilde{A}_{k,l} = A_{\sigma (k),\sigma (l)}$  and  $\tilde{\mu}_k = \mu_{\sigma (k)}$ .
For notational simplicity, and without loss of generality, we can assume the components are ordered such that  $c = \sigma(c)$ . That leads us to the identifiability of the nonstationarity in the system i.e. up to label swapping of the hidden states, the conditional emission distributions  $p(\mathbf{x}_t | \mathbf{x}_{t-1}, c_t)$  and transition matrix  $\mathbf{A}$  are identifiable, hence providing us a bridge to further leverage the temporal independence condition in the latent space to establish the identifiability result for demixing function or in other words the latent variables  $\mathbf{z}_t$ .
3.2 Identifiability of Latent Causal Processes
To incorporate nonlinear ICA into the Markov Assumption we define the emission distribution  $p(\mathbf{x}_t \mid \mathbf{x}_{t-1}, c)$  as a deep latent variable model. First, the latent independent component variables  $\mathbf{z}_t \in \mathbb{R}^n$  are generated from a factorial prior, given the hidden state  $c_t$  and previous  $\mathbf{z}_{t-1}$ , as
$$
p \left(\mathbf {z} _ {t} \mid \mathbf {z} _ {t - 1}, c _ {t}\right) = \prod_ {k = 1} ^ {n} p \left(z _ {k t} \mid \mathbf {z} _ {t - 1}, c _ {t}\right). \tag {5}
$$
Second, the observed data  $\mathbf{x}_t$  is generated by a nonlinear mixing function as in Eq. (1) which is assumed to be bijective with inverse given by  $\mathbf{z}_t = \mathbf{g}(\mathbf{x}_t)$ . Let  $\eta_{kt}(c_t) \triangleq \log p(z_{kt}|\mathbf{z}_{t-1}, c_t)$ , and assume that  $\eta_{kt}(c_t)$  is twice differentiable in  $z_{kt}$  and is differentiable in  $z_{l,t-1}, l = 1, 2, \ldots, n$ . Note that the parents of  $z_{kt}$  may be only  $c_t$  and a subset of  $\mathbf{z}_{t-1}$ ; if  $z_{l,t-1}$  is not a parent of  $z_{kt}$ , then  $\frac{\partial \eta_k}{\partial z_{l,t-1}} = 0$ .
Theorem 2. (identifiability of the independent components) Suppose there exists an invertible function  $\hat{\mathbf{g}}^{-1}$ , which is the estimated demixing function that maps  $\mathbf{x}_t$  to  $\hat{\mathbf{z}}_t$ , i.e.,
$$
\hat {\mathbf {z}} _ {t} = \hat {\mathbf {g}} ^ {- 1} \left(\mathbf {x} _ {t}\right) \tag {6}
$$
such that the components of  $\hat{\mathbf{z}}_t$  are mutually independent conditional on  $\hat{\mathbf{z}}_{t - 1}$ . Let
$$
\mathbf {v} _ {k, t} (c) \triangleq \left(\frac {\partial^ {2} \eta_ {k t} (c)}{\partial z _ {k , t} \partial z _ {1 , t - 1}}, \frac {\partial^ {2} \eta_ {k t} (c)}{\partial z _ {k , t} \partial z _ {2 , t - 1}}, \dots , \frac {\partial^ {2} \eta_ {k t} (c)}{\partial z _ {k , t} \partial z _ {n , t - 1}}\right) ^ {\intercal},
$$
$$
\mathbf {\hat {v}} _ {k, t} (c) \triangleq \left(\frac {\partial^ {3} \eta_ {k t} (c)}{\partial z _ {k , t} ^ {2} \partial z _ {1 , t - 1}}, \frac {\partial^ {3} \eta_ {k t} (c)}{\partial z _ {k , t} ^ {2} \partial z _ {2 , t - 1}}, \dots , \frac {\partial^ {3} \eta_ {k t} (c)}{\partial z _ {k , t} ^ {2} \partial z _ {n , t - 1}}\right) ^ {\intercal}.
$$
$$
\mathbf {s} _ {k t} \triangleq \left(\mathbf {v} _ {k t} (1) ^ {\intercal},..., \mathbf {v} _ {k t} (C) ^ {\intercal}, \frac {\partial^ {2} \eta_ {k t} (2)}{\partial z _ {k t} ^ {2}} - \frac {\partial^ {2} \eta_ {k t} (1)}{\partial z _ {k t} ^ {2}},..., \frac {\partial^ {2} \eta_ {k t} (C)}{\partial z _ {k t} ^ {2}} - \frac {\partial^ {2} \eta_ {k t} (C - 1)}{\partial z _ {k t} ^ {2}}\right) ^ {\intercal},
$$
$$
\mathring {\mathbf {s}} _ {k t} \triangleq \left(\mathring {\mathbf {v}} _ {k t} (1) ^ {\intercal}, \dots , \mathring {\mathbf {v}} _ {k t} (C) ^ {\intercal}, \frac {\partial \eta_ {k t} (2)}{\partial z _ {k t}} - \frac {\partial \eta_ {k t} (1)}{\partial z _ {k t}}, \dots , \frac {\partial \eta_ {k t} (C)}{\partial z _ {k t}} - \frac {\partial \eta_ {k t} (C - 1)}{\partial z _ {k t}}\right) ^ {\intercal}.
$$
If for each value of  $\mathbf{z}_t$ ,  $\mathbf{s}_{1t}, \mathbf{\dot{s}}_{1t}, \mathbf{v}_{2t}, \mathbf{\dot{s}}_{2t}, \dots, \mathbf{s}_{nt}, \mathbf{\dot{s}}_{nt}$ , as  $2n$  function vectors  $\mathbf{s}_{k,t}$  and  $\mathbf{\dot{s}}_{k,t}$ , with  $k = 1, 2, \dots, n$ , are linearly independent, then  $\hat{\mathbf{z}}_t$  must be an invertible, component-wise transformation of a permuted version of  $\mathbf{z}_t$ .
So far, the identifiability result has been established without observing the nonstationarity indicators such as domain indices. In the next section, a novel Variational Auto-Encoder based method is introduced to estimate the demixing function  $\hat{\mathbf{g}}^{-1}$ .
4 NCTRL: Nonstationary Causal Temporal Representation Learning
In this section, we present the details of NCTRL to estimate the latent causal processes under unobserved nonstationary distribution shift, given the identifiability results in Sec 3. First, we show that our framework includes three modules, Autoregressive Hidden Markov Module, Prior Network, and Encoder-Decoder Module. Then, we provide the optimization objective of our model training including an HMM free energy lower bound, a reconstruction likelihood loss, and a KL divergence.
Our framework extends Sequential Variational Auto-Encoders [21] with tailored modules to model nonstationarity, and enforces the conditions in Sec. 3 as constraints. We give the estimation procedure of the latent causal dynamics model in Eq. (3). The model architecture is showcased in Fig. 2. The framework has three major components (1) Autoregressive Hidden Markov Module (ARHMM), (2) Prior Network Module, and (3) Encoder-Decoder Module.
Figure 2: Illustration of NCTRLwith (1) Autoregressive Hidden Markov Module, (2) Prior Network, and (3) Encoder-Decoder Module.
Autoregressive Hidden Markov Module (ARHMM) The first component of our framework is ARHMM which deals with the nonstationarity with unobserved domains. As discussed in Thm 1, the transition function or the conditional emission distributions across different domains together with the Markov transition matrix  $\mathbf{A}$  are identifiable. This module estimates the transition function of different domains  $p(\mathbf{x}_t|\mathbf{x}_{t - 1},c_t)$  and the transition matrix  $\mathbf{A}$  of the Markov process, and ultimately decodes the optimal domain indices  $\{\hat{c}_1,\hat{c}_2,\dots ,\hat{c}_T\}$  via the Viterbi algorithm.
Prior Network Module To better estimate the prior distribution  $p(\hat{z}_t|\hat{\mathbf{z}}_{\mathrm{Hx}},c_t)$ , let  $\mathbf{z}_{\mathrm{Hx}}$  denote the lagged latent variables up to maximum time lag  $L$ . We evaluate  $p(\hat{z}_t|\hat{\mathbf{z}}_{\mathrm{Hx}},c_t) = p_\epsilon \left(\hat{f}_z^{-1}(\hat{z}_t,\hat{\mathbf{z}}_{\mathrm{Hx}},\hat{\boldsymbol{\theta}}_{c_t})\right)\left|\frac{\partial\hat{f}_z^{-1}}{\partial\hat{z}_t}\right|$  by learning a holistic inverse dynamics  $\hat{f}_z^{-1}$  that takes the estimated change factors for dynamics  $\hat{\boldsymbol{\theta}}_{c_t}$  as inputs. Conditional independence of the estimated latent variables  $p(\hat{\mathbf{z}}_t|\hat{\mathbf{z}}_{\mathrm{Hx}})$  is enforced by summing up all estimated component densities when obtaining the joint  $p(\mathbf{z}_t|\mathbf{z}_{\mathrm{Hx}},c_t)$  in Eq. 9. Given that the Jacobian is lower-triangular, we can compute its determinant as
the product of diagonal terms. The detailed derivations are given in Appendix B.2.
$$
\log p \left(\hat {\mathbf {z}} _ {t} \mid \hat {\mathbf {z}} _ {\mathrm {H x}}, c _ {t}\right) = \underbrace {\sum_ {i = 1} ^ {n} \log p \left(\hat {\epsilon} _ {i} \mid c _ {t}\right)} _ {\text {C o n d i t i o n a l i n d e p d e n c e}} + \underbrace {\sum_ {i = 1} ^ {n} \log \left| \frac {\partial \hat {f} _ {i} ^ {- 1}}{\partial \hat {z} _ {i t}} \right|} _ {\text {L o w e r - t r i a n g u l a r J a c o b i a n}} \tag {9}
$$
Encoder-Decoder Module The third component is a Variational Auto-Encoder based module which utilizes reconstruction loss to enforce the invertibility of learned mixing function  $\hat{\mathbf{g}}$ . Specifically, the encoder fits the demixing function  $\hat{\mathbf{g}}^{-1}$  and the decoder fits the mixing function  $\hat{\mathbf{g}}$ . The implementation details are in Appendix B.
The first training objective of NCTRL is to maximize the Log-likelihood of the observed data:
$$
\log p _ {\boldsymbol {\theta} _ {\mathrm {H M M}}} \left(\left\{\mathbf {x} _ {1}, \mathbf {x} _ {2}, \dots , \mathbf {x} _ {T} \right\}\right) \tag {10}
$$
where  $\theta_{\mathrm{HMM}}$  represents the HMM training parameters. Then the free energy lower bound can be defined as:
$$
- \mathcal {L} _ {\mathrm {H M M}} = \mathcal {L} (q (\mathbf {c}), \boldsymbol {\theta} _ {\mathrm {H M M}}) \triangleq \mathbb {E} _ {q (\mathbf {c})} \left[ \log p _ {\boldsymbol {\theta} _ {\mathrm {H M M}}} \left(\mathbf {x} _ {1}, \mathbf {x} _ {2}, \dots , \mathbf {x} _ {T}, \mathbf {c}\right) \right] - \mathbf {H} (q (\mathbf {c})) \tag {11}
$$
Consistent with the theory part, the first training objective is to maximize data log-likelihood in the ARHMM module to get optimal  $q(\mathbf{c}^{\star})$ .
$$
q \left(\mathbf {c} ^ {\star}\right) \triangleq \underset {q (\mathbf {c})} {\arg \max } \mathcal {L} (q (\mathbf {c}), \boldsymbol {\theta} _ {\mathrm {H M M}}) \tag {12}
$$