# Supervised Dynamic Correlated Topic Model for Classifying Categorical Time Series
## 2 Supervised Dynamic Correlated Topic Model  
Topic models are traditionally developed by treating words as interchangeable to identify semantic themes within each document. Our method aims to develop a family of probabilistic time series models to analyze the evolution of words over time within document collections. We assume that each document, represented by a categorical time series of words, arises from a generative process that includes latent variables. sDCTM provides a supervised framework for time series classification, allowing it to model class labels associated with each word time series. Alternatively, by removing the response component of the generative process, we can derive an unsupervised dynamic correlated topic model (DCTM). Suppose we have a corpus \(\mathcal{C}\) consisting of \(M\) documents. We represent the \(d\)th document as \(\mathcal{D}_{d}=(\mathbf{w}_{d,1},\mathbf{w}_{d,2},\ldots,\mathbf{w}_{d,T})\), where \(\mathbf{w}_{d,t}\) corresponds to a word observed at time point \(t\) on the \(d\)th document for times \(t=1,2,\ldots,T\). Each word is represented by a \(V\)-dimensional unit (basis) vector, i.e., \(\mathbf{w}_{d,t}=(w^{1}_{d,t},\ldots,w^{V}_{d,t})^{\prime}\) such that,  
\[w^{u}_{d}= \begin{cases}1&\text{if }u=v,\\ 0&u\
eq v,\end{cases}  \tag{1}\]  
when \(\mathbf{w}_{d,t}\) corresponds to the \(v\)th word from the vocabulary for \(t=1,\ldots,T\) and \(V\) denote the number of levels associated with the categorical word time series (document). Additionally, let \(\mathbf{y}_{d}\) be a multinomial response associated with \(d\)th document for \(d=1,2,\ldots,M\). Each \(\mathbf{y}_{d}\) is represented by a \(C\times 1\) unit (basis) vector, i.e., \(\mathbf{y}_{d}=(y_{d,1},\ldots,y_{d,C})^{\prime}\) such that,  
\[y^{g}_{d}= \begin{cases}1&\text{if }g=c,\\ 0&g\
eq c,\end{cases}  \tag{2}\]  
where \(\mathbf{y}_{d}\) corresponds to the \(c\)th class for \(c\in\{1,2,\ldots C\}\), where \(C\) denotes the number of levels associated with the response variable. Suppose \(J\) is the number of assumed latent topics for capturing the hidden thematic structure in the corpus; then, \(\delta_{d,t}\) is a \(J\) dimensional vector corresponding to the latent topic structure for the \(d\)th document at time \(t\). In the promoter identification data, we treat each DNA sub-sequence to be a word time series (document) with \(V=4\) levels (indicating nucleotides \(a,g,c\), and \(t\)). The response variable \(\mathbf{y}_{d}\) for \(d=1,2,\ldots,M\) indicates the presence/absence of a promoter region with \(C=2\) levels. The number of assumed topics \(J\) in the sDCTM model capture the hidden patterns within the observed DNA sub-sequences. Under the sDCTM framework, the \(d\)th document (DNA sub-sequence) \(\mathcal{D}_{d}\) and its associated class label (presence/absence of a promoter) \(\mathbf{y}_{d}\) for \(d=1,2,\ldots,M\) arises from the following generative process. For \(d=1,2,\ldots,M\),  
* For \(t=1,2,\ldots,T\) Choose \(\delta_{d,t}\mid\delta_{d,t-1}\sim N(\mathbf{\Phi}\delta_{d,t-1},\mathbf{\Sigma})\). Choose a topic \(\mathbf{Z}_{\{d,t\}}\mid\delta_{d,t}\sim Mult(1,f(\delta_{d,t}))\), where \[f(\delta_{d,t,j})=\frac{\exp(\delta_{d,t,j})}{\sum_{l=1}^{J}\exp( \delta_{d,t,l})}\text{ for }j=1,2,\ldots,J.\] Choose a word \(\mathbf{w}_{d,t}\mid\{\mathbf{Z}_{\{d,t\}},\mathbf{\beta}\}\sim Mult(1,\mathbf{\beta}_{\{\mathbf{ z}_{\{d,t\}}}})\). * Draw class label \(\mathbf{y}_{d}\mid\mathbf{Z}_{d,1:T},\mathbf{\eta}\sim softmax(\mathbf{\tilde{z}}_{d},\mathbf{\eta})\), where \[\mathbf{\tilde{z}}_{d}=\frac{1}{T}\sum_{t=1}^{T}\mathbf{Z}_{\{d,t\}},\] is the topic frequencies for the \(d\)th document. The per-word topic indicator for each word \(\mathbf{w}_{d,t}\) is represented by \(\mathbf{Z}_{d,t}\) for \(t=1,\ldots,T\) and \(d=1,\ldots,M\).