# A Novel Bi-Dual Inference Approach for Detecting Six-Element Emotions
## 3 Bi-Dual Inference for Detecting Emotions
### Primal Model Construction  
To enhance the primal task model \(f\), we introduce a classical LSTM-based emotion classification method that utilizes a long short-term memory network (2016). This approach involves supervised learning within the previous sequence-to-sequence learning framework. The methodology employs an LSTM model to sequentially process the input sequence, encoding it word-by-word according to Equations (2)-(6). This process allows for the extraction of a fixed-dimensional vector representation. Subsequently, another LSTM model is employed to extract the output from the vector sequence. This second LSTM model functions as a recursive neural network language model, adapting to the input sequence. The modified sequence-to-sequence learning model, as depicted in Figure 1, aims to reconstruct the input sequence itself by replacing it with the output sequence within the sequence framework. In this sequence auto-encoder, a recurrent network reads the input sequence, generates a hidden state, and reconstructs the original sequence. Notably, the decoder and encoder networks share the same weights, as illustrated in Figure 1.  
In the context of LSTM, our objective is to estimate the conditional probability \(p(y_{t}|v,y_{1},\cdots,y_{t-1})\). Here, \(v=(x_{1},\cdots,x_{T})\) represents the input sequence, and \((y_{1},\cdots,y_{T^{\prime}})\) denotes the corresponding output sequence, which may have a different length \(T^{\prime}\) compared to \(T\). The LSTM achieves this by first obtaining a fixed-dimensional representation \(v\) of the input sequence \((x_{1},\cdots,x_{T})\), which is derived from the last hidden state of the LSTM. Subsequently, the LSTM computes the probability of \((y_{1},\cdots,y_{T^{\prime}})\) using a standard LSTM formulation, where the initial hidden state is set to the representation \(v=(x_{1},\cdots,x_{T})\). \[p(y_{1},\cdots,y_{T^{\prime}}|x_{1},\cdots,x_{T})=\mathop{\Pi}\limits_{t=1}^{ T^{\prime}}p(y_{t}|v,y_{1},\cdots,y_{t-1}). \tag{1}\]  
In Equation (1), each conditional probability distribution \(p(y_{t}|v,y_{1},\cdots,y_{t-1})\) is represented using a softmax function over all words in the vocabulary. The cyclic network of the automatic sequence encoder is employed to process the input sequence and generate a hidden state, which is then utilized to reconstruct the original sequence. In other words, by combining Equation (1) with the basic LSTM formulas (Equations (2)-(6)), the LSTM model first reads the input sentence \"\(WXYZ\)\" to calculate the vector representations of \"W\", \"X\", \"Y\", \"Z\", and \"eos\". Here, \"eos\" is a special symbol denoting the end of the sentence, allowing the model to define distributions for sentences of varying lengths. Subsequently, the obtained vector representations are used to calculate the probabilities of \"\(W^{\prime\prime}\)\", \"\(X^{\prime\prime}\)\", \"\(Y^{\prime\prime}\)\", and \"Z\", resulting in the generation of a new sentence \"\(W^{\prime}X^{\prime}Y^{\prime}Z^{\prime\prime}\)\". This generated sentence is then read into the hidden state as the input sequence for the next step, facilitating the reconstruction of the original sequence. Notably, LSTM reads the input statements in reverse order, which enhances the modeling of short-term dependencies and simplifies the optimization problem. \[i_{t}=\sigma\big{(}W_{x_{t}}x_{t}+W_{h_{t}}h_{t-1}+W_{c_{t}}c_{t-1}+b_{i}\big{)} \tag{2}\]  
\[f_{t}=\sigma\big{(}W_{xf}x_{t}+W_{hf}h_{t-1}+W_{cf}c_{t-1}+bf\Big{)} \tag{3}\]  
\[c_{t}=f_{t}c_{t-1}+i_{t}\tanh(W_{xc}x_{t}+W_{hc}h_{t-1}+b_{c}) \tag{4}\]  
\[o_{t}=\sigma(W_{xo}x_{t}+W_{ho}h_{t-1}+W_{co}c_{t-1}+bo) \tag{5}\]  
Figure 1: The sequence autoencoder for the sequence “WXYZ”. \[h_{t}=o_{t}\tanh(c_{t}). \tag{6}\]  
In the given equation, the logistic sigmoid function \(\sigma\) is used, and the vectors \(i_{t}\), \(f_{t}\), \(o_{t}\), and \(c_{t}\) represent the input gate, forget gate, output gate, and cell activation vectors, respectively. These vectors are of the same size as the hidden vector \(h\).