# A Novel Bi-Dual Inference Approach for Detecting Six-Element Emotions
## 3 Bi-Dual Inference for Detecting Emotions
### Bi-Dual Inference Construction  
In this subsection, we will provide an explanation of how the above two models (dual model and primal model) are jointly trained. We establish a standard dual learning loss function [41] for a single primal model \(f_{0}\) and a single dual model \(g_{0}\), as illustrated in Equation (9). \[\ell_{dual} =\frac{1}{|M_{x}|}\sum_{x\in D_{x}}\Delta x(x,g(f(x)))+\frac{1}{|M_{ y}|}\sum_{y\in D_{y}}\Delta y(y,f(g(x))). \tag{9}\]  
\(D_{x}\) and \(D_{y}\) refer to the data collected from domains \(X\) and \(Y\), respectively. In this context, \(X\) represents sentiment-laden sentence data, while \(Y\) represents sentiment polarity labels. It is worth noting that \(D_{x}\) is a subset of \(\mathcal{X}\) and \(D_{y}\) is a subset of \(\mathcal{Y}\). The quantities \(|M_{x}|\) and \(|M_{y}|\) represent the number of samples in the datasets \(D_{x}\) and \(D_{y}\), respectively. Taking inspiration from pairwise inference [37], if the inference of the original model or the pairwise model relies on the most natural and direct approach, we can train the loss functions of the original and pairwise tasks together by combining them. By minimizing the loss of the combined functions, we consider the output as the inference result. Specifically, for the original task and the dyadic task, their inference methods are given by Equations (10) and (11), respectively. \[f_{dual}(x)=\underset{y^{\prime}\in Y}{\arg\min}\Big{\{}\alpha \ell_{f}\big{(}x,y^{\prime}\big{)}+(1-\alpha)\ell_{g}\big{(}x,y^{\prime}\big{)} \Big{\}} \tag{10}\] \[g_{dual}(y)=\underset{x^{\prime}\in X}{\arg\min}\Big{\{}\beta \ell_{f}\big{(}x^{\prime},y\big{)}+(1-\beta)\ell_{g}\big{(}x^{\prime},y\big{)} \Big{\}}. \tag{11}\]  
In this context, the hyperparameters \(\alpha\) and \(\beta\) are used to balance the trade-off between the two losses, and their values will be adjusted based on the performance on the validation set. The loss functions for the single original model and the pairwise model are denoted as \(\ell_{f}\) and \(\ell_{g}\), respectively. As sentiment analysis is considered a multi-categorical problem in this paper, we utilize the negative log-likelihood function as the loss function, as shown in Equations (12) and (13). \[\ell_{f}(x,y)=-\sum{}_{(x,y)}\log P(y|x;f) \tag{12}\]  
\[\ell_{g}(x,y)=-\sum{}_{(x,y)}\log P(x|y;g) \tag{13}\]  
As the loss functions \(\ell_{f}\) and \(\ell_{g}\) are negative log-likelihood functions, their values range from 0 to infinity. The closer their values are to zero, the better the trained original and pairwise models are considered to represent the data. Many commonly used inference rules in machine learning tasks can be described using Equations (14) and (15). \[f(x)=\underset{y^{\prime}\in Y}{arg\min}\ell_{f}\big{(}x,y^{\prime}\big{)} \tag{14}\]  
\[g(y)=\underset{x^{\prime}\in X}{arg\min}\ell_{g}\big{(}x^{\prime},y\big{)}. \tag{15}\]  
By setting \(\alpha\) and \(\beta\) to one, we can consider the extreme cases in dual inference. From this perspective, we perceive dual inference as a more comprehensive framework for inference. It is important to note that the dual inference approach studied by [PERSON] et al. [37] does not involve retraining or making any modifications to the model for the original and pairwise tasks. However, in this paper, due to the presence of multiple \(f\) and \(g\), multiple training processes are required. To address this, we utilize the probabilisticduality method [36] to train the multiple \(f\) and \(g\). The core algorithm is presented in Equations (16) and (17). \[G_{f} =\
abla\theta_{xy}(1/m)\sum_{j=1}^{m}\big{[}\ell_{1}\big{(}f(x_{j}; \theta_{xy}),y_{j})+\lambda_{xy}\ell_{dual}\big{(}x_{j},y_{j};\theta_{xy}, \theta_{yx}\big{)}\big{]} \tag{16}\] \[G_{g} =\
abla\theta_{yx}(1/m)\sum_{j=1}^{m}\big{[}\ell_{2}\big{(}g(y; \theta_{yx}),x_{j}\big{)}+\lambda_{yx}\ell_{dual}\big{(}x_{j},y_{j};\theta_{ xy},\theta_{yx}\big{)}\big{]}. \tag{17}\]  
In the context of this discussion, \(m\) represents the total number of training sample pairs \((x,y)\).