Question: What does "explicitly leverages their probabilistic correlation to guide the training process of both models" mean?
Evidence: Moreover, QA and QG have probabilistic correlation as both tasks relate to the joint probability between $q$ and $a$ . Given a question-answer pair $\langle q, a \rangle $ , the joint probability $P(q, a)$ can be computed in two equivalent ways.
$$P(q, a) = P(a) P(q|a) = P(q)P(a|q)$$ (Eq. 1)
The conditional distribution $P(q|a)$ is exactly the QG model, and the conditional distribution $P(a|q)$ is closely related to the QA model. Existing studies typically learn the QA model and the QG model separately by minimizing their own loss functions, while ignoring the probabilistic correlation between them.
Based on these considerations, we introduce a training framework that exploits the duality of QA and QG to improve both tasks. There might be different ways of exploiting the duality of QA and QG. In this work, we leverage the probabilistic correlation between QA and QG as the regularization term to influence the training process of both tasks. Specifically, the training objective of our framework is to jointly learn the QA model parameterized by $\theta _{qa}$ and the QG model parameterized by $\theta _{qg}$ by minimizing their loss functions subject to the following constraint.
$$P_a(a) P(q|a;\theta _{qg}) = P_q(q)P(a|q;\theta _{qa})$$ (Eq. 3)
$P_a(a)$ and $P_q(q)$ are the language models for answer sentences and question sentences, respectively.
We describe the proposed algorithm in this subsection. Overall, the framework includes three components, namely a QA model, a QG model and a regularization term that reflects the duality of QA and QG. Accordingly, the training objective of our framework includes three parts, which is described in Algorithm 1.
The QA specific objective aims to minimize the loss function $l_{qa}(f_{qa}(a,q;\theta _{qa}), label)$ , where $label$ is 0 or 1 that indicates whether $a$ is the correct answer of $q$ or not. Since the goal of a QA model is to predict whether a question-answer pair is correct or not, it is necessary to use negative QA pairs whose labels are zero. The details about the QA model will be presented in the next section.
For each correct question-answer pair, the QG specific objective is to minimize the following loss function,
$$l_{qg}(q, a) = -log P_{qg}(q|a;\theta _{qg})$$ (Eq. 6)
where $a$ is the correct answer of $q$ . The negative QA pairs are not necessary because the goal of a QG model is to generate the correct question for an answer. The QG model will be described in the following section.
The third objective is the regularization term which satisfies the probabilistic duality constrains as given in Equation 3 . Specifically, given a correct $\langle q, a \rangle $ pair, we would like to minimize the following loss function,
$$ \nonumber l_{dual}(a,q;\theta _{qa}, \theta _{qg}) &= [logP_a(a) + log P(q|a;\theta _{qg}) \\ & - logP_q(q) - logP(a|q;\theta _{qa})]^2$$ (Eq. 9)
where $P_a(a)$ and $P_q(q)$ are marginal distributions, which could be easily obtained through language model. $P(a|q;\theta _{qg})$ could also be easily calculated with the markov chain rule: $P(q|a;\theta _{qg}) = \prod _{t=1}^{|q|} P(q_t|q_{<t}, a;\theta _{qg})$ , where the function $P(q_t|q_{<t}, a;\theta _{qg})$ is the same with the decoder of the QG model (detailed in the following section).
Moreover, QA and QG have probabilistic correlation as both tasks relate to the joint probability between $q$ and $a$ . Given a question-answer pair $\langle q, a \rangle $ , the joint probability $P(q, a)$ can be computed in two equivalent ways.

$$P(q, a) = P(a) P(q|a) = P(q)P(a|q)$$ (Eq. 1)

The conditional distribution $P(q|a)$ is exactly the QG model, and the conditional distribution $P(a|q)$ is closely related to the QA model. Existing studies typically learn the QA model and the QG model separately by minimizing their own loss functions, while ignoring the probabilistic correlation between them.

Based on these considerations, we introduce a training framework that exploits the duality of QA and QG to improve both tasks. There might be different ways of exploiting the duality of QA and QG. In this work, we leverage the probabilistic correlation between QA and QG as the regularization term to influence the training process of both tasks. Specifically, the training objective of our framework is to jointly learn the QA model parameterized by $\theta _{qa}$ and the QG model parameterized by $\theta _{qg}$ by minimizing their loss functions subject to the following constraint.

$$P_a(a) P(q|a;\theta _{qg}) = P_q(q)P(a|q;\theta _{qa})$$ (Eq. 3)

$P_a(a)$ and $P_q(q)$ are the language models for answer sentences and question sentences, respectively.
Overall, the framework includes three components, namely a QA model, a QG model and a regularization term that reflects the duality of QA and QG.
The QA specific objective aims to minimize the loss function $l_{qa}(f_{qa}(a,q;\theta _{qa}), label)$ , where $label$ is 0 or 1 that indicates whether $a$ is the correct answer of $q$ or not.
For each correct question-answer pair, the QG specific objective is to minimize the following loss function,

$$l_{qg}(q, a) = -log P_{qg}(q|a;\theta _{qg})$$ (Eq. 6)

where $a$ is the correct answer of $q$ .
The third objective is the regularization term which satisfies the probabilistic duality constrains as given in Equation 3 . Specifically, given a correct $\langle q, a \rangle $ pair, we would like to minimize the following loss function,

$$ \nonumber l_{dual}(a,q;\theta _{qa}, \theta _{qg}) &= [logP_a(a) + log P(q|a;\theta _{qg}) \\ & - logP_q(q) - logP(a|q;\theta _{qa})]^2$$ (Eq. 9)

where $P_a(a)$ and $P_q(q)$ are marginal distributions, which could be easily obtained through language model.
Answer: The framework jointly learns parametrized QA and QG models subject to the constraint in equation 2. In more detail, they minimize QA and QG loss functions, with a third dual loss for regularization.