Teacher: In this task, you will be presented with a context from an academic paper and a question based on the context. You have to classify the questions into "Extractive", "Abstractive", or "Yes-no" questions. Extractive questions can be answered by concatenating extracts taken from a context into a summary while answering abstractive questions involves paraphrasing the context using novel sentences. Yes-no question is a question whose expected answer is one of two choices, one that affirms the question and one that denies the question. Typically, the choices are either yes or no.
Teacher: Now, understand the problem? If you are still confused, see the following example:
We build a dataset of Twitter accounts based on two lists annotated in previous works. For the non-factual accounts, we rely on a list of 180 Twitter accounts from BIBREF1. On the other hand, for the factual accounts, we use a list with another 32 Twitter accounts from BIBREF19 that are considered trustworthy by independent third parties.
Question: How did they obtain the dataset?
Solution: Extractive
Reason: The answer to this question has been explicitly mentioned in the context, so the question is extractive.

Now, solve this instance: Moreover, QA and QG have probabilistic correlation as both tasks relate to the joint probability between $q$ and $a$ . Given a question-answer pair $\langle q, a
angle $ , the joint probability $P(q, a)$ can be computed in two equivalent ways.

$$P(q, a) = P(a) P(q|a) = P(q)P(a|q)$$ (Eq. 1)

The conditional distribution $P(q|a)$ is exactly the QG model, and the conditional distribution $P(a|q)$ is closely related to the QA model. Existing studies typically learn the QA model and the QG model separately by minimizing their own loss functions, while ignoring the probabilistic correlation between them.

Based on these considerations, we introduce a training framework that exploits the duality of QA and QG to improve both tasks. There might be different ways of exploiting the duality of QA and QG. In this work, we leverage the probabilistic correlation between QA and QG as the regularization term to influence the training process of both tasks. Specifically, the training objective of our framework is to jointly learn the QA model parameterized by $ heta _{qa}$ and the QG model parameterized by $ heta _{qg}$ by minimizing their loss functions subject to the following constraint.

$$P_a(a) P(q|a; heta _{qg}) = P_q(q)P(a|q; heta _{qa})$$ (Eq. 3)

$P_a(a)$ and $P_q(q)$ are the language models for answer sentences and question sentences, respectively. Overall, the framework includes three components, namely a QA model, a QG model and a regularization term that reflects the duality of QA and QG. The QA specific objective aims to minimize the loss function $l_{qa}(f_{qa}(a,q; heta _{qa}), label)$ , where $label$ is 0 or 1 that indicates whether $a$ is the correct answer of $q$ or not. For each correct question-answer pair, the QG specific objective is to minimize the following loss function,

$$l_{qg}(q, a) = -log P_{qg}(q|a; heta _{qg})$$ (Eq. 6)

where $a$ is the correct answer of $q$ . The third objective is the regularization term which satisfies the probabilistic duality constrains as given in Equation 3 . Specifically, given a correct $\langle q, a
angle $ pair, we would like to minimize the following loss function,

$$
onumber l_{dual}(a,q; heta _{qa}, heta _{qg}) &= [logP_a(a) + log P(q|a; heta _{qg}) \ & - logP_q(q) - logP(a|q; heta _{qa})]^2$$ (Eq. 9)

where $P_a(a)$ and $P_q(q)$ are marginal distributions, which could be easily obtained through language model.
Question: What does "explicitly leverages their probabilistic correlation to guide the training process of both models" mean?
Student:
Abstractive