In this task, you will be presented with a context from an academic paper and you have to write an answerable question based on the context. Your questions can be extractive, abstractive, or yes-no questions.

[Q]: Methods ::: Baselines ::: BiGRU s with attention:
This is very similar to Sum-QE but now $\mathcal {E}$ is a stack of BiGRU s with self-attention BIBREF21, instead of a BERT instance. The final summary representation ($h$) is the sum of the resulting context-aware token embeddings ($h = \sum _i a_i h_i$) weighted by their self-attention scores ($a_i$). We again have three flavors: one single-task (BiGRU-ATT-S-1) and two multi-task (BiGRU-ATT-M-1 and BiGRU-ATT-M-5).

Methods ::: Baselines ::: ROUGE:
This baseline is the ROUGE version that performs best on each dataset, among the versions considered by BIBREF13. Although ROUGE focuses on surface similarities between peer and reference summaries, we would expect properties like grammaticality, referential clarity and coherence to be captured to some extent by ROUGE versions based on long $n$-grams or longest common subsequences.

Methods ::: Baselines ::: Language model (LM):
For a peer summary, a reasonable estimate of $\mathcal {Q}1$ (Grammaticality) is the perplexity returned by a pre-trained language model. We experiment with the pre-trained GPT-2 model BIBREF22, and with the probability estimates that BERT can produce for each token when the token is treated as masked (BERT-FR-LM). Given that the grammaticality of a summary can be corrupted by just a few bad tokens, we compute the perplexity by considering only the $k$ worst (lowest LM probability) tokens of the peer summary, where $k$ is a tuned hyper-parameter.

Methods ::: Baselines ::: Next sentence prediction:
BERT training relies on two tasks: predicting masked tokens and next sentence prediction. The latter seems to be aligned with the definitions of $\mathcal {Q}3$ (Referential Clarity), $\mathcal {Q}4$ (Focus) and $\mathcal {Q}5$ (Structure & Coherence). Intuitively, when a sentence follows another with high probability, it should involve clear referential expressions and preserve the focus and local coherence of the text. We, therefore, use a pre-trained BERT model (BERT-FR-NS) to calculate the sentence-level perplexity of each summary:

where $p(s_i|s_{i-1})$ is the probability that BERT assigns to the sequence of sentences $\left< s_{i-1}, s \right>$, and $n$ is the number of sentences in the peer summary.
[A]: What simpler models do they look at?


[Q]: We first describe our models individually and then the ensembling technique that we employ. In the following, MN denotes Memory Networks to encode conversational history, RCNN signify R-CNN for object level representations of an image, Wt represents additional linear layer in the decoder, and LF a late fusion mechanism as defined in BIBREF0.

Models ::: LF-RCNN
Late fusion encoder BIBREF0 with concatenated history. We use two-layered LSTMs with 512 hidden units for embedding questions and history. The object-level features are weighed using only question embeddings. The word embeddings from Glove vectors are frozen and are not fine-tuned. Figure FIGREF6 gives an overview of the architecture.

Models ::: MN-RCNN
Memory network encoder BIBREF0 with bi-directional GRUs and word embeddings fine-tuned. Object-level features are weighed by question and caption embedding. The rest of the scheme is same as above. (Figure FIGREF6)

Models ::: MN-RCNN-Wt
Same as above but with an additional linear layer applied to the dot product of candidate answer and encoder output, and gated using tanh function. Compare Figure FIGREF6 with Figure FIGREF6
[A]: Which three discriminative models did they use?


[Q]: We collected a simplified dataset from Simple English Wikipedia that are freely available, which has been previously used for many text simplification methods BIBREF0 , BIBREF10 , BIBREF3 .
[A]:
what language does this paper focus on?