In this task, you will be presented with a context from an academic paper and a question separated with a
. You have to answer the question based on the context.

Input: Consider Input: In order to represent individual sentences, we insert external [cls] tokens at the start of each sentence, and each [cls] symbol collects features for the sentence preceding it. We also use interval segment embeddings to distinguish multiple sentences within a document. This way, document representations are learned hierarchically where lower Transformer layers represent adjacent sentences, while higher layers, in combination with self-attention, represent multi-sentence discourse. Position embeddings in the original Bert model have a maximum length of 512; we overcome this limitation by adding more position embeddings that are initialized randomly and fine-tuned with other parameters in the encoder.
Question: What is novel about their document-level encoder?

Output: Bert model have a maximum length of 512; we overcome this limitation by adding more position embeddings we insert external [cls] tokens at the start of each sentence, and each [cls] symbol collects features for the sentence preceding it document representations are learned hierarchically

Input: Consider Input: The English online magazine of ISIS was named Dabiq and first appeared on the dark web on July 2014 and continued publishing for 15 issues. This publication was followed by Rumiyah which produced 13 English language issues through September 2017. Looking through both Dabiq and Rumiyah, many issues of the magazines contain articles specifically addressing women, usually with " to our sisters " incorporated into the title.
Question: Do they report results only on English data?

Output: Yes

Input: Consider Input: In this paper, we analyze the task of automatically correcting run-on sentences. We develop two methods: a conditional random field model (roCRF) and a Seq2Seq attention model (roS2S) and show that they outperform models from the sister tasks of punctuation restoration and whole-sentence grammatical error correction.
Question: Which machine learning models do they use to correct run-on sentences?
Output: conditional random field model Seq2Seq attention model