In this task, you will be presented with a context from an academic paper and a question separated with a 
. You have to answer the question based on the context.

Q: In order to represent individual sentences, we insert external [cls] tokens at the start of each sentence, and each [cls] symbol collects features for the sentence preceding it. We also use interval segment embeddings to distinguish multiple sentences within a document. This way, document representations are learned hierarchically where lower Transformer layers represent adjacent sentences, while higher layers, in combination with self-attention, represent multi-sentence discourse. Position embeddings in the original Bert model have a maximum length of 512; we overcome this limitation by adding more position embeddings that are initialized randomly and fine-tuned with other parameters in the encoder. 
 Question: What is novel about their document-level encoder?

A: Bert model have a maximum length of 512; we overcome this limitation by adding more position embeddings we insert external [cls] tokens at the start of each sentence, and each [cls] symbol collects features for the sentence preceding it document representations are learned hierarchically
****
Q: The library design of Torch-Struct follows the distributions API used by both TensorFlow and PyTorch BIBREF29. 
 Question: Does API provide ability to connect to models written in some other deep learning framework?

A: Yes
****
Q: A document $d$ can be defined as a finite sequence of terms (independent textual entities within a document, for example, words), namely $d=(t_1,t_2,\dots ,t_n)$. 
 Question: What representations for textual documents do they use?

A:
finite sequence of terms
****