Question: What are the two factors that their experiment focuses on?
Evidence: Natural language generation and document classification have been widely conducted using neural sequence models based on the encoder–decoder architecture. The underlying technique relies on the production of a context vector as the document representation, to estimate both tokens in natural language generation and labels in classification tasks. By combining recurrent neural networks with attention BIBREF0, the model is able to learn contextualized representations of words at the sentence level. However, higher-level concepts, such as discourse structure beyond the sentence, are hard for an RNN to learn, especially for longer documents. We hypothesize that NLP tasks such as summarization and document classification can be improved through the incorporation of discourse information.
In this paper, we propose to incorporate latent representations of discourse units into neural training. A discourse parser can provide information about the document structure as well as the relationships between discourse units. In a summarization scenario, for example, this information may help to remove redundant information or discourse disfluencies. In the case of document classification, the structure of the text can provide valuable hints about the document category. For instance, a scientific paper follows a particular discourse narrative pattern, different from a short story. Similarly, we may be able to predict the societal influence of a document such as a petition document, in part, from its discourse structure and coherence.
Specifically, discourse analysis aims to identify the organization of a text by segmenting sentences into units with relations. One popular representation is Rhetorical Structure Theory (RST) proposed by mann1988rhet, where the document is parsed into a hierarchical tree, where leaf nodes are the segmented units, known as Entity Discourse Units (EDUs), and non-terminal nodes define the relations.
As an example, in Figure FIGREF1 the two-sentence text has been annotated with discourse structure based on RST, in the form of 4 EDUs connected with discourse labels attr and elab. Arrows in the tree capture the nuclearity of relations, wherein a “satellite” points to its “nucleus”. The Nucleus unit is considered more prominent than the Satellite, indicating that the Satellite is a supporting sentence for the Nucleus. Nuclearity relationships between two EDUs can take the following three forms: Nucleus–Satellite, Satellite–Nucleus, and Nucleus–Nucleus. In this work, we use our reimplementation of the state of the art neural RST parser of BIBREF1, which is based on eighteen relations: purp, cont, attr, evid, comp, list, back, same, topic, mann, summ, cond, temp, eval, text, cause, prob, elab.
This research investigates the impact of discourse representations obtained from an RST parser on natural language generation and document classification. We primarily experiment with an abstractive summarization model in the form of a pointer–generator network BIBREF2, focusing on two factors: (1) whether summarization benefits from discourse parsing; and (2) how a pointer–generator network guides the summarization model when discourse information is provided. For document classification, we investigate the content-based popularity prediction of online petitions with a deep regression model BIBREF3. We argue that document structure is a key predictor of the societal influence (as measured by signatures to the petition) of a document such as a petition.
Our primary contributions are as follows: (1) we are the first to incorporate a neural discourse parser in sequence training; (2) we empirically demonstrate that a latent representation of discourse structure enhances the summaries generated by an abstractive summarizer; and (3) we show that discourse structure is an essential factor in modelling the popularity of online petitions.
We primarily experiment with an abstractive summarization model in the form of a pointer–generator network, focusing on two factors: (1) whether summarization benefits from discourse parsing; and (2) how a pointer–generator network guides the summarization model when discourse information is provided.
Answer: (1) whether summarization benefits from discourse parsing; and (2) how a pointer–generator network guides the summarization model when discourse information is provided.