ABCD : A Graph Framework to Convert Complex Sentences to a Covering Set of Simple Sentences (2021)

Abstract: Atomic clauses are fundamental text units for understanding complex sentences. Identifying the atomic sentences within complex sentences is important for applications such as summarization, argument mining, discourse analysis, discourse parsing, and question answering. Previous work mainly relies on rule-based methods dependent on parsing. We propose a new task to decompose each complex sentence into simple sentences derived from the tensed clauses in the source, and a novel problem formulation as a graph edit task. Our neural model learns to Accept, Break, Copy or Drop elements of a graph that combines word adjacency and grammatical dependencies. The full processing pipeline includes modules for graph construction, graph editing, and sentence generation from the output graph. We introduce DeSSE, a new dataset designed to train and evaluate complex sentence decomposition, and MinWiki, a subset of MinWikiSplit. ABCD achieves comparable performance as two parsing baselines on MinWiki. On DeSSE, which has a more even balance of complex sentence types, our model achieves higher accuracy on the number of atomic sentences than an encoder-decoder baseline. Results include a detailed error analysis.

## introduction atomic clauses are fundamental text units for understanding complex sentences. the ability to decompose complex sentences facilitates research that aims to identify, rank or relate distinct predications, such as content selection in summarization @xcite , labeling argumentative discourse units in argument mining @xcite or elementary discourse units in discourse analysis @xcite @xcite , or extracting atomic propositions for question answering @xcite . in this work, orig sokuhi was born in fujian and was ordained at 17. ## related work related work falls largely into parsing-based methods, neural models that rewrite, and neural segmenters. @xcite propose a decomposition parser (dcp) that extracts vp constituents and clauses from complex sentences as part of a summarization evaluation tool. @xcite present a system (dissim) based on parsing to extract simple sentences from complex ones. @xcite propose seven rules to extract complete propositions from parses of complex questions and imperatives for argumentation mining. though performance of these methods depends on parser quality, they often achieve very good performance. we include two whose code is available (dcp, dissim) among our baselines. sprp models are based on encoder-decoder architectures, and the output is highly depending on the training corpus. aharoni and goldberg (2018) present a copy-augmented network (copy 512 ) based on @xcite that encour-ages the model to copy most words from the original sentence to the output. as it achieves improvement over an earlier encoder-decoder sprp model @xcite , we include copy 512 among our baselines. finally, recent neural edu segmenters @xcite achieve state-of-the-art performance on a discourse relation corpus, rst-dt @xcite . as they do not output complete sentences, we do not include any among our baselines. our abcd model leverages the detailed information captured by parsing methods, and the powerful representation learning of neural models. as part of a larger pipeline that converts input sentences to graphs, abcd learns to predict graph edits for a post processor to execute. ## datasets here we present desse, a corpus we collected for our task, and minwiki, a modification of an existing sprp corpus (minwikisplit @xcite ) to support our aims. we also give a brief description of differences in their distributions. neural models are heavily biased by the distributions in their training data @xcite , and we show that desse has a more even balance of linguistic phenomena. ## problem formulation we formulate the problem of converting complex sentences into covering sets of simple sentences as a graph segmentation problem. each sentence is represented as a word relation graph (wrg), a directed graph constructed from each input sentence with its dependency parse. every word token and its positional index becomes a wrg vertex. for every pair of words, one or more edges are added as follows: a neighbor edge that indicates that the pair of words are linearly adjacent; a dependency edge that shows every pair of words connected by a dependency relation, adding critical grammatical relations, such as subject. figure 3 shows an example sentence and a simplified version of its wrg (edge directions are not shown, for readability). vertices are labeled with word-index pairs in red font, and edges are labeled as ngbh for neighboring words, or with the tags corresponding to their dependency relations, such as nsubj between sokuhi-1 and ordained-13. an edge can have both types of relation, e.g. neighbor and dependency for was-12 and ordained-13. the graph is stored as an edge triple set, a set of triples with (source node, target node, label) representing each pair of words connected by an edge, as shown in figure 3 , bottom left. given a sentence and its wrg, our goal is to decompose the graph into n connected components (cc) where each cc is later rewritten as an output simple sentence. to perform the graph decomposition, decisions are made on every edge triple.we define four edit types: a training example consists of an input sentence, and one or more output sentences. if the input sentence is complex, the ground truth output consists of multiple simple sentences. the next section presents the abcd pipeline. two initial modules construct the wrg graphs for each input sentence, and the abcd labels for the edge triple sets based on the ground truth output. a neural model learns to assign abcd labels to input wrg graphs, and a final graph segmenter generates simple sentences from the labeled wrg graphs. details about the neural model are in the subsequent section. the full processing pipeline consists of five major components, as shown in figure 4 . three preprocessing modules handle the wrg graph construction, conversion of graph triples to vectors, and creation of distant supervision labels for the graph. the fourth component is the abcd neural model that learns to label a wrg graph, which is described in section 6. the last part of the pipeline is a post-processing module to segment wrg graphs based on the labels learned by the abcd model, and to map each graph segment to a simple sentence. graph constructor the first module in the system is a graph constructor that converts an input sentence and its dependency parse into a collection of vertices and edges. it is used during training and inference. it first extracts words and their indices from the input sentences of the training examples for the vertices of each wrg graph. a directed edge and ngbh label is assigned to all pairs of adjacent words. a directed edge and label is also assigned to every governing and dependent word pair (cf. figure 3 ). edge triples db the edge triples db, which is used during training and inference, creates vector representations for the input edge triples sets for each training instance, using latent representations learned by an encoder component of the abcd model. using the word indices, a function maps the source and target words from every triple into its hidden representation learned by the encoder, and the triple's edge label is converted into a one-hot encoding with dimension d. for an edge triples set with m triples, the source and target word hidden states are each stacked into an m × h matrix, and the one-hot vectors for edge labels are stacked into an m × d matrix. these three source, target, edge matrices that represent an edge triple set are then fed into an attention layer, as discussed in section 6. ## neural model the abcd model consists of three neural modules depicted in figure 5 : a sentence encoder to learn a hidden representation for the input sentence, a self-attention layer to generate attention scores on every edge label, and a classifier that generates a predicted distribution over the four edit types, based on the word's hidden representation, the edge label representation, and the attention scores. ## experiments we carry out two intrinsic evaluations of abcd performance on minwiki and desse. section 7.1 presents an intrinsic evaluation of abcd variants on edit prediction, with error analysis and ablation studies. section 7.2 compares the best abcd model with several baselines on the quality of output propositions. we discuss evaluation metrics in section 7.3. results show that abcd models show consistently good performance compared to other baseline models on both datasets. ## discussion in this section, we discuss limitations of abcd to guide future work. the first limitation is the low performance of abcd on b. we observe that in desse, some annotators did not break the sentences appropriately. we randomly selected 50 samples, and found 13 out of 50 (26%) examples orig he did not do anything wrong, yet he was targeted and his family was murdered. human he did not do anything wrong. || he was targeted. || his family was murdered. ## conclusion we presented a new task to decompose complex sentences into simple ones, along with desse, a new dataset designed for this task. we proposed the neural abcd model to predict four edits operations on sentence graphs, as part of a larger pipeline from our graph-edit problem formulation. abcd performance comes close to or outperforms the parsing-based and encoder-decoder baselines. our work selectively integrates modules to capitalize on the linguistic precision of parsing-based methods, and the expressiveness of graphs for encoding different aspects of linguistic structure, while still capitalizing on the power of neural networks for representation learning.