A bird's eye view on coherence, and a worm's eye view on cohesion

1 INTRODUCTION . The terms coherence and cohesion in linguistics are commonly defined as follows ( Williams & Colomb , 1995 ) . Cohesion : sentence pairs fitting together the way two pieces of a jigsaw puzzle do . Coherence : what all the sentences in a piece of writing add up to , the way all the pieces in a puzzle add up to the picture on the box . In layman ’ s terms , cohesion indicates that two consecutive sentences are locally well-connected , and coherence indicates that multiple sentences globally hold together . Generating cohesive and coherent natural language texts that span multiple sentences is a challenging task mainly due to two reasons . First , there is no formal specification of cross-sentence linguistic properties , such as coherence and cohesion of a text . Second , there is no widely accepted surrogate neural model to evaluate the two properties . Most state-of-the-art neural approaches to natural language generation ( NLG ) relied on a large amount of human-generated texts to train language models ( Cho et al. , 2014 ; Graves , 2013 ; Sutskever et al. , 2014 ) . Although these models can generate sentences that , if judged individually , are similar to human-generated ones , they often fail to capture the local and global dependencies among sentences , resulting in a neither coherent nor cohesive text . For example , neural language models based on Recurrent Neural Networks ( RNNs ) are widely applied to response generation for dialogue ( Vinyals & Le , 2015 ; Shang et al. , 2015 ; Sordoni et al. , 2015 ; Li et al. , 2015 ) . Although the responses by themselves look reasonable , they are either bland such as “ I don ’ t know ” , or incoherent with the whole dialogue session . See Gao et al . ( 2018 ) for a comprehensive survey . In this paper , we strive to address the challenge in a principled manner . We propose a pair of discriminators to score whether and to what extent a text is coherent or cohesive . The coherence discriminator measures the compatibility among all sentences in a text using sentence-level features , thus providing a macro-level view on the multi-sentence text . The cohesion discriminator , on the other hand , measures the compatibility of each pair of consecutive sentences using only word-level features , thus providing a micro-level view on neighboring sentences . These models , given a conditional input text and multiple candidate output texts , are learned to score the candidates with respect to the criterion by optimizing pairwise ranking losses . These scores are then used as reward signals to train an RNN-based language model to generate ( more ) coherent and cohesive texts . Our main contributions are three-fold : ( 1 ) we propose two linguistic discriminators for modeling coherence and cohesion of a text ; ( 2 ) we present a simple yet effective training mechanism to encode these linguistic properties ; and ( 3 ) we propose negative-critical sequence training , a variant of policy gradient method , which uses negative samples to construct its reward baseline . This paper proposes a neural approach to explicitly model cross-sentence linguistic properties , i.e. , coherence and cohesion , for long text generation . Despite the encouraging initial results , we only scratched the surface of the problem . The proposed method is yet to be significantly improved to meet the ultimate goal of generating meaningful and logical long-form texts . We cast the text generation as an RL problem and review recent work in Section 2 , and detail our approach in Section 3 . 2 RELATED WORK . Coherence and cohesion . Coherence and cohesion have been extensively studied in the computational linguistics community , particularly in the ‘ pre-deep-learning ’ era . Lack of formal specifications for coherence and cohesion ( Mani et al. , 1998 ) , resulted in many different formalism , such as Rhetorical Structure Theory ( Mann & Thompson , 1988 ) , and other forms of coherence and cohesion relations and their quantification ( Mani et al. , 1998 ; Hobbs , 1985 ; Hovy , 1988 ; McKeown , 1985 ; Cohen & Levesque , 1985 ; Hovy , 1991 ; Cristea et al. , 1998 ; Halliday & Hasan , 1996 ; Liddy , 1991 ; Van Dijk , 2013 ; Edmundson , 1969 ; Barzilay & Lapata , 2008 ) . In fact , the list is not exhaustive . However , modeling coherence and cohesion of a text using models parametrized by neural networks have not been previously explored . Word sequence generation in a reinforcement learning framework . A word sequence generation task can be framed as a reinforcement learning ( RL ) problem , in which the generatorG is acting as a policy π , with parameters θπ , and each generated word at time t , wt , can be viewed as an action to be chosen by the policy from a large discrete space , or vocabulary , conditioned on state st−1 = w≤t−1 , which encodes the previously generated text sequence . Let rt be the reward for a partially generated text sequence w≤t . We define the long-term expected reward J ( π ) = Es0∼q , π [ ∑∞ t=1 γ t−1rt ] , where q is the initial distribution of conditional input texts . Following Sutton et al . ( 1999 ) , the gradient of J with respect to θπ is ∇θπJ = Es∼ρπ , a∼π ( ·|s ) [ Qπ ( s , a ) ∇θπ log πθπ ( a|s ) ] where ρπ is the stationary distribution and Qπ ( s , a ) is the expected return from state s and taking action a , both following policy π . For brevity , we omit the derivation . In our work , we formulate text generation as an episodic RL problem with episode length L , rewards rL being available only at the end of episode and γ = 1 . There are many works on training neural language models using reward signals , such as Ranzato et al . ( 2015 ) and Paulus et al . ( 2017 ) . These works directly optimize for specific metrics , such as BLEU ( Papineni et al. , 2002 ) or ROUGE ( Lin & Hovy , 2003 ) , using REINFORCE ( Williams , 1992 ; Sutton et al. , 1999 ) . However , it is well-known that these metrics do not give a complete picture on the quality of the generated text . Only recently have there been efforts to provide more relevant quality objectives for which to optimize ( Li et al. , 2015 ; 2016a ; Holtzman et al. , 2018 ) the quality of interest such as consistency , repetition of text . But these works use the objective function to re-rank candidate outputs , not to reward or penalize outputs when they are generated in the first place . Li et al . ( 2016b ) constructed a set of reward models , such as information flow and semantic coherence , to tune the generator , yet they do not provide an ablation study to elaborate the relative contribution of these reward models individually . GANs for text generation . Another line of research is to use Generative Adversarial Networks ( GANs ) ( Goodfellow et al. , 2014 ) to incorporate feedback signals for text generation ( Yu et al. , 2017 ; Lin et al. , 2017 ; Zhang et al. , 2017c ; Guo et al. , 2017 ; Fedus et al. , 2018 ; Zhang et al. , 2018 ) . However , the discriminator in these works are trained to distinguish real texts from the generated ones , operating as a black-box rather than providing fine-grained feedback on particular linguistic aspects of the texts . In fact , Yang et al . ( 2018 ) has partially addressed this issue by using a trained language model as the discriminator . Although the discriminator provides a fine-grained feedback at the word-level , it does not critique on many important linguistic properties of generated texts , such as cohesion and coherence . These text generators , when facing a long-form text generation task that span multiple sentences , are by no means perfect and often exhibit some critical errors , such as a breakdown of local connections between consecutive sentences ( cohesion ) , let alone globally solid intention ( coherence ) . As a result , readers can easily take these cues and discriminate such generated texts against real texts . In this paper , we argue that the primary reason is the lack of an effective mechanism of measuring and controlling the text quality in the generation process . The method we propose in the next section is intended to address the problem . 3 MODEL . We assume that global coherence of a text depends to a large degree upon how its individual sentences with different meanings are organized . So we focus our evaluation of coherence solely on the sentence-level . If the sentences are not organized properly , the intention of the paragraph as a whole is obscure , regardless of seamless local connectivity between consecutive sentences . This is not to say that local connections between any two neighboring sentences should be overlooked . One can easily distinguish a model-generated sentence from a real one , simply by looking at whether the sentence followed by another sentence relates , besides grammar . We instill these two different yet important concepts in two discriminators , each operating on the sentence level and word level . Our models closely resemble successful models for computer vision , such as StackGAN ( Zhang et al. , 2017a ; b ) and PatchGAN ( Isola et al. , 2017 ) in that they all provide hierarchical signals to their corresponding generators , where the signals are derived from raw lowlevel data . We call the sentence-level discriminator the coherence discriminator Dcoherence , and the word-level discriminator the cohesion discriminator Dcohesion . 3.1 COHERENCE DISCRIMINATOR : DCOHERENCE This discriminator measures how likely two text chunks form a coherent paragraph . Let S : = [ s1 , s2 , ... , sn ] be the source text chunk that consists of n sentences , T : = [ t1 , t2 , ... , tm ] be the real target text chunk that consists of m sentences , and T̃ : = [ t̃1 , t̃2 , ... , t̃m̃ ] be the model-generated target text chunk that consists of m̃ sentences . Dcoherence is designed to distinguish a real pair ( S , T ) from a synthetic pair ( S , T̃ ) by assigning them with different scores , i.e. , Scorecoherence ( S , T ) > Scorecoherence ( S , T̃ ) . Design . Our design ofDcoherence is inspired by the Deep Structured Semantic Model ( DSSM ) ( Huang et al. , 2013 ; Gao et al. , 2014 ; Xu et al. , 2017 ) . Given source text chunk S and target text chunk T , Dcoherence computes their coherence score in two steps , as illustrated in Figure 1 . First , a pair of convolutional networks ( CNNs ) 1 or RNNs are applied to encode both S and T into two lowdimensional continuous vectors . Second , the coherence score is computed as the cosine similarity of the two vectors . The score is a real value between −1 and 1 , where 1 indicates the maximal coherence score , and −1 the minimal coherence score . Dcoherence measures how likely S and T add up to a single coherent passage . The score depends on the parameters of the two CNNs/RNNs , or in other words , how S and T are encoded . Since we focus solely on the sentence-level features , we view a text chunk as a sequence of sentences , and view each sentence as a bag-of-words . Therefore , we represent each word using its pre-trained word embedding vector ( Pennington et al. , 2014 ) and represent each sentence using a vector which takes the average of its word embedding vectors . A text chunk is then represented as a sequence of sentence vectors which are fed to the CNN/RNN ( either f ( · ) or g ( · ) in Figure 1 ) . Notice that f ( · ) and g ( · ) do not share parameters . The parameters of f ( · ) and g ( · ) are optimized in such a way that a real pair scores higher than a synthetic pair : ∆ ( θf , θg ) : = Dcoherence ( fθf ( S ) , gθg ( T ) ) −Dcoherence ( fθf ( S ) , gθg ( T̃ ) ) > 0 Formally , the task of optimizingDcoherence can be stated as follows . Given a set of training samples of the form ( ( S , T ) , ( S , T̃ ) ) ( i ) , i = 1 ... M , we optimize parameters ( θf , θg ) by minimizing the pairwise rank loss on training data defined as 1 M M∑ i=1 L ( ∆ ( θf , θg ) ( i ) ) where L ( · ) is a loss function , differentiable w.r.t . ( θf , θg ) . In the following subsection , we will describe in turn how we construct the pairwise training samples and the form of the loss function . Since Dcoherence is used as a pairwise ranker , we employ the metrics commonly used in information retrieval for evaluation , such as recall at K ( R @ K ) , which is defined as the fraction of correctly identifying an item in the TOP-K retrieved list ( Baeza-Yates & Ribeiro-Neto , 1999 ) . We present the retrieval results on test data in Table 2 . Training mechanism . How do we construct these list of candidate target sentences T , given the source sentences S ? We assume the T that follows an S in the data is a positive target sample , or the correct item to retrieve . Negative samples are constructed using three different methods within a batch while iterating through the training data , motivated by Wieting et al . ( 2016 ) : • Rotate T with S fixed ( mismatch S and T ) across a batch . For a single S , this method yields B − 1 negative samples , where B is the batch size . • Shuffle the sentence order once , different from its original order , known as a derangement , in each T to break coherence , and this yields one negative sample . • Combine the previous two methods : rotate T across a batch and shuffle sentences within T , yielding B − 1 negative samples . These 2B − 1 negative samples and a single positive sample , in sum , pose a significant challenge in learning . To fit this training task into a ranking framework , we optimize over Dcoherence ( fθf ( S ) , gθg ( T ) ) − AMλi∈ { 1 , ... ,2B−1 } [ Dcoherence ( fθf ( S ) , gθg ( T̃i ) ) ] where the arithmetic mean parametrized by λ : AMλ ( x ) = N∑ i wixi and wi = eλxi/ ∑ j e λxj . In our experiments , we fix λ = 2 and this assigns more weight to a more challenging negative sample2 . Notice that AMλ is the mean if λ = 0 , and approaches the max as λ → ∞ . Empirically , training the models using the weighted mean resulted in faster convergence , as opposed to using the single most challenging negative sample score ( max ) or the mean of all negative sample batch . 1We explored with deeper networks . However , the performance difference was marginal . For simplicity , we decided to use a 1-layer convolutional network architecture ( Kim , 2014 ; Collobert et al. , 2011 ) . 2We performed a coarse grid search over the values of λ and setting λ = 2 resulted in fast convergence to high recall scores on the dev dataset . 3.2 COHESION DISCRIMINATOR : DCOHESION Our second discriminator pays attention only to low-level features to capture a local connection between arbitrary two consecutive sentences because the pairwise connectivity influence readability . For simplicity , Dcohesion is similar toDcoherence , except that its architecture , input , and negative sample construction are modified to encode cohesion between any pair of sentences on the word-level . A single input sample to Dcohesion is a pair of two consecutive sentences : [ si,1 , si,2 , ... , si , n ] and [ si+1,1 , si+1,2 , ... , si+1 , m ] , where si , k denotes the k-th word in sentence i . We construct the negative samples using the three methods for training Dcoherence , where shuffling occurs on the word level within each sentence , rather than shuffling multiple sentences on the sentence level .

Summary: The paper proposes a method for improving the quality of text generation by optimizing for coherence and cohesion. The authors develop two discriminators--a "coherence discriminator" which takes as input all of the sentence embeddings (i.e. averaged word embeddings) of the document and assigns a score, and a "cohesion discriminator" which takes as input the word embeddings of two consecutive sentences and assigns a score. In the former, the score is the cosine similarity between the encodings of the first and second half of the document. In the latter, the score is the cosine similarity between the encodings of the two sentences. Both discriminators use CNNs to encode the inputs. The discriminators are trained to rank true text over randomly drawn negative samples, which consist of randomly permuted sentence orderings and/or random combinations of first/second half of documents. This discriminators are then used to train a text generation model. The output of the text generation model is scored by various automatic metrics, including NLL, PPL, BLEU, and number of unique ngrams in the outputs. The improvements over a generically-trained generation model are very small.