A bird's eye view on coherence, and a worm's eye view on cohesion

1 INTRODUCTION . The terms coherence and cohesion in linguistics are commonly defined as follows ( Williams & Colomb , 1995 ) . Cohesion : sentence pairs fitting together the way two pieces of a jigsaw puzzle do . Coherence : what all the sentences in a piece of writing add up to , the way all the pieces in a puzzle add up to the picture on the box . In layman ’ s terms , cohesion indicates that two consecutive sentences are locally well-connected , and coherence indicates that multiple sentences globally hold together . Generating cohesive and coherent natural language texts that span multiple sentences is a challenging task mainly due to two reasons . First , there is no formal specification of cross-sentence linguistic properties , such as coherence and cohesion of a text . Second , there is no widely accepted surrogate neural model to evaluate the two properties . Most state-of-the-art neural approaches to natural language generation ( NLG ) relied on a large amount of human-generated texts to train language models ( Cho et al. , 2014 ; Graves , 2013 ; Sutskever et al. , 2014 ) . Although these models can generate sentences that , if judged individually , are similar to human-generated ones , they often fail to capture the local and global dependencies among sentences , resulting in a neither coherent nor cohesive text . For example , neural language models based on Recurrent Neural Networks ( RNNs ) are widely applied to response generation for dialogue ( Vinyals & Le , 2015 ; Shang et al. , 2015 ; Sordoni et al. , 2015 ; Li et al. , 2015 ) . Although the responses by themselves look reasonable , they are either bland such as “ I don ’ t know ” , or incoherent with the whole dialogue session . See Gao et al . ( 2018 ) for a comprehensive survey . In this paper , we strive to address the challenge in a principled manner . We propose a pair of discriminators to score whether and to what extent a text is coherent or cohesive . The coherence discriminator measures the compatibility among all sentences in a text using sentence-level features , thus providing a macro-level view on the multi-sentence text . The cohesion discriminator , on the other hand , measures the compatibility of each pair of consecutive sentences using only word-level features , thus providing a micro-level view on neighboring sentences . These models , given a conditional input text and multiple candidate output texts , are learned to score the candidates with respect to the criterion by optimizing pairwise ranking losses . These scores are then used as reward signals to train an RNN-based language model to generate ( more ) coherent and cohesive texts . Our main contributions are three-fold : ( 1 ) we propose two linguistic discriminators for modeling coherence and cohesion of a text ; ( 2 ) we present a simple yet effective training mechanism to encode these linguistic properties ; and ( 3 ) we propose negative-critical sequence training , a variant of policy gradient method , which uses negative samples to construct its reward baseline . This paper proposes a neural approach to explicitly model cross-sentence linguistic properties , i.e. , coherence and cohesion , for long text generation . Despite the encouraging initial results , we only scratched the surface of the problem . The proposed method is yet to be significantly improved to meet the ultimate goal of generating meaningful and logical long-form texts . We cast the text generation as an RL problem and review recent work in Section 2 , and detail our approach in Section 3 . 2 RELATED WORK . Coherence and cohesion . Coherence and cohesion have been extensively studied in the computational linguistics community , particularly in the ‘ pre-deep-learning ’ era . Lack of formal specifications for coherence and cohesion ( Mani et al. , 1998 ) , resulted in many different formalism , such as Rhetorical Structure Theory ( Mann & Thompson , 1988 ) , and other forms of coherence and cohesion relations and their quantification ( Mani et al. , 1998 ; Hobbs , 1985 ; Hovy , 1988 ; McKeown , 1985 ; Cohen & Levesque , 1985 ; Hovy , 1991 ; Cristea et al. , 1998 ; Halliday & Hasan , 1996 ; Liddy , 1991 ; Van Dijk , 2013 ; Edmundson , 1969 ; Barzilay & Lapata , 2008 ) . In fact , the list is not exhaustive . However , modeling coherence and cohesion of a text using models parametrized by neural networks have not been previously explored . Word sequence generation in a reinforcement learning framework . A word sequence generation task can be framed as a reinforcement learning ( RL ) problem , in which the generatorG is acting as a policy π , with parameters θπ , and each generated word at time t , wt , can be viewed as an action to be chosen by the policy from a large discrete space , or vocabulary , conditioned on state st−1 = w≤t−1 , which encodes the previously generated text sequence . Let rt be the reward for a partially generated text sequence w≤t . We define the long-term expected reward J ( π ) = Es0∼q , π [ ∑∞ t=1 γ t−1rt ] , where q is the initial distribution of conditional input texts . Following Sutton et al . ( 1999 ) , the gradient of J with respect to θπ is ∇θπJ = Es∼ρπ , a∼π ( ·|s ) [ Qπ ( s , a ) ∇θπ log πθπ ( a|s ) ] where ρπ is the stationary distribution and Qπ ( s , a ) is the expected return from state s and taking action a , both following policy π . For brevity , we omit the derivation . In our work , we formulate text generation as an episodic RL problem with episode length L , rewards rL being available only at the end of episode and γ = 1 . There are many works on training neural language models using reward signals , such as Ranzato et al . ( 2015 ) and Paulus et al . ( 2017 ) . These works directly optimize for specific metrics , such as BLEU ( Papineni et al. , 2002 ) or ROUGE ( Lin & Hovy , 2003 ) , using REINFORCE ( Williams , 1992 ; Sutton et al. , 1999 ) . However , it is well-known that these metrics do not give a complete picture on the quality of the generated text . Only recently have there been efforts to provide more relevant quality objectives for which to optimize ( Li et al. , 2015 ; 2016a ; Holtzman et al. , 2018 ) the quality of interest such as consistency , repetition of text . But these works use the objective function to re-rank candidate outputs , not to reward or penalize outputs when they are generated in the first place . Li et al . ( 2016b ) constructed a set of reward models , such as information flow and semantic coherence , to tune the generator , yet they do not provide an ablation study to elaborate the relative contribution of these reward models individually . GANs for text generation . Another line of research is to use Generative Adversarial Networks ( GANs ) ( Goodfellow et al. , 2014 ) to incorporate feedback signals for text generation ( Yu et al. , 2017 ; Lin et al. , 2017 ; Zhang et al. , 2017c ; Guo et al. , 2017 ; Fedus et al. , 2018 ; Zhang et al. , 2018 ) . However , the discriminator in these works are trained to distinguish real texts from the generated ones , operating as a black-box rather than providing fine-grained feedback on particular linguistic aspects of the texts . In fact , Yang et al . ( 2018 ) has partially addressed this issue by using a trained language model as the discriminator . Although the discriminator provides a fine-grained feedback at the word-level , it does not critique on many important linguistic properties of generated texts , such as cohesion and coherence . These text generators , when facing a long-form text generation task that span multiple sentences , are by no means perfect and often exhibit some critical errors , such as a breakdown of local connections between consecutive sentences ( cohesion ) , let alone globally solid intention ( coherence ) . As a result , readers can easily take these cues and discriminate such generated texts against real texts . In this paper , we argue that the primary reason is the lack of an effective mechanism of measuring and controlling the text quality in the generation process . The method we propose in the next section is intended to address the problem . 3 MODEL . We assume that global coherence of a text depends to a large degree upon how its individual sentences with different meanings are organized . So we focus our evaluation of coherence solely on the sentence-level . If the sentences are not organized properly , the intention of the paragraph as a whole is obscure , regardless of seamless local connectivity between consecutive sentences . This is not to say that local connections between any two neighboring sentences should be overlooked . One can easily distinguish a model-generated sentence from a real one , simply by looking at whether the sentence followed by another sentence relates , besides grammar . We instill these two different yet important concepts in two discriminators , each operating on the sentence level and word level . Our models closely resemble successful models for computer vision , such as StackGAN ( Zhang et al. , 2017a ; b ) and PatchGAN ( Isola et al. , 2017 ) in that they all provide hierarchical signals to their corresponding generators , where the signals are derived from raw lowlevel data . We call the sentence-level discriminator the coherence discriminator Dcoherence , and the word-level discriminator the cohesion discriminator Dcohesion . 3.1 COHERENCE DISCRIMINATOR : DCOHERENCE This discriminator measures how likely two text chunks form a coherent paragraph . Let S : = [ s1 , s2 , ... , sn ] be the source text chunk that consists of n sentences , T : = [ t1 , t2 , ... , tm ] be the real target text chunk that consists of m sentences , and T̃ : = [ t̃1 , t̃2 , ... , t̃m̃ ] be the model-generated target text chunk that consists of m̃ sentences . Dcoherence is designed to distinguish a real pair ( S , T ) from a synthetic pair ( S , T̃ ) by assigning them with different scores , i.e. , Scorecoherence ( S , T ) > Scorecoherence ( S , T̃ ) . Design . Our design ofDcoherence is inspired by the Deep Structured Semantic Model ( DSSM ) ( Huang et al. , 2013 ; Gao et al. , 2014 ; Xu et al. , 2017 ) . Given source text chunk S and target text chunk T , Dcoherence computes their coherence score in two steps , as illustrated in Figure 1 . First , a pair of convolutional networks ( CNNs ) 1 or RNNs are applied to encode both S and T into two lowdimensional continuous vectors . Second , the coherence score is computed as the cosine similarity of the two vectors . The score is a real value between −1 and 1 , where 1 indicates the maximal coherence score , and −1 the minimal coherence score . Dcoherence measures how likely S and T add up to a single coherent passage . The score depends on the parameters of the two CNNs/RNNs , or in other words , how S and T are encoded . Since we focus solely on the sentence-level features , we view a text chunk as a sequence of sentences , and view each sentence as a bag-of-words . Therefore , we represent each word using its pre-trained word embedding vector ( Pennington et al. , 2014 ) and represent each sentence using a vector which takes the average of its word embedding vectors . A text chunk is then represented as a sequence of sentence vectors which are fed to the CNN/RNN ( either f ( · ) or g ( · ) in Figure 1 ) . Notice that f ( · ) and g ( · ) do not share parameters . The parameters of f ( · ) and g ( · ) are optimized in such a way that a real pair scores higher than a synthetic pair : ∆ ( θf , θg ) : = Dcoherence ( fθf ( S ) , gθg ( T ) ) −Dcoherence ( fθf ( S ) , gθg ( T̃ ) ) > 0 Formally , the task of optimizingDcoherence can be stated as follows . Given a set of training samples of the form ( ( S , T ) , ( S , T̃ ) ) ( i ) , i = 1 ... M , we optimize parameters ( θf , θg ) by minimizing the pairwise rank loss on training data defined as 1 M M∑ i=1 L ( ∆ ( θf , θg ) ( i ) ) where L ( · ) is a loss function , differentiable w.r.t . ( θf , θg ) . In the following subsection , we will describe in turn how we construct the pairwise training samples and the form of the loss function . Since Dcoherence is used as a pairwise ranker , we employ the metrics commonly used in information retrieval for evaluation , such as recall at K ( R @ K ) , which is defined as the fraction of correctly identifying an item in the TOP-K retrieved list ( Baeza-Yates & Ribeiro-Neto , 1999 ) . We present the retrieval results on test data in Table 2 . Training mechanism . How do we construct these list of candidate target sentences T , given the source sentences S ? We assume the T that follows an S in the data is a positive target sample , or the correct item to retrieve . Negative samples are constructed using three different methods within a batch while iterating through the training data , motivated by Wieting et al . ( 2016 ) : • Rotate T with S fixed ( mismatch S and T ) across a batch . For a single S , this method yields B − 1 negative samples , where B is the batch size . • Shuffle the sentence order once , different from its original order , known as a derangement , in each T to break coherence , and this yields one negative sample . • Combine the previous two methods : rotate T across a batch and shuffle sentences within T , yielding B − 1 negative samples . These 2B − 1 negative samples and a single positive sample , in sum , pose a significant challenge in learning . To fit this training task into a ranking framework , we optimize over Dcoherence ( fθf ( S ) , gθg ( T ) ) − AMλi∈ { 1 , ... ,2B−1 } [ Dcoherence ( fθf ( S ) , gθg ( T̃i ) ) ] where the arithmetic mean parametrized by λ : AMλ ( x ) = N∑ i wixi and wi = eλxi/ ∑ j e λxj . In our experiments , we fix λ = 2 and this assigns more weight to a more challenging negative sample2 . Notice that AMλ is the mean if λ = 0 , and approaches the max as λ → ∞ . Empirically , training the models using the weighted mean resulted in faster convergence , as opposed to using the single most challenging negative sample score ( max ) or the mean of all negative sample batch . 1We explored with deeper networks . However , the performance difference was marginal . For simplicity , we decided to use a 1-layer convolutional network architecture ( Kim , 2014 ; Collobert et al. , 2011 ) . 2We performed a coarse grid search over the values of λ and setting λ = 2 resulted in fast convergence to high recall scores on the dev dataset . 3.2 COHESION DISCRIMINATOR : DCOHESION Our second discriminator pays attention only to low-level features to capture a local connection between arbitrary two consecutive sentences because the pairwise connectivity influence readability . For simplicity , Dcohesion is similar toDcoherence , except that its architecture , input , and negative sample construction are modified to encode cohesion between any pair of sentences on the word-level . A single input sample to Dcohesion is a pair of two consecutive sentences : [ si,1 , si,2 , ... , si , n ] and [ si+1,1 , si+1,2 , ... , si+1 , m ] , where si , k denotes the k-th word in sentence i . We construct the negative samples using the three methods for training Dcoherence , where shuffling occurs on the word level within each sentence , rather than shuffling multiple sentences on the sentence level .

Summary: This paper addresses long-text generation, with a specific task of being given a prefix of a review and needing to add the next five sentences coherently.  The paper proposes adding two discriminators, one trained to maximize a cosine similarity between source sentences and target sentences (D_{coherence}) and one trained to maximize a cosine similarity between two consecutive sentences.  On some automatic metrics like BLEU and perplexity, an MLE model with these discriminators performs a little bit better than without.