Language Model Sentence Completion with a Parser-Driven Rhetorical Control Method (2024)

Abstract: Controlled text generation (CTG) seeks to guide large language model (LLM) output, that statistical language generation would conform to desired criteria. The current study presents a novel CTG algorithm that enforces adherence toward specific rhetorical relations in an LLM sentence-completion context by a parser-driven decoding scheme that requires no model fine-tuning. The method is validated both with automatic and human evaluation.

## introduction despite outstanding success, large language models (llms) are black-box in nature and perform unpredictably. they are known to generate nonfacts and to deviate from desired criteria for generation @xcite . controlled text generation (ctg) seeks to enforce constraints upon llmgenerated text, such as favoring the generation of pre-specified words or phrases or sentence structures, or requiring adherence to pre-specified communicative goals @xcite . for a piece of text to be articulate, it must present a cohesive story using grammatically correct components that are also logically related to one another. this paper presents a novel algorithm that attempts to influence the text generation behavior of an llm by mandating that certain rhetorical relationships exist between spans of text. the introduced algorithm incorporates a pre-existing parser that identifies discourse relationships among spans of text, within an llm's probabilistic process of generation of text tokens, to produce text components that satisfy desired logical relationships. in particular, given an input span of text, the approach generates the next span that holds a desired relation with the given input. the direct use of the proposed system is the downstream task of generation of an entire rhetorical structure theory (rst) tree. such guided generation could aid specialized domains such as machine translation, where different languages have different expected orderings of rhetorical relations @xcite . the results show that the proposed method retains generation quality of an llm, while enhancing it with a pronounced ability to control the rhetorical relations between adjacent sentence components. automatic and human evaluation verify the effectiveness of the control method in generating high-quality english text. ## related work large language models (llms), such as the gpt series (openai, 2023), palm (narang and chowdhery, 2022), llama @xcite , and bloom (hugging face, 2022) generate text autoregressively, generating the next token conditioned on previously generated text. llms, pretrained on vast corpora of text data, have demonstrated versatility in fluent text generation across domains @xcite . however, these probabilistic models generate text in a black-box manner without the user's full understanding or control of the underlying generative process. controlled text generation attempts to modify the generation of text by llms by exerting influence on the next token being produced. an attempt at controlling text generation includes baumler and ray's use of phrase-structure parses of sentences generated by a language model and a database of world knowledge to modify generated phrases by inserting fact-driven words as applicable (2022). @xcite utilize a common-sense database to append knowledge to a language model prompt, enabling the language model to incorporate relevant information. @xcite use prompt engineering to instruct a language model to generate sentences with specific lexical, syntactic, semantic, style, or length constraints. @xcite introduced a theory of discourse called rhetorical structure theory formally to articulate how clausal units in a sentence and between sentences relate to one another to deliver meaning coherently. rst represents a collection of elementary discourse units (edus) as a tree structure. although a descriptive theory, rst has been used to drive objectives in natural language processing, including summarization, machine translation, and generation @xcite @xcite . these early efforts in using rst to generate text were able to impose structure but were unable to generate fluent text. on the flip side llms are good at generating fluent text, although are not amenable to being explicitly controlled. the so-called plug-and-play approaches to ctg allow for controlled generation of text without expensive fine-tuning of the language models @xcite . for example, @xcite train a parser relevant to recipe generation and use it to re-rank the token distribution from a language model, resulting in controlled generation of recipes. building off the success of recent methods in integrating traditional computational linguistics tools, the present study integrates rst with large language modeling through a plug-and-play combination of an rst parser and a language model. to the best of our knowledge, there is no equivalent method against which to test our system. we attempted to utilize prompting to guide bloom 1.7b toward generation of relation completions as a baseline; but this smaller model showed no ability to complete these relations with prompt engineering. this further bolsters the proposed method because, through it, the model can generate accord-ing to instructions that the model otherwise could not follow. ## models the proposed method uses two models for text generation. the first is a general language model without any rst pretraining. the second is an rst parser. bloom 1.7b: bloom is a multilingual decoder-only transformer language model trained on the 1.61 terabyte roots corpus, which contains 46 natural languages alongside 13 programming languages @xcite . the current study uses the 1.7-billionparameter version of the model because of computational limitations for this study. a bloom model is decoder-only, allowing autoregressive generation of text. dmrst: rst parsing consists of two taskssegmentation and relation attribution. segmentation is the task of converting a document into a collection of edus, the basic units in rst. relation attribution, on the other hand, arranges these edus into a binary tree, assigning each edge to be a specific relation between two edus. dmrst segments and parses raw text into an rst tree @xcite . importantly for the present study, dmrst also can be configured to perform relation attribution for a preset segmentation upon a document. dmrst classifies between 42 relations, where varying nuclearity configurations count as different relations. each relation's name is of the form {relation}_{nuclearities}, where relation is any of 18 categories, such as contrast or attribution, and nuclearities is nn to mean the relation is between two nuclei, ns to mean the left component is a nucleus and the right component is a satellite, and sn for the other ordering of the nucleus and satellite. the code for dmrst is publicly available. the language model first yields a distribution across all tokens conditioned on the prompt and the already generated tokens. then, the rst parser reranks the top of the distribution to favor tokens that fit the desired relation. finally, the next token is selected from this re-ranked top of the distribution and the process continues until the parser detects the end of the edu. ## experiments the proposed text generation method is evaluated both by automatic measures and by human feedback. the method is tested with seven relations that were selected for their ease of understanding to lay annotators. four volunteer native english speakers each composed 20 short english sentences according to instructions (appendix b). the instructions requested that the sentences be diverse in content, including tense. the proposed method generated eight completions for each of these 80 sentencesseven for the seven relations being tested and one for no relation, that is, regular generation with the language model. the parameter values used in the generation are @xmath0.75, @xmath1, ⌧ = 0.1, ↵ = 0.7. for all completions, generation was forced, if it had not already stopped by itself, to cease after 30 tokens or a period had been generated. ## perturbation analysis knowing where the proposed method most compels an alteration in token choice to occur grants insight to the problem of ctg with rst. we measure the degree of perturbation for each step of generation in a way semi-independent of ↵, the generation parameter that determines how much the proposed method may perturb the language model's distribution. after the top-p nucleus vocabulary from the language model is obtained, the dmrst parser reranks each of these by creating a new token distribution, wherein each token is likely in as much as the parser sees the token to fit the desired relation. the difference between the score of the highest and lowest parser-scored token is a proxy for how much the parser will re-rank, or perturb, the regular distribution. when the difference is smaller, tokens are not re-ranked as much as when the difference is larger. this, when only considering a single step of generation, is a measure independent of ↵. figure 3 displays the average, across 560 generations, of this difference for each generation step. the generations comprise of seven completions influenced by the relations heretofore used for each of the 80 human-generated prompts. generation here used the same parameters as were used in experiments. after the first token's generation, which has an average of 0.42, the average difference drops to 0.18 and then after the fourth step below 0.1. hence, the most control is exerted during the generation of the first tokens, which makes sense when considering that the words that explicitly begin the relation completions tested in this study for english are often headed with specific words or phrases. one example is contrast_nn, for which completions typically begin with "but" or another adversative such as "instead." after generating this first word or phrase, the decreased value of the difference, in conjunction with human evaluation confirming that the proposed method maintains comparable fluency, means that the language model, now generating conditioned on this initial relation-specific start, successfully adjusts to the desired relation without much further assistance from the parser. ## conclusion validated by automatic and human evaluation, the proposed control method is able to enforce a rhetorical relation during english sentence completion without sacrificing fluency or reasonableness. the perplexity, fluency, and reasonableness metrics show that the proposed method does not degrade the quality of generated text while correct% and relation-fit indicates the control method's success.