Standardize: Aligning Language Models with Expert-Defined Standards for Content Generation (2024)

Abstract: Domain experts across engineering, healthcare, and education follow strict standards for producing quality content such as technical manuals, medication instructions, and children’s reading materials. However, current works in controllable text generation have yet to explore using these standards as references for control. Towards this end, we introduce Standardize, a retrieval-style in-context learning-based framework to guide large language models to align with expert-defined standards. Focusing on English language standards in the education domain as a use case, we consider the Common European Framework of Reference for Languages (CEFR) and Common Core Standards (CCS) for the task of open-ended content generation. Our findings show that models can gain 45% to 100% increase in precise accuracy across open and commercial LLMs evaluated, demonstrating that the use of knowledge artifacts extracted from standards and integrating them in the generation process can effectively guide models to produce better standard-aligned content.

## introduction one of the most realized benefits of large language model (llm) research is how it became widely adopted by the public. in particular, the rise of chatstyle model interfaces, such as chatgpt and perplexity, has allowed non-technical users to fully utilize these tools in accomplishing day-to-day tasks and activities, such as getting help with writing, documenting code, and providing recommendations. a key technological advancement behind this is the use of reward-based methods such as reinforcement learning for human feedback (rlhf, @xcite ), which embeds human preferences to generative models for better-aligned outputs with respect to the task at hand. given this prompt: in the dark old forest up ahead, a solitary figure emerged from the corner of the shadowy grove… continue the story and make sure they are readable for b1 learners in the cefr scale and observes the following specifications: ## standard-aligned content generation (standard-ctg) given the importance of adhering to expert-defined standards in the context of language assessment, we introduce a new task we refer to as standardaligned content generation (standard-ctg). the overarching goal of standard-ctg is to pave the way for new approaches that aim to integrate the conventional methodologies of controllable text generation in nlp with actual constraints provided by domain experts across interdisciplinary fields such as education, engineering, and medicine through documented standards. to align with terminologies used in education and other noncomputing literature, in this work, we use the term content generation instead of text generation as usually seen in technical nlp literature. we represent the task of standard-ctg using the following formulation: ## the standardize framework given that expert-defined standards are naturally information-rich, lengthy, and complex, our main hypothesis in this study is that in order for a generative language model to produce content that is aligned with the specifications provided by a standard, the information found in the standard must be considered in the generation process. the challenge then is redirected towards how any information extracted can be represented as something that the generative model will find useful. towards addressing standard-ctg, we propose standardize, a retrieval-style in-context learning-based framework that exploits the rich information found in standards and transforms this into knowledge artifacts to improve the quality of content produced by generative models. figure 1 encapsulates this framework in a visual manner. in the succeeding sections, we discuss the proposed standardize framework more thoroughly. target specification extraction is performed first to obtain informative tags in the prompt and to correctly match this information within the standards. for academic standards in language assessment, these specifications should provide information about who will be content delivered to (target audience) and using what specific standard out of many (cefr or ccs). thus, these two information tags are the basic required input for the process. as an example shown in figure 1 , the extracted specifications provided in the prompt are a2 readers, which points to a particular group of learners requiring low-leveled reading materials, and cefr scale, which denotes the selected standard where properties of a2-level texts are described. specification lookup and retrieval is then performed next upon extracting the target specifications. a lookup process is done to find a match with the selected standard, usually in the form of a database or an external machine-readable file. in this work, we simply transformed the level-specific descriptors from natova (2021) into a .csv file. the information from the standard in the form of aspects (or characteristics) that match the target specifications is then retrieved. the length and complexity of a standard's level of information regarding its specifications may vary. as shown in figure 1 for the cefr standard, the retrieved information that matches the desired level of complexity for the target audience (a2 readers) can be checked at knowledge augmentation is done last but is the most important process of the pipeline. we propose a further technical augmentation of information found in standards to obtain knowledge artifacts in the prompts. these knowledge artifacts can range from simple additional information already present in the standard to complex representations, such as incorporating actual linguistic features to control the granularity of the generation process. re-cent works surveying the performance of open and closed models have shown that non-informative style of prompting language models, such as the teacher style shown in figure 1 , is effective only to a certain extent and may be biased towards content generation in lower levels, such as a2 or b1 in the cefr standards @xcite . ## knowledge artifacts for standardize in this section, we discuss the knowledge artifacts kstandard extracted from the two educational standards d standard used in the standardize framework and how they are integrated into the generation setup via in-context learning. baseline (teacher style) we treat the teacher style method as seen in figure 1 , where a simple, non-enriched prompt contains the target category from each standard, as the baseline for performance. we use this term in observance of how non-technical users, especially teachers, interact with generative chat interfaces @xcite . ## results and discussion we discuss the results of our experiments procedures with the methods from the standardize framework. ## implications to generative models for education we discuss important points highlighting the real-world implications of our study within and beyond language model experimentations. ## related work research in complexity-controlled generation has explored diverse variables in terms of text format, granularity, and task variation. the work of @xcite introduced controlling for specific complexity in the machine translation task. the following works of @xcite . ## conclusion in this work, we proposed the standardize framework using knowledge artifacts that allowed large language models such as llama2 and gpt-4 to gain significant performance boosts (45% -100%) on generating content aligned with educational standards as well as preserving important narrative qualities such as fluency, grammaticality, coherence, and grade distinctness. from this, we see a very promising potential for cross-domain and cross-standard generalization of our proposed method with the range of educational contexts around the world and invite future work to build on our baseline models.