From Complex Word Identification to Substitution: Instruction-Tuned Language Models for Lexical Simplification (2025)

Abstract: Lexical-level sentence simplification is essential for improving text accessibility, yet traditional methods often struggle to dynamically identify complex terms and generate contextually appropriate substitutions, resulting in limited generalization. While prompt-based approaches with large language models (LLMs) have shown strong performance and adaptability, they often lack interpretability and are prone to hallucinating. This study proposes a fine-tuning approach for mid-sized LLMs to emulate the lexical simplification pipeline. We transform complex word identification datasets into an instruction–response format to support instruction tuning. Experimental results show that our method substantially enhances complex word identification accuracy with reduced hallucinations while achieving competitive performance on lexical simplification benchmarks. Furthermore, we find that integrating fine-tuning with prompt engineering reduces dependency on manual prompt optimization, leading to a more efficient simplification framework.

## introduction lexical simplification (ls) aims to enhance text readability and understandability by replacing complex words or phrases with simpler alternatives without shifting the original meaning or introducing grammatical errors. traditional ls methodologies typically follow the pipeline, including identifying complex components, finding substitution candidates, and choosing the optimal candidate @xcite @xcite @xcite . although this strategy is widely adopted, it suffers from fundamental limitations in both design and execution. in particular, the sequential architecture is prone to error propagation, where errors in early steps negatively impact the quality of the subsequent outcomes. consequently, accurately identifying complex words and phrases within a given sentence plays a pivotal role in the entire ls system. however, existing ls systems identify complex words by evaluating each word separately. the word semantic complexity relies on its context in real-world linguistic environments. the isolated evaluation strategy tends to reduce the recall of the complex word identification (cwi) module, lowering overall system performance. although n-gram-based tokenization strategies can partially alleviate this issue, this static segmentation scheme limits its effectiveness in dealing with dynamic language environments @xcite . furthermore, previous approaches face challenges in generating appropriate substitutions and effectively ranking them. static approaches, such as vector space-based approaches @xcite , rank the substitute candidates by measuring the lexical semantic distance, which may result in semantic drift @xcite or part-of-speech (pos) tag mismatching @xcite . although dynamic approaches mitigate these limitations by leveraging pre-trained language models that utilize contextual knowledge to generate substitutes @xcite , similar to static approaches, these models are constrained to replace complex terms using isolated lexical units, which results in limited effectiveness in handling complex multi-word terms. as shown later in section 4.2, our fine-tuned system is competitive with traditional models in preserving both grammaticality and adequacy. large language models have demonstrated remarkable language understanding capabilities in text generation tasks. using llms enables effective adaptation to diverse linguistic contexts, efficiently identifying complex words and phrases, and generating appropriate substitutions by analyzing the entire context @xcite . however, since llms suffer from hallucinations, inconsistency, and limited controllability, depending on prompting alone is unreliable for ls tasks. moreover, llms are sensitive to the prompt format, leading to an extensive manual design and tuning, making the development process both time-consuming and resource-intensive. this paper proposes an ls approach that finetunes llms on a modified corpus and instructs the model to identify and simplify complex terms within given sentences. the experimental results demonstrate that the proposed system can identify variable-length phrases or single words by leveraging its advanced language understanding capability. it simultaneously replaces all complex terms during simplification, while avoiding semantic drift arising from iterative substitution processes in traditional approaches. moreover, this system addresses the limitations of purely prompt-based methods. since our system only requires the user to write the instructions in the same format as the training corpora, it does not require users to engage in tuning prompt format. besides, the fine-tuning process effectively enhances the accuracy rate with decreasing hallucinations. to enable llms to comprehend and execute instructions precisely, we construct an ls-instruction-answer (ls-i-a) dataset based on the cwig3g2 english dataset @xcite . experimental results indicate that the proposed approach, integrating fine-tuning with a prompt mechanism, achieves superior simplification efficiency compared with traditional ls pipelines. moreover, unlike few-shot and zero-shot methods, this approach eliminates the requirements for prompt tuning while effectively mitigating hallucination rates. the key contributions of this paper are as follows. ## related works traditional ls approaches follow a three-step pipeline-complex word identification, substitution generation, and ranking-yet suffer from error propagation and context-insensitive evaluations @xcite . to address these issues, edit-based models such as editnts @xcite and edit-ts @xcite perform explicit tokenlevel operations (e.g., delete, replace, keep), achieving practical simplification with interpretable edits. similarly, grs @xcite combines generation and revision in an unsupervised manner, incorporating paraphrasing and deletion at the lexical level. dress-ls @xcite introduces a sequence-to-sequence model trained with reinforcement learning to optimize simplification quality, but offers limited control over specific linguistic properties. access @xcite extends this by enabling controllable simplification through attribute-specific constraints such as length and lexical complexity. despite these advances, many of these systems rely on rigid heuristics or lack the scalability to handle diverse inputs dynamically. recent work also explores the use of llms such as llama @xcite and @xcite for lexical simplification @xcite . with strong instruction-following and contextual understanding abilities, these models can simplify complex terms in context via prompting or fine-tuning, offering greater flexibility and scalability. however, they often suffer from limited interpretability and occasional hallucinations, making them less reliable for controlled ls tasks. in contrast, our work leverages llms fine-tuned on a structured instruction-based dataset to identify and directly simplify complex terms in context. unlike edit-based systems or prompt-only llm approaches, our method balances controllability and robustness while mitigating hallucinations and format sensitivity. figure 1 : the workflow of the ls system: it first identifies complex words using cwi instructions. then, it builds ls instructions by utilizing the extracted complex words. finally, the system outputs the simplified versions under the guidance of the constructed ls instructions. ## methodologies we design a two-stage workflow for lexical simplification using a large language model, as figure 1 shows. the first stage identifies complex words and phrases within a given sentence. in the second stage, the llm is prompted with the original sentence and the list of identified complex terms. it is instructed to generate simpler, contextually appropriate alternatives that preserve the original meaning and ensure grammatical correctness. the model may also perform minor syntactic adjustments such as reordering words, modifying prepositions, or removing redundant tokens to maintain fluency. this approach leverages the llm's contextual understanding and generation capabilities to perform high-quality, semantic-preserving simplification without requiring manually crafted rules or external linguistic resources. in the following sections, we detail methodologies used in designing this ls system, including constructing the ls-i-a corpus, fine-tuning strategy for llms, and evaluation metrics. ## analysis as shown in to gain deeper insight into the behavior of our instruction-tuned lexical simplification system, we analyze the sari subcomponents alongside the structural and lexical metrics presented in figures 2 and 3 . these fine-grained evaluations go beyond aggregate performance scores, offering a more nuanced understanding of how the model balances adequacy, fluency, and simplicity in its simplifica-tion strategy. ## conclusion this study proposes a lexical-level sentence simplification system based on llms that emulate the traditional ls pipeline. we revise the cwig3g2 dataset to develop this system and construct a new instruction-answer corpus, ls-i-a, for fine-tuning llms. experimental results demonstrate that finetuning significantly enhances llm performance in cwi and ls tasks. on ls tasks, our system achieves performance comparable to several stateof-the-art models. despite the strong performance, our system still requires further improvements. notably, while ls-i-a performs well in practice, it lacks reasons for explaining the selection of complex words and the generation of simplified alternatives. as a result, the system does not support chain-of-thought reasoning in either the identification or substitution stages, limiting the inter-pretability of its outputs. in future work, we aim to refine the ls-i-a dataset and incorporate strategies that enhance the system's reasoning capabilities.