# Fine-Tuning Large Language Models for Kazakh Text Simplification
## 1 Introduction  
Text simplification is a task in natural language processing (NLP) that reduces linguistic complexity while preserving the original meaning. It has applications in education, public communication, and accessibility. In education, simplification allows students in primary and secondary schools to better understand complex texts. In public services, simplified documents improve the readability of legal and administrative content. In NLP pipelines, simplification can be used as a preprocessing step to improve downstream tasks such as machine translation, summarization, and information retrieval, especially for low-resource and morphologically rich languages like Kazakh. Classical rule-based approaches and early statistical methods show limited performance when applied to morphologically rich languages. In contrast, recent large-scale language models (LLMs) [1, 2, 3, 4], pre-trained on multilingual corpora and fine-tuned for generative tasks, have demonstrated promising results on simplification tasks in other languages [5, 6, 7]. Despite the growing availability of language technologies, Kazakh remains underrepresented in NLP applications, particularly in the area of text simplification. Unlike English and other high-resource languages, Kazakh lacks any publicly available automatictext-simplification tools, has no parallel simplification corpus, and to date no study has systematically evaluated large language models on this task. In this paper, we present a generative approach for Kazakh text simplification by fine-tuning multilingual large language models. To support model training, complex-simple sentence pairs are constructed through a heuristic-based method in a semi-synthetic manner. In the data construction process, the complexity of Kazakh words in a sentence was measured by their frequency and morphological complexity, and used these scores to filter common words from the text. The final selection of Kazakh complex sentences was performed using a heuristic approach, based on maximum token length, maximum allowed number of conjunctions, and the ratio of common words in the text. To support the development of automatic complexity classification for Kazakh, we conducted a comparative evaluation of two approaches: a simple heuristic method and a large language model (GPT-4o-mini). We randomly sampled sentences and manually annotated them as either complex or simple with the help of a native Kazakh speaker. The heuristic classifier, based on sentence length, achieved better overall accuracy and balanced precision and recall. Based on the assembled dataset, we fine-tuned various multilingual LLMs for the Kazakh text simplification task. We evaluated both instruction-tuned and LLMs, including domain-specific LLMs such as kazLLM and Sherkala, as well as general-purpose LLMs in a zero-shot setting. To benchmark performance, we introduce KazSim, an instruction-fine-tuned model optimized specifically for simplification. Evaluations were conducted on two test sets: one automatically constructed from the same pipeline used during training, and another semi-manually curated benchmark designed to reflect more natural simplification patterns. We present results based on standard evaluation metrics such as BLEU, ROUGE, and SARI. Experimental results showed that zero-shot and domain-adapted models are limited in their ability to produce structurally simplified and length-controlled outputs. In contrast, KazSim achieved consistently better scores across all metrics and evaluation settings, confirming the importance of task-specific supervision and targeted data construction for simplification in low-resource languages. We further investigated the impact of instruction language by comparing English and Kazakh prompts for the task. While most models showed comparable performance across both settings, slight gains were observed when prompts are given in Kazakh, particularly for zero-shot and domain-specific models.