# Fine-Tuning Large Language Models for Kazakh Text Simplification
## 3 Methodology
### Automatic Identification of Complex Kazakh Sentences  
Figure 1 outlines a two-step process for selecting complex sentences from a Kazakh corpus. Several Kazakh books were pre-processed and tokenized for this process. The first step focuses on identifying complex and common Kazakh words. In the second step, the text was split into sentences. Each sentence was evaluated using a heuristic method to decide whether it is complex. Sentences that meet the criteria were selected and saved as the final output. To identify morphologically simple and high-frequency lexical words in Kazakh, we define a scoring mechanism that integrates corpus-driven frequency statistics with morphological complexity estimation. The underlying objective is to assign higher priority to words that are both frequently observed in naturally occurring texts and exhibit minimal morphological complexity. Morphological complexity is estimated through QazAnalyzer [18; 19], a finite-state morphological analyzer developed for the Kazakh language. The analyzer decomposes surface word forms into a stem and a sequence of morphological suffixes. We approximate a word's structural complexity by computing the count of attached suffixes. In cases where the analyzer fails to produce a valid segmentation or classifies a token as \"unknown\", a predefined penalty is assigned to reflect maximal complexity, thereby discouraging the selection of such out-of-vocabulary items. Formally, for a given word \(w\), the scoring function is defined as  
\[\text{score}(w)=\text{morph}_{\text{score}}(w)-\alpha\cdot\log(\text{freq}(w)) \tag{2}\]  
where \(\text{morph}_{\text{score}}(w)\) denotes the number of suffixes extracted from the morphological analysis, and \(\text{freq}(w)\) is the word frequency obtained from a large-scale corpus. The parameter \(\alpha>0\) is a tunable coefficient that adjusts the influence of frequency in the overall ranking. Figure 1: Pipeline for selecting complex Kazakh sentences from text corpora. Table 1 presents Pearson correlation coefficients between final score, morphological complexity, and log frequency. A moderate positive correlation (\(r=0.558\)) is observed between final score and morphological score, indicating that structurally complex words tend to receive higher scores. A strong negative correlation (\(r=-0.886\)) between final score and log frequency confirms that frequently used words are consistently ranked lower. The weak negative correlation between morphological score and log frequency (\(r=-0.120\)) suggests that morphologically rich forms occur less frequently. Table 2 shows the top 20 most frequent and morphologically simple Kazakh words based on the lowest final scores. Most have a morph score of 1, reflecting minimal suffixation and occur frequently in the corpus. A few structurally richer forms (morph score 2) are included due to their high usage. To extract complex sentences, we implement a rule-based filtering approach that identifies complex candidates from an input set. Algorithm 1 first tokenizes each sentence and computes three primary features: (i) Total number of tokens; (ii) The count of known coordinating or subordinating conjunctions; (iii) The proportion of common words (extracted from the first step). \begin{table}
\begin{tabular}{l c c} \hline
**Variable Pair** & **Correlation (\(r\))** & \(p\)**-Value** \\ \hline score vs. morph\({}_{\text{score}}\) & \(0.558\) & \(<\)\(1\times 10^{-300}\) \\ score vs. Log freq & \(-0.886\) & \(<\)\(1\times 10^{-300}\) \\ morph\({}_{\text{score}}\) vs. Log freq & \(-0.120\) & \(1.084\times 10^{-158}\) \\ \hline \end{tabular}
\end{table} 
Table 1: Pearson correlation coefficients between key variables.