Application of Existing Readability Methods to the U krainian Language: A Comprehensive Study (2025)

Abstract: The Ukrainian language currently lacks a well-developed framework for assessing text readability. This study addresses this gap by focusing on three key contributions. First, we present the creation of UkrTB, a Ukrainian-language corpus of texts categorized by reader age. Second, we conduct a statistical analysis of the corpus, evaluating key linguistic features such as sentence length, word complexity, and part-of-speech distribution. Third, we systematically assess the applicability of existing readability formulas, including Flesch, Flesch-Kincaid, Matskovskii, Pisarek, and Solnyshkina et al., to Ukrainian texts. Our findings indicate that readability models developed for English and other Slavic languages exhibit significant limitations when applied to Ukrainian. While some methods demonstrate partial correlation with expected readability levels, others produce inconsistent results, underscoring the need for a specialized readability metric tailored to Ukrainian. This work lays the foundation for further research in Ukrainian readability assessment and the development of language-specific models

## introduction research on quantitative readability assessment began in the 1940s with flesch's early work @xcite , leading to the development of various readability metrics. however, these methods were primarily designed for english and did not account for structural differences in other languages. english, as an analytical language, relies on a fixed word order and auxiliary words to convey meaning, whereas slavic languages, including ukrainian, use a more synthetic structure characterized by extensive inflection, case systems, and grammatical gender. these differences complicate the direct application of english-based readability models to ukrainian, as they fail to capture the complexity introduced by its flexible word order and morphological variation. ukrainian is characterised by rich morphology (cases, long word forms), syllabic structure (complex syllable divisions, less vowel reduction) and syntax (long sentences, free word order). polish has consonant clusters (ex. 'szczegółowy'). at the same time, russian is characterised by vowel reduction -this does not coincide with ukrainian, where vowels are fuller. formula coefficients must be calibrated on the ukrainian corpus of texts and tested on native speakers, otherwise they give incorrect results. since the 1960s, readability studies have been conducted for some slavic languages, particularly polish and russian, leading to the development of language-specific models. however, ukrainian remains underexplored in this context, despite having a substantial number of native speakers and a distinct grammatical system. the absence of a dedicated readability assessment framework for ukrainian presents a challenge for text classification, educational content adaptation, and nlp applications. existing readability formulas, whether developed for english or other slavic languages, may not be directly transferable to ukrainian due to its unique linguistic features. to address this gap, this study makes three key contributions. first, we construct a ukrainianlanguage text corpus categorized by readability levels to provide a foundation for further research. second, we conduct a statistical analysis of the corpus, evaluating linguistic features such as sentence structure, word complexity, and part-ofspeech distribution. third, we systematically assess the applicability of existing readability formulas, including those developed for english, polish, and russian, to determine their effectiveness for ukrainian. our findings will help establish whether current models can be adapted or if a new methodology is required to develop an accurate readability assessment framework for the ukrainian language. ## related work to consider the task of determining the applicability of existing methods for calculating readability to ukrainian-language texts, it is necessary to solve several successive tasks: prior research has explored readability assessment across multiple domains. studies on text complexity for second-language learners have focused @xcite on cefr-graded english datasets, addressing the challenge of limited annotated data. these works have adapted models trained on native corpora, leveraging domain adaptation and self-training techniques to improve performance. high accuracy and strong correlation coefficients suggest that similar approaches could be explored for the ukrainian language.lexical richness has also been shown to correlate with perceived quality in esl learners' oral narratives @xcite , suggesting that vocabulary diversity is an important dimension of readability and language proficiency assessment. efforts to standardize readability assessment in educational materials have also been undertaken, particularly in the context of ukrainian textbooks. research from lviv polytechnic @xcite ) investigated text complexity using the "chitanka" program, analyzing the accessibility of scientific content. the study emphasized the importance of systematically increasing textual complexity in educational materials, advocating for a more structured approach to readability in ukrainian academia. recent advancements in multilingual readability estimation highlight the potential of cross-lingual transfer learning. the paper "an open multilingual system for scoring readability of wikipedia" @xcite presents a novel approach to assessing the readability of wikipedia articles across multiple languages. it introduced a model trained on wikipedia articles in 14 languages, demonstrating significant improvements over previous benchmarks. by aligning document pairs across different languages, the model effectively assessed text complexity even in languages with limited resources, providing a strong foundation for future ukrainian readability models. the ukrmed corpus @xcite , focuses on ukrainian medical texts, including clinical protocols and forums, categorized by complexity. its creation involved preprocessing, tokenization, and statistical analysis, with crowdsourcing to improve quality. studies using pymorphy2 showed that frequencybased methods are insufficient, highlighting the need for advanced linguistic features and lexical resources-relevant for broader readability assessment. next studies @xcite ) highlight the diverse methodologies for readability evaluation and corpus development. while significant progress has been made in multilingual and domain-specific readability assessment, ukrainian remains underexplored. by leveraging insights from existing research, this study aims to construct a robust ukrainian readability framework, integrating statistical, linguistic, and computational approaches. they used the pymorphy2 morphological analyzer and stopword lists for preprocessing. their results underscored the need for specialized lexical resources and ontologies to simplify medical texts for better comprehension by non-specialist readers effectively. the article (cherednichenko and kanishcheva. 2021) examined the application of readability formulas to ukrainian medical texts using the ukrmed corpus, which categorizes texts into three levels of complexity: simple, moderate, and complex. the results indicated that while existing formulas produce similar rankings, medical texts remain inherently difficult to understand. the study suggests that further refinement of readability metrics and detailed text markup could improve accessibility for both non-native speakers and individuals with varying educational backgrounds. the paper @xcite ) investigated the impact of second-language acquisition (sla) research on readability classification. by integrating sla-based complexity measures with traditional readability metrics, the researchers improved classification accuracy. their model, which combined lexical and syntactic features, outperformed conventional methods, achieving over 93% accuracy in predicting text difficulty across different grade levels. this work demonstrates the potential of interdisciplinary approaches in enhancing automated readability assessment, particularly in educational and language-learning contexts. for russian texts, @xcite ) proposed a modified readability formula incorporating syntactic, lexical, and frequency features. tested on russian school textbooks, it showed improved accuracy over previous models and highlighted the potential of regression-based approaches for genre-specific readability assessment. similarly, research @xcite on polish texts evaluated multiple readability assessment methods, including traditional formulas such as gunning-fog and pisarek's method. in addition, the study introduced novel approaches, such as distributional lexical similarity and an automated taylor test based on statistical language modelling. the authors developed an online tool for readability evaluation, showing a strong correlation between various readability indices and effective classification of text complexity levels. corpus parameterization has also been a focus in readability studies @xcite , as highlighted in research on optimizing corpus balance and representativity. a hybrid methodology combining statistical models, expert evaluations, and adaptive monitoring was proposed to address the proportionality of text types in linguistic corpora. this study emphasizes the importance of dynamic, data-driven strategies in corpus design, ensuring that robust and wellstructured datasets support readability research. adapting readability assessment methods for the german language has seen significant advancements through various innovative approaches. recent research has focused on integrating machine learning and natural language processing techniques to enhance the accuracy and applicability of these methods. an online service has been developed that utilizes five statistical methods and two machine learning models, including bert, to evaluate german text readability at the sentence level @xcite . this tool is particularly beneficial in educational contexts, helping to assess the suitability of teaching materials for different grade levels. recent research in german has demonstrated the use of neural models (e.g., bert-based) for sentence-level readability prediction, showing potential for future adaptation to low-resource languages. @xcite , @xcite . ## methods this study follows a structured approach to evaluate the applicability of existing readability metrics to ukrainian texts. the first step involved constructing ukrtb, a corpus of 750 texts extracted from ukrainian educational materials, categorized into five readability levels corresponding to different school grades. the texts were preprocessed through cleaning, normalization, and tokenization. part-of-speech tagging was performed using pymorphy3, leveraging opencorpora dictionaries to ensure consistency in morphological analysis. we computed statistical features -including sentence length, word length, and syllable count -to quantitatively evaluate text readability. to assess readability, we applied six established formulas: flesch readability ease @xcite , flesch-kincaid grade level @xcite ), @xcite linear and nonlinear models, matskovskii's readability index @xcite , and solnyshkina et al.'s model @xcite . these metrics were chosen based on their relevance to english and slavic languages. each formula was systematically applied to the ukrtb corpus, producing readability scores for each text. the evaluation of these methods involved a correlation analysis using spearman and pearson coefficients to examine the relationship between formula scores and the intended educational levels. the correlation between computed readability scores and the predefined grade levels was quantified using both pearson's and spearman's coefficients. these statistical measures allow for evaluating the degree of linear and monotonic relationships, respectively. a distribution analysis was also conducted to assess intra-class variability, determining whether readability scores were consistent within the same grade level. ## collection of the dataset creating a text corpus for a source dataset is challenging, as the source data strongly influences the processing results. currently, there are no standard techniques for creating text corpora in linguistics. our method for collecting a corpus of texts has similarities with the weebit corpus, which involved weekly reader texts ranked by reader age @xcite . we propose our own method for collecting texts to create a training and general corpus based on the work of (cherednichenko and kanishcheva. 2021) @xcite . the main requirement for the corpus is to provide ukrainian text data ranked by readers' ages to study language problems. the corpus of ukrainian-language texts we are creating, which we call ukrtb -ukrainian textbooks 1 , was originally conceived as a corpus without saved tags and attributes. the first reason is the availability of modern tagging and attribution techniques, including the rapid creation of keyword 1 github.com/prykhodchenkosd/ukrtb lists using artificial intelligence elements. the second reason for not tagging is the perceived versatility of the corpus, which, as research progresses, is planned to be expanded to the size of weebit, i.e. about 6000 texts, divided into appropriate ages of potential readers, where texts range from folk tales to excerpts from history or physics textbooks. this study proposes the following process (fig. 1 ) based on the work @xcite to create a text corpus ranging by readers' ages in ukrainian. as data sources, texts from the electronic library formed by the ministry of education of ukraine, which contains all textbooks and auxiliary literature recommended by the ministry of education from the first to the 11th grade, are considered. these texts include both folk tales and samples of literature from the 11th to 21st centuries, as well as scientific explanations of, for example, physics or chemistry, it should be explained that there is no single textbook for the discipline studied in ukraine; an author may submit their textbook, which meets the specified criteria, to the ministry of education, get approval, and then their textbook will be included in the list of recommended textbooks. thus, on the recommended textbooks website, we can find 5 history textbooks for 7th grade and 6 ukrainian literature textbooks for 8th grade, which may differ in the selection of materials and writing style. nevertheless, the ministry's recommendation confirms that these texts fulfil the criteria set out for textbooks, which means that a pupil of the relevant grade will be able to read the material. the corpus design and validation processes were inspired by the works of vasyl starko @xcite , who is one of the creators of the brown corpus of the ukrainian language. obviously, his works are based on the ideas of creating the brown corpus of english (standard corpus of present-day edited american english, or shortened brown corpus). at the same time, we will note that the exact date of the creation of the text in this work is of little importance to us in comparison with the indication of the age audience of the reader. thus, according to our criteria, a folk tale recorded more than a hundred years ago but present in a modern textbook for the 3rd grade is suitable for inclusion in the corpus. the average number of words per page of a textbook depends on the format, font and text density. still, it averages around 250-300 words for a standard a4 page or a textbook without many illustrations and formulas. textbooks with a dense layout (e.g., high school textbooks) may have up to 400 words, and those with many graphics or examples (e.g., elementary school textbooks) may have about 200 words. to create this corpus, we used an average article length of 2 pages, i.e., an average of about 500 words on the topic of one lesson. the data are collected by dividing the texts by the reader's age group. we identify five levels of text complexity: level2, level3, level4, ks3, and gcse, similar to the age categories of the weebit corpus @xcite ) ks3 (key stage 3, uk educational stage for ages 11-14) and gcse (general certificate of secondary education, typically for ages 14-16). in the first stage, we collected equal amounts of data for each group using semantically similar sources, prioritizing textbooks from the same authors or editors. texts were then cleaned, encoded uniformly, and processed using tokenization, normalization, and statistical analysisat the processing stage, the program calculates statistical indicators of the text and metrics, such as the number of letters in a sentence, the number of words in a sentence, the average number of letters in a word, the average number of words in a sentence, the average number of syllables in a word, etc. the primary metrics and statistical indicators are given in to evaluate statistical information about parts of speech, the work used automatic pos tagging, carried out using the pymorphy3 2 project. the original versions 1 and 2 of the pymorphy project can currently be considered abandonware; the development and support of the latest versions of python is provided by its fork, pymorphy3. this project is based on the dictionaries of the opencorpora project, which are the basis for a significant number of scientific works in the field of text corpora processing @xcite , (d. kalugin-balashov. 2023), (tmienova and sus, 2 https://pypi.org/project/pymorphy3/ 2019), which allows us to talk about the quality and reliability of the assessments of this solution. the program using pymorphy3 produces a significant array of data on each of the words of the text, including the pos decision, which is issued in an abbreviated form foot_0 . as a result of processing the corpus texts, categorizations of words by parts of speech were obtained for each text passage that makes up the corpus. the results are presented in ## applicability of existing readability methods to ukrainian texts classic works on calculating the readability coefficient are the works of flesch, who defined a general readability formula and, later, kincaid derived the grade-level readability formula, which determined the readability index depending on the level of education of the reader. flesch's works were aimed at studying the english-language texts; they also served as a starting point for several similar studies of later times, which resulted in the methods for determining readability ari, smog, and several others, which were also focused on english-language texts. attempts to use these methods for other languages, even of the related germanic language group, usually concluded with the need to create a method entirely adapted to the corresponding language @xcite , @xcite , @xcite . for the languages of the slavic group, such studies began to be conducted in the 1960s-70s, gaining recognition for such languages as russian and polish, for which studies of readability the ukrainian language has the most remarkable lexical similarity with the belarusian language and, to a lesser extent, with russian and polish. studies of readability assessment methods for the belarusian language have also not been conducted, so in this paper, we will try to apply methods applicable to russian and polish, as well as the flesch and flesch-kincaid methods, which have become classics, for assessing the readability of ukrainian-language texts collected in the ukrtb corpus. on the way to this part of the study, we manually checked the automatically performed morphological partitioning of the corpus presented in flesch's work assumed the calculation of the readability coefficient based on such parameters as total words, sequences, and syllables. its consideration is still encountered today when considering readability coefficients for languages different from english. formula (1), proposed by flesch, was calculated for the texts of the ukrtb corpus, resulting in the graph presented in fig. 2 . this graph shows each class's maximum, average, and minimum values. in 1975, kincaid at al. improved flesch's readability method @xcite by including a proposed division by educational level, thus calibrating the readability coefficient by the level of the proposed reader. this method relies on the same parameters as in the original work -total words, sequences, and syllables, and formula 2 is a mathematical description of this method. in this study, all corpus texts were also processed using this method, resulting in the graphs presented in fig. 3 , which also determined the maximum, minimum, and average coefficients for each class of readers. although figure 3 shows an upward trend in flesch-kincaid scores with increasing grade level, the within-group variance is substantial. furthermore, the score ranges overlap significantly between neighboring levels, making it difficult to reliably distinguish readability tiers based on these scores alone. this explains the assessment of low correspondence, despite the overall trend. 𝐹𝐾 = 0.39 ( 𝑡𝑜𝑡𝑎𝑙 𝑤𝑜𝑟𝑑𝑠 𝑡𝑜𝑡𝑎𝑙 𝑠𝑒𝑛𝑡𝑒𝑛𝑐𝑒𝑠 ) + 11.8 ( 𝑡𝑜𝑡𝑎𝑙 𝑠𝑦𝑙𝑙𝑎𝑏𝑙𝑒𝑠 𝑡𝑜𝑡𝑎𝑙 𝑤𝑜𝑟𝑑𝑠 ) --15.59 (2) later, linguists developed readability methods for slavic languages, as english-based formulas proved unsuitable. matskovskii and pisarek's formulas were chosen due to their relevance to slavic structures. pisarek's polish method, adapted for inflected languages with complex morphology, is a strong candidate for ukrainian. matskovskii's formula, designed for russian, incorporates syntactic and lexical complexity, offering insights into how readability metrics perform in another east slavic language. given ukrainian's linguistic position between polish and russian, testing these models provides a comparative framework for assessing their applicability and identifying necessary modifications for a more accurate ukrainian readability metric. the author of the readability coefficient for the polish language is prof. @xcite proposed calculating the linear (3) and nonlinear (4) dependencies for the readability coefficient of the polish language based on asl (average number of words per sentence) and pcw (percentage of complex words). his work in this field is still used to analyze polish-language texts (broda et al., 2014 ). in our implementation, complex words (pcw) were defined as those exceeding three syllables and not belonging to a predefined list of stopwords or functional parts of speech. this operationalization was chosen to approximate the lexical difficulty as perceived by native readers, following practices in polish readability research. this study applied these techniques to ukrtb corpus. the results can be seen in figure 4 . some methods have been proposed for the russian language, which differs significantly from each other in the time of creation and the number and composition of the studied text features. in this study, we considered the method (5) @xcite , and based on such parameters as average sentence length and x3 is the percentage of words of more than 3 syllables in the text. we also considered a relatively recent work by m. solnyshkina and co-authors @xcite the results of applying formulas 5 and 6 to the corpus texts can be seen in fig. 5 . the overall figure comparing the applications of the above-described methods can be seen in fig. 6 . to facilitate the visual comparison of readability scores across methods with different scales and directionalities, we applied min-max normalization to each set of scores before plotting. the normalized scores are shown on a unified vertical axis, where higher values indicate higher perceived text complexity. this allows for approximate comparison of method behavior across grade levels. figure 5: classes of ukrtb by matskovskii and solnyshkina readability ease the examination of the above-described methods applied to the texts of the ukrtb corpus showed the following results: pisarek's methods, on average, showed a proportional increase in the expected complexity of texts. still, the difference is not obvious when considering the range of values within classes, which is especially important for the linear model. ## conclusions as a result of the research work, two main results were obtained, the first of which is the creation of a ukrainian-language corpus of texts divided by the assumed levels of education into 5 classes. the second result of this work is a series of tests of the applicability of existing methods for determining readability coefficients to corpus texts written in ukrainian. to summarize, some methods for determining readability in average results show dependencies between the level of readability and the assumed level of education. in some cases, this dependence is the opposite of what was declared by the creators of the method. nevertheless, the spread of values obtained as a result of processing is such that in the presence of two texts of different classes, false identifications of the readability level are possible not only with the nearest class but -through a class, two, and in some cases -through three, i.e., at the other end of the readability level and the assumed level of education. consequently, the methods proposed earlier for languages other than ukrainian can be applied to ukrainian texts only in limited cases, using additional coefficients, and for large average samples, which makes their real application difficult. for instance, one text designed for level 2 (age 7-8), consisting of short sentences and simple vocabulary, was assessed by the solnyshkina model as equivalent in complexity to gcse-level texts. this highlights how nonadapted formula parameters may misinterpret language features that are not penalized in ukrainian, such as long noun phrases or complex morphologies. in general, this study shows that readability formulas such as flesch, flesch-kincaid (for english), pisarek (for polish) and matskowski and solnyskina (for russian) are not suitable for ukrainian because of linguistic and statistical differences. formulas like flesch count the length of sentences and words, but their coefficients are customised for english. readers' cultural expectations are also important: complex constructions in ukrainian are perceived naturally, but existing formulas "penalise" them. the ukrainian language needs a new formula with empirical data and adapted parameters that consider specificity. thus, as a result of these studies, it can be concluded that additional research is necessary to determine the dependence of the parameters of ukrainian-language texts on the expected level of education and thus determine a specialized methodology for determining the readability coefficient for the ukrainian language.