B asaha C orpus: An Expanded Linguistic Resource for Readability Assessment in C entral P hilippine Languages (2023)

Abstract: Current research on automatic readability assessment (ARA) has focused on improving the performance of models in high-resource languages such as English. In this work, we introduce and release BasahaCorpus as part of an initiative aimed at expanding available corpora and baseline models for readability assessment in lower resource languages in the Philippines. We compiled a corpus of short fictional narratives written in Hiligaynon, Minasbate, Karay-a, and Rinconada—languages belonging to the Central Philippine family tree subgroup—to train ARA models using surface-level, syllable-pattern, and n-gram overlap features. We also propose a new hierarchical cross-lingual modeling approach that takes advantage of a language’s placement in the family tree to increase the amount of available training data. Our study yields encouraging results that support previous work showcasing the efficacy of cross-lingual models in low-resource settings, as well as similarities in highly informative linguistic features for mutually intelligible languages.

## introduction to ensure optimal reading comprehension, literary resources such as books need to be assigned to readers based on their reading level assessment @xcite . readability assessment can be tackled with a variety of methods ranging from the application of rule-based approaches using such widely accepted formulas as @xcite , to the use of software for linguistic feature extraction such as @xcite or lftk @xcite , to the application of extensive machine learning models @xcite . recently, the latter have been the focus of research on ara due to the availability of increasingly complex models, including deep learning architectures, that yield figure 1 : the central subgroup of the philippine language family tree highlighting the origins of the target languages hiligaynon, minasbate, karay-a, and rinconada (both underlined and marked with *). tagalog, bikol, and cebuano are also underlined as they are part of further experiments. the complete visualization of this language tree can be found in @xcite . beyond the limelight on high-resource languages like english, languages that do require extensive research efforts and initiatives are commonly considered low-resource. one such group of languages comprises over 170 living languages spoken in the philippines, and one of the three concluding recommendations in the final report of the five-year (2013-2018) usaid basa pilipinas ("read philippines") program 2 targeting the im-provement of literacy in the philippines was the "provision of curriculum-based teaching and learning materials (tlms) and quality supplementary reading materials". this entails the need for a variety of age-appropriate texts for young learners in the philippines to be accessible at home and in school. following this, the department of education further encouraged the development of reading materials for the mother tongue-based multilingual education (mtb-mle) scheme implemented at primary levels of school education in the philippines. 3 thus, the philippine education sector needs access not only to sufficient age-appropriate learning and reading materials but also to automated tools for assessing text difficulty that can work effectively for the variety of languages covered by mtb-mle. this work contributes a number of resources to the philippine language research landscape, catering to the challenges identified above, and aligns with other recent research efforts aimed at lowresource languages such as the indonlp @xcite . first, to increase accessibility and encourage more research endeavors in low-resource philippine languages, we collect and release basahacorpus, foot_3 a compilation of children's narratives and short stories from let's read asia online library in four philippine languages (hiligaynon, minasbate, karay-a, and rinconda). second, we train a number of baseline readability assessment models using various cross-lingual setups leveraging the hierarchy in the language family tree. lastly, we apply a simple model interpretation technique to identify how different linguistic features act as predictors of text complexity for each language. ## a closer look into central philippine languages the philippines is an archipelago with a rich linguistic background, as evidenced by over 170 languages at the backbone of its culture. 5 in particular, the central philippine language subtree is considered the largest among other subtrees as it is composed of a number of branches, including the national language tagalog (also known as filipino), the bikol languages, the bisayan languages, and the east mindanao languages @xcite . focusing on the languages of interest for this study, we show where hiligaynon, minasbate, karay-a, and rinconada are situated in this tree (see figure 1 ) and how they are classified within specific subgroups @xcite . in the following sections, we integrate the hierarchical relationships of these four languages with other languages in the family tree into our experiments. ## linguistic features for low-resource languages due to the lack of available nlp tools to extract advanced linguistic information from texts that would work for all four target languages as well as enough data to use deep learning approaches, we derive the same linguistic features as used by the previous work on ara in other low-resource philippine languages such as tagalog @xcite @xcite and @xcite . however, we note that this study is the first to explore baseline traditional features. we extract 7 frequencybased features using count-based text characteristics. despite being often considered simplistic, these predictors have consistently proven to be effective in text readability detection @xcite . as in the previous work on a cebuano dataset imperial et al. (2022), our extracted text-level features include the total number of unique words, total number of words per document, average word length per document, average number of syllables (per word), total sentence count per document, average sentence length per document, and the total number of polysyllable words. syllable-pattern orthographic features. we also extract 11 orthographic predictors taking into account all possible syllable patterns in the philippine language space. these consonant (c) and vowel (v) patterns include v, cv, vc, cvc, vcc, ccv, cvcc, ccvcc, ccvccc. we also extract consonant clusters and measure the average length of consonant groups in a word without an intervening vowel which have been proven to influence reading difficulty @xcite . ## hierarchical cross-lingual modeling cross-lingual modeling for automatic readability assessment typically involves extracting languageindependent features or experimenting with a classifier trained on the data in one language and applied to comparable data in another language (madrazo azpiazu and pera, 2020; @xcite @xcite . in this study, we introduce a new approach to training readability assessment models using close relationships between languages in the family tree, which we refer to as hierarchy-based cross-lingual modeling. for this setup, our cross-lingual training process is divided into iterations involving different combinations of language data in the training split with respect to the hierarchy the languages occupy in the central philippine language subtree, as illustrated in figure 1 . we list all such feature combinations below: monolingual (l). this setup uses the standard train-test split involving one language only. lang + parent lang (l+p). this setup combines the parent language or lingua franca with respect to their subgroup. for hiligaynon, minasbate, and karay-a, belonging to the bisayan languages in the visayas region, we add the cebuano language, while for rinconada, classified as a member of the bikol languages belonging to the luzon region, we add the central bikol language both from imperial and kochmar (2023). lang + national lang (l+n). this setup combines the four target languages with tagalog in the training data extracted from imperial and kochmar (2023), imperial and ong (2020) and @xcite . tagalog is the recognized national language taught as a subject in all elementary and secondary schools in the philippines. ## related work in the past, cross-lingual modeling was applied to classification-based ara both for non-related @xcite @xcite @xcite and highly-related languages (imperial and kochmar, 2023), with the latter reporting favorable results under predefined language similarity constraints such as high n-gram overlap. 7 research efforts that greatly contribute to 7 here, we interpret language relatedness as a measure of similarity of linguistic characteristics such as n-gram overlap the development of low-resource and cross-lingual readability assessment systems often focus on corpus building and the development of baseline models. previous efforts of this kind have covered a wide array of languages, including bangla @xcite , tibetan @xcite , arabic @xcite , vietnamese @xcite , bengali @xcite @xcite . as compared to the previous works, ours is the first study in readability assessment that investigates the effects of different language hierarchies in modeling text complexity for languages belonging to different subgroups of a greater family tree, with the focus on central philippine languages. ## summary we introduce basahacorpus, a compilation of language resources that includes a collected corpus of short stories and baseline readability assessment models for four central philippine languages. we show that, through a hierarchical crosslingual modeling approach, a readability model trained with all the available philippine language data generally performs better compared to using single-language datasets. through model interpretation, we also provide further support for the use of frequency-based features such as word and sentence counts as effective predictors of complexity in philippine languages. this study serves as a response to the call for more research efforts, theoretically grounded baselines, and accessible data for low-resource languages. @xcite .