C ree Corpus: A Collection of nêhiyawêwin Resources (2022)

Abstract: Plains Cree (nêhiyawêwin) is an Indigenous language that is spoken in Canada and the USA. It is the most widely spoken dialect of Cree and a morphologically complex language that is polysynthetic, highly inflective, and agglutinative. It is an extremely low resource language, with no existing corpus that is both available and prepared for supporting the development of language technologies. To support nêhiyawêwin revitalization and preservation, we developed a corpus covering diverse genres, time periods, and texts for a variety of intended audiences. The data has been verified and cleaned; it is ready for use in developing language technologies for nêhiyawêwin. The corpus includes the corresponding English phrases or audio files where available. We demonstrate the utility of the corpus through its community use and its use to build language technologies that can provide the types of support that community members have expressed are desirable. The corpus is available for public use.

## introduction recent work with indigenous persons has shown that some want advanced technologies to support the learning and use of their languages. the cree and métis persons involved in this study stated a desire for technologies such as an app to help with learning the structure of the language for conversation, translation, and ai agents that resemble a speaker @xcite . participants wanted these tools to support interaction in nêhiyawêwin (plains cree) or the learning of this language. all of these larger ideas are dependent on core language technologies such as language models, speech recognition, speech synthesis, or machine translation. however, a lack of publicly available corpora hinders the development of such technologies for low-resource languages like nêhiyawêwin. government policies have contributed towards supporting the preservation and revitalization of some indigenous languages, e.g., inuktitut @xcite . however, many have not benefited from this level of support for developing resources and technologies. recently, some government informational material such as voter guides or covid-19 pamphlets have been translated into nêhiyawêwin. nevertheless, the availability of resources is still limited and short texts or other resources are distributed across libraries and the internet. to understand why this is the case, we need to reflect on the colonial practices that have attempted to eradicate a language and people. previous and on-going government policies and practices, such as the implementation of residential schools @xcite , have left a small number of fluent speakers and language resources for nêhiyawêwinspeaking communities. these practices prevented and continue to prevent the development of language technologies because state-of-the-art statistical and neural models require large amounts of text. to work towards addressing this issue, we created a nêhiyawêwin corpus from various sources. our corpus is composed of 49,038 words and 3,727 lines of text in standard roman orthography (sro), 10 texts in syllabics, and 1,026 lines of english-nêhiyawêwin parallel data. to the best of our knowledge, this is the first collection of processed nêhiyawêwin data ready for use to build language technologies. the most similar existing work includes a small collection of nêhiyawêwin text, lexical, and audio resources in their original formats (open language archives community). there is also a morphosyntactic tagged corpus @xcite which can be accessed by searching for words, lemmas, and mor-phosyntactic information through a web interface. a targeted corpus of child-directed speech in cree has been shared through the acqdiv database @xcite . however, this corpus contains materials in the northern dialect of east cree (iyiyiu-ayamiwin) rather than plains cree @xcite . in response to the limited availability of resources and tools, this work contributes a collection of ready to use resources to enable the development of language technologies that can support the preservation and revitalization of nêhiyawêwin. we demonstrate the practicality of the corpus through its use by community-based teachers of nêhiyawêwin. using these materials has informed their lesson plans. further, we describe the ongoing development of predictive language models using the contributed corpus. these models enable predictive text that is expected to provide some of the language support needs that have been expressed by nêhiyawêwin speakers. with this work, we aim to inspire future data collection and sharing of nêhiyawêwin resources that are aligned with community interests. ## nêhiyawêwin and technology plains cree is called nêhiyawêwin by its speakers, and it is not capitalized. nêhiyawêwin is a widely-spoken dialect of the indigenous language that english-speakers call cree: nêhiyawêwin is the mother tongue for approximately 3,655 speakers, and it is the language spoken most at home for approximately 2,165 persons @xcite . nêhiyawêwin is an extremely low resource language, with the official designation of being a "developing" language; it is at stage 5 on the expanded graded intergenerational disruption scale (egids) (lewis and simons, 2010) so it "is in vigorous use, with literature in a standardized form being used by some though this is not yet widespread or sustainable". therefore, the ability to create language technologies for nêhiyawêwin is limited due to the minimal amount of monolingual and parallel data available. current language technologies for nêhiyawêwin include finite state transducers (fsts) that have been used for tasks such generating word forms and conjugating verbs in online dictionaries @xcite , representing nominal morphology @xcite , and spell checking @xcite . it is not surprising that fsts are one of the few technologies that exist given that nêhiyawêwin is a polysynthetic, agglutinative, and highly inflective language, which complicates the task of creating language technologies. these characteristics allow the meaning of a single token or word to map to that of a full phrase or sentence in english. for example, 'kimîciso' maps to 'you all eat' in english. nêhiyawêwin has two writing systems: sro and syllabics. a single character in syllabics represents one or more sro characters (e.g., σ is ni in sro and △ is i). complicating this, is the variability in how these writing systems have been used and continue to be used across regions and time. this variability means that choices must be made with respect to the writing systems and 'standards' that are followed when developing language technologies. these are difficult choices and each community may have different preferences, which means that tools for converting across varied writing systems would help to maintain community norms. an example of such a tool is the sro-syllabics converter @xcite . while any one project cannot address all considerations, these considerations are an important part of developing language technologies to support the revitalization and use of this language. ## corpus our corpus contains text from several domains making it a diverse collection of nêhiyawêwin resources (see the material is organized into folders by category or source along with its copyright information for how the public can use them. where nêhiyawêwin-english parallel texts exist, the folder contains a cleaned and aligned version of these texts; a given line in one language file corresponds to the same line in the other language file. syllabics versions of texts are provided where available. some texts also have an accompanying audio file. before adding a text to our corpus, we checked the copyright and license or obtained permission from the content creator. we provide a bag-ofwords (bow) representation when text was under copyright or the content owners felt this was an acceptable alternative to sharing the original text. these bow files contain a list of words from the original text and their usage counts. as these files only contain individual words, there is no nêhiyawêwin-english mapping because there is often no one-to-one translation between nêhiyawêwin and english words. ## creating the dataset to build this corpus we first identified sources of nêhiyawêwin text. we then extracted the text. following extraction, we aligned the texts across languages and performed additional processing. ## ethical considerations now that we have described how the corpus was created, we need to discuss ethical considerations around the creation and use of such resources. the process of creating language technologies for any community of speakers should be guided by the goals and interests of the respective community. natural language processing (nlp) research should directly involve the language communities for which the technologies are being designed, as it will directly impact the speakers of the language. further, the process of constructing these technologies should be clear to the community so there is an understanding of the data required for the model and how it will be used. for example, communities may wish to see language technologies such as text-to-speech to honor an oral tradition. however, these systems require an underlying model trained on corresponding audio and text for the language, which may or may not be in accordance with a community's wishes. in direct terms, the existence of this corpus in itself is not an invitation to make indigenous language models and technologies independently and without consultation. as discussed by pine and turin (2017), successful indigenous language revitalization projects must be "grounded in local understandings of impact and success, rooted in the lived experiences and aspirations of indigenous communities." an important consideration when developing language technologies using corpora and language models is the nature of the language used to train those models. for example, language models trained on internet texts (e.g., gpt-3) have been subject to scrutiny following the revelation of racist and generally offensive outputs @xcite . those who use the developed nêhiyawêwin corpus should note the potential for problematic outcomes when the data is used to support certain types of language technologies. this potential comes from the inclusion of biblical texts. while biblical texts are widely used for tasks such as machine translation @xcite , they could advance the harmful legacies of christianity-related efforts and government policies that used religion to control and harm indigenous groups @xcite . the translation of bibles into local indigenous languages was a means of furthering colonization @xcite . in addition, there are certain bible passages within our corpus that may be considered violent or aggressive in nature, e.g., "may sinners be destroyed from the earth. . . may the wicked be no more" (psalm 1). this kind of text, paired with the history between the church and indigenous peoples, should be used with caution, especially when designing language technologies that produce language (e.g., machine translation). an additional important note is that, between aligning texts and automatically extracting text from varied sources, it is possible for there to be mistakes or inconsistencies. this should be taken into consideration when using the corpus. we also welcome edits and contributions. beyond the above considerations, each of the choices that we made during data cleaning has the potential to have normative effects on the language. some may view norming and standardization as a benefit @xcite . however, it also risks the loss of language variety that is often valued by community members. consequently, we include our data cleaning scripts within the repository so that others may adapt them and transform the data into the version of sro or syllabics that meets their needs. ## corpus use the corpus is already being used to support community needs as part of a broader project for developing language learning technologies and technologies to support language use. within this context, corpus materials are being used to help people learn nêhiyawêwin. materials are also being used to develop language models that support tasks that community members who are learning nêhiyawêwin would like supported. we briefly discuss these ongoing activities to demonstrate the utility of the corpus. ## future work this corpus can be used to support several lines of future work. an immediate next direction would be further supporting the development of nêhiyawêwin learning materials using the corpus. for example, creating additional read-along activities and other game-based learning activities. soundhunters is one such game that aims to improve learner phonological awareness @xcite . the frequency statistics of different sounds, syllables, and words could be used to select learning materials for use in this and other games. the corpus could also be used to provide additional content. another avenue, would be applying the corpus to support the further creation of nlp technologies for nêhiyawêwin. as mentioned, predictive text models were created for nêhiyawêwin because this type of language technology is both desired and can be supported through the corpus. to determine if these models are helpful for nêhiyawêwin speakers when typing, we will perform user studies. from these studies, we aim to learn if the predictive models support text entry in a timely way and whether people perceive them to be useful. we will collect perceptual data and feedback from potential users after they have completed several text-entry tasks through the developed predictivetext system. we will use the same measures that are commonly employed to determine the performance of new text-entry techniques. these measures include response time, error rates, and key strokes per character @xcite . we will also analyze how often predictions are used and the ranking of the prediction selected. with this information, we can determine if the predictive text model meets a community's needs and preferences. it is simply not enough to rely on model performance metrics without obtaining feedback from potential users. we recognize that by preprocessing sro text, we have enabled easier use of this writing system for developing language technologies compared to syllabics. future work should create a similar pipeline for syllabics that aligns with language rules used by communities, so that it can receive the same status and attention in the development of language technologies. ## conclusion this work contributes a collection of nêhiyawêwin resources that have been cleaned, processed, and shared for creating language technologies. care was taken to collect, align, and preprocess the material so it could be used by others. it is hoped that sharing these resources along with the documentation of how they have been prepared will support language preservation and revitalization efforts. the utility of this corpus was shown via its community use in teaching nêhiyawêwin and by building language models to enable the creation of language technologies desired by speakers. this preliminary and on-going work demonstrates the value of the developed corpus for this low-resource language. through these efforts in developing the corpus we hope to pave the way for the future creation of language technologies for and by nêhiyawêwin speakers.