A Simple Audio and Text Collection-Annotation Tool Targeted to B razilian Indigenous Language Native Speakers (2025)

Abstract: In this paper we present an audio and text annotation tool for native speakers, with a particular focus on Brazilian indigenous languages. Our tool simplifies the process of language resource annotation and employs gamefication techniques typically found in language learning games. Then we describe the annotation tool and present preliminary results for the Bororo language. We discuss the limitations of our tool, highlighting ethical and practical implementation concerns.

## introduction audio and text annotation tools are key for documenting and building resources for endangered languages @xcite . existing tools are mostly designed for linguistic professionals and focus on formal description of language resources, such as dependency treebanks and lexical databases. while such tools are fundamental for properly documenting languages, only linguist experts can operate them, and they remain often unknwon outside academia. hence, despite the pressing need for annotated corpora, language annotation tools remain costly and dependent on scarcely available experts. annotation tools, in their current form, can hardly scale to address the 2,680 languages at risk of extinction by the end of this century @xcite . furthermore, ethical and practical concerns arise when we consider that experts who operate language annotation tools are often not members of the indigenous communities themselves @xcite . it is hard to ensure that data annotation procedures are compliant with ethical guidelines @xcite , such as the los pinos declaration foot_0 , or even that annotations are validated by actual indigenous speakers. we argue that next-generation tools should be designed for use by lay indigenous speakers to accelerate the data collection and annotation process. while there are few linguist experts, indigenous communities are large. in particular, brazil is home to a significant number of languages. for example, the xavante language population alone represents more than 27,000 people. these languages are collectively referred to here as brazilian indigenous languages (bils). in spite of the high number of languages spoken in brazil (estimated around 180, see @xcite ), this number is declining fast as populations age and many languages are not learned by younger generations. in this work, we propose and implement an initial language annotation tool that can be used directly by native speakers in indigenous communities without expert linguistic knowledge. our proposal simplifies the annotation process so as to only collect words in audio and written text format. our tool allows indigenous speakers to annotate words with their own speech, perform translations and associate morphemes to word tokens. the main goal is to achieve a source dataset of paired instances, which doe not require further work to develop dependency treebanks, natural language processing tools, and other resources. we employ a gamification-based design @xcite to maximize engagement among native speakers, encouraging them to produce a high volume of annotations in the shortest possible time. recognizing the limited availability of indigenous community members, we prioritize a highly userfriendly interface to ensure accessibility and ease of use. we guide speakers/users through the annotation process by specifying the target word and direct-ing their input to an internal speech recognition component, which transcribes the audio into written text. this transcription includes preliminary annotations, such as morphological information and translations. speakers can then review, refine, and confirm the prefilled text and annotations before proceeding to the next word. to enhance usability and minimize friction, we integrate automated annotation components, such as speech recognition. we also address challenges associated with limited computational resources. in our prototype, we employ lightweight models and heuristics that can run offline in a standard web browser or mobile app. finally, we present preliminary results for the bororo language as a proof of concept. the paper is organized as follows. section 2 describes our annotation tool design and its development, including data sources and methods. section 3 presents preliminary results for the bororo language. section 4 discusses the challenges and limitations of our prototype and offers concluding remarks. ## methodology our data collection and annotation tool aims to empower native speaker communities to collect and annotate language resources by themselves without requiring expert linguistic knowledge. our tool takes the form of a game, similar to formats often found in language learning game apps from both industry (e.g. duolingo) and the literature @xcite @xcite . the tool follows a linear progression structure, where the user advances by completing units. in order to do so, the user is asked to annotate a series of specific words, similar to language exercises. in the annotation screen, depicted in figure 1 (top), the user is asked to provide speech audio translation in native language for a given portuguese word. in the figure, the tool asks for a speech translation of the portuguese word for jaguar, "onc ¸a pintada", to the bororo language. first, the user records their speech in native language. the user should say the given word only once within 10 seconds. after the audio recording finishes, we run a speech recognition model to generate a transcript. in our bororo language example, we have a bororo-portuguese dictionary available @xcite , thus, we know in advance that the target word, or the bororo translation for "onc ¸a ointada", is "adugo". however, there are many alternative ortographies or even regional synonyms that can be absent in our knowledge base. in order to avoid enforcing a specific ortography by presenting the target word beforehand, we allow the user to freely annotate so as to preserve linguistic diversity. next, we check whether the produced transcript matches the target word from the dictionary entry. if a match is found (figure 1a ), we retrieve an image representing the word concept, the written word in native language and its description from the dictionary entry. finally, the user can make editions if necessary (such as providing an alternative orthography), confirm changes and move on to the next. on the other hand, if we cannot assert that the transcript matches the target word (figure 1b ), the user is required to fill up the written translation and description manually before moving to the next. the tool may fail to properly identify a match by several reasons; for example, the speech recognition may fail, the dictionary may be incomplete or may not contain all synonyms or simply the user translation may be incorrect. we allow the user to retry recording the speech translation multiple times, so if the speech recognition fails due to background noises, computer glitches or any other intermitent issues, it can succeed in a second attempt. if the matching keeps failing even after multiple retries, users can always fill the written translation and description manually. we provide autocomplete options based on lexical similarity to speed up the manual filling process. additionally, we also provide an option for the user to skip the current annotation and move to the next. for example, if the user does not know the translation for the given word, we want to save time and avoid incorrect annotations by giving them the option to immediately move on to the next. the whole annotation process is depicted in figure 2 . now we focus on the speech recognition model and on the word matching heuristic. we propose to reuse speech recognition models that were trained for other languages to be used for lowresource languages. in our proof of concept for the bororo language, we employed the web speech api's speech recognition model for brazilian portuguese (pt-br), which can run offline and is available in most web browsers (e.g. chrome, edge, safari, except firefox). back to our example, note that "adugo" is a romanized word. most writing systems for brazilian indigenous languages were romanized with strong portuguese language influence, bororo language included. we observed that the speech recognition model often produced transcripts of portuguese words that are phonetically similar the original bororo word. for example, the transcript for the word "adugo" results in "adubo", which is a portuguese word with completely different meaning but phonetically similar to the bororo word. since bororo writing system is romanized, we could perform a lexical similarity search between the portuguese transcript and the known bororo vocabulary to find good match candidates. additionally, since we know the target word, we can consider a match if the target word has high lexical similarity to the transcript. in our prototype, we built a similarity score based on levenshtein distance and applied an arbitrary 0.9 threshold as the heuristic criteria to tell whether the speech to text process matches or not the target word. we define our similarity score as: ## results we still need to evaluate our proposal more broadly with the bororo indigenous community to measure community adoption and engagement. this will require a more comprehensive evaluation of our processes and methods to measure, for example, how effective the speech recognition model is in speeding up the annotation process. at this point, we ran simulated experiments to get preliminary results on: (1) the speech to text recall, (2) how much time the speech to text saves in the annotation process, i.e. the speed up. first, to measure recall, we sampled 50 words from the bororo dictionary, generated correct speech audio for them and ran a simulation to evaluate how many instances our speech to text process was able to find a match, the fraction of matches over the total number of instances is what we refer to as recall. we obtained 0.56 recall, 28 matches out of 50 words, as presented in ## concluding remarks & limitations the annotation tool introduced in this work represents a significant step forward in the advancement of resources for brazilian indigenous languages. our proposed design allows native speakers, who do not necessarily require specific linguistic knowledge, to perform annotations in audio and text resources. our design avoids biases towards specific ortographies by allowing the user to freely annotate their speech and written forms. at the same time, we incorporate speech to text and autocomplete components to speed up the annotation process. despite the promising benefits, our prototype falls short in multiple aspects that we now examine. first, our tool currently only supports word annotation. we consider it to be a natural step to evolve our methods to enable sentence annotation. before we can support sentences, we must require word annotation to be fully functional, which implies better autocomplete and speech recognition capabilities. additionally, users annotations can vary significantly and we still do not have a proper process to create consensus around them. the orthography currently used by the bororo people was developed by catholic missionaries and is not well-suited to their language (see @xcite . recent publications have adopted a different orthography, which occasionally leads to minor discrepancies. for example, we have two ortographies for the word "rain" in bororo, which are "bubutu" (old) and "bybyty" (new). if our tool presents "bubutu" to users, they may be confused as our tool is incentivizing an outdated ortography. once the bororo corpus @xcite is completed, this issue is expected to be resolved, as all sources will be unified under a standardized orthography. one significant issue stems from the fact that bororo territories are not contiguous, resulting in variations in pronunciation among different regions. these differences can sometimes lead to mockery of speakers from areas where the language is less commonly spoken, as if their way of speaking were "incorrect." this poses an important ethical concern, as it may cause speakers to feel that a new orthography privileges certain pronunciations over others. this concern becomes even more relevant when we consider that our tool employs automatic speech recognition models, which may incentivize specific accents. given that the speech recognition models were trained in foreign languages, biases towards pronunciation similar to the portuguese language may occur. there is still room for improvement in our speech to text process. we considered applying more sophisticated approaches, such as accoustic models @xcite , for zero shot speech recognition in indigenous languages, but models like those require stable internet connectivity as they are too large to run in offline devices. we are currently limited to work with models that can run in the web browser or mobile app so they can be actually used in the field. future work should conduct evaluate varied speech to text methods and improve their performance. at this point, we have only implemented a proof of concept for the bororo language; thus, it is still necessary to assess how well the methods introduced in this work generalize to other languages. endangered language revitalization requires the development of annotated resources @xcite . we believe that our proposal can be extended to annotate languages beyond brazilian ones. similar strategies around phonetical similarities have already been employed in other contexts @xcite . future work should evaluate the effectiveness of our annotation tool in partnership with native speakers and assert its value. we hope our preliminary research can help scaling up data annotation for endangered languages and produce rich data sources to support revitalization initiatives.