Wav2pos: Exploring syntactic analysis from audio for H ighland P uebla N ahuatl (2024)

Abstract: We describe an approach to part-of-speech tagging from audio with very little human-annotated data, for Highland Puebla Nahuatl, a low-resource language of Mexico. While automatic morphosyntactic analysis is typically trained on annotated textual data, large amounts of text is rarely available for low-resource, marginalized, and/or minority languages, and morphosyntactically-annotated data is even harder to come by. Much of the data from these languages may exist in the form of recordings, often only partially-transcribed or analyzed by field linguists working on language documentation projects. Given this relatively low-availability of text in the low-resource language scenario, we explore end-to-end automated morphosyntactic analysis directly from audio. The experiments described in this paper focus on one piece of morphosyntax, part-of-speech tagging, and builds on existing work in a high-resource setting. We use weak supervision to increase training volume, and explore a few techniques for generating word-level predictions from the acoustic features. Our experiments show promising results, despite less than 400 sentences of audio-aligned, manually-labeled text.

## introduction automatic morphosyntactic processing, such as morphological analysis or syntactic parsing, is an important task in natural language processing (nlp) for the purposes of language documentation, feature-extraction for downstream nlp tasks @xcite @xcite , and for quantitative corpus-based linguistic analysis @xcite . the ample research exploring these tasks has, overwhelmingly, focused on textual data. however, text for low-resource, endangered, marginalized, and/or minority languages, which constitute a majority of the world's languages, is often sparsely available, if at all. instead, much of the data from these languages may exist in the form of recordings, potentially only partially-transcribed or analyzed by field linguists working on language documentation projects. at the same time, recent progress in speech processing has resulted in powerful, pretrained speech representation models such as wav2vec2.0 @xcite , which make it possible to achieve impressive asr systems via fine-tuning on relatively little data @xcite . these same representations have also been shown to be useful in audio classification problems, such as speaker recognition @xcite and emotion detection @xcite . in the remainder of this paper, we explore endto-end automated part-of-speech (pos) tagging directly from audio for an endangered nahuatl variant, using a modest amount of transcribed data and only a few hundred sentences of annotated text. in light of a very limited data set. highland puebla nahuatl, (or sierra puebla nahuatl, also referred to by inali as náhuatl del noreste central, iso-639-3 azz, henceforth hpn) is a nahuatl variant group spoken by about 70,000 people (ethnologue's 2007 estimate) in the northeastern sierra region of the state of puebla, mexico (see figure 1 ) in 24 municipalities @xcite . hpn has been the subject of documentary and descriptive linguistic efforts @xcite @xcite ; key and richie de @xcite . furthermore it is one of two nahuatl variants with a free and open morphosyntactically-annotated corpus, in the form of a universal dependencies treebank. as an indigenous language of mexico, hpn is considered at risk of being lost @xcite . ## related work while most effort in the area of morphosyntactic analysis has focused on textual input, some recent work explores the idea of performing natural language processing directly from audio. @xcite perform end-to-end dependency parsing for french from audio, by extracting audio features, aggregating them into audio word embeddings using lstms, and performing dependency parsing using these embeddings. these experiments used a dataset size consisting of 169,500 training sentences. @xcite describe a non-autoregressive (non-transformer) method for performing end-to-end asr and downstream nlp tasks such as named-entity recognition, performing part-of-speech tagging on a large. foot_0 corpus of spoken japanese, and ner on a corpus of english containing approximately 10k training sentences @xcite . @xcite create a speech translation corpus of hpn using the same dataset as in the present paper, leveraging the fact that the entire dataset has transcriptions and translations (dataset details presented in section 4). the labeled hpn part-of-speech data comes from recently-released highland puebla nahuatl ud treebank, which consists of (1) a small subset of the openslr corpus annotated for morphosyntax, (2) a subset of texts in the azz variant from the multi-variant parallel corpus axolotl @xcite , and (3) technical publications by the sociedad mexicana de física. 4 only (1), which contains 399 sentences and 3,463 tokens, has corresponding audio, and is held out for system evaluation. ( 2 ) and ( 3 ), totaling 838 sentences and 6,671 tokens, are used in training a simple text-based part-of-speech tagger, with which we generate synthetic data as described in section 5.1. the remaining openslr corpus data (i.e. all of the audio/transcriptions that has not been annotated) is partitioned into a larger dataset for finetuning wav2vec2 (about 40k sentences), and a smaller dataset for training the audio-based pos tagger (about 7k sentences). we chose to use the majority of the data for wav2vec2 fine-tuning in order to ensure a high-performing asr model since without that a pos tagger would have no words to align its pos tags with. 5 we chose not to use overlapping data for training both wav2vec2 and the wav2pos tagger to avoid overfitting and poor performance on unseen data. ## results the results of our three experiments are reported in there is a significant difference in performance between the two wav2pos models, the the wb model much worse on all metrics. while without more detailed analysis we can only speculate, it appears as though the aggregation step, which involves passing a sequence of acoustic vectors through an lstm to produce a single "audio word vector," may introduce too many additional parameters for the model to learn given the relatively small amount of training data. while these results are certainly interesting, they raise more questions than they answer, primarily as the result of the constrained set of experiments we performed. for future work, we plan to replicate these experiments, but using the same, larger wav2vec2 training dataset to train the wav2pos models. we would also like to explore the hyperparameter space of these models in more depth, and try using a stronger text-based tagger, such as a multilingual pretrained transformer-based model, to create the silver data. finally, given the promising results for pos tagging, we are interested in expanding these efforts to other aspects of syntactic analysis such as dependency parsing. ## concluding remarks we have presented a preliminary investigation of automated morphosyntactic analysis from audio with no human-labeled training data. we leveraged a large set of transcribed audio to fine-tune a wav2vec2 acoustic feature-extraction model, and experimented with producing pos-tags directly from the acoustic embeddings. we created our training data by tagging unlabeled transcription data using a simple classifier model. the results showed that one of our audio-based pos-tagging models performed slightly better than using the text-based tagger to tag the transcriptions.