Grapheme-Based Cross-Language Forced Alignment: Results with U ralic Languages (2021)

Abstract: Forced alignment is an effective process to speed up linguistic research. However, most forced aligners are language-dependent, and under-resourced languages rarely have enough resources to train an acoustic model for an aligner. We present a new Finnish grapheme-based forced aligner and demonstrate its performance by aligning multiple Uralic languages and English as an unrelated language. We show that even a simple non-expert created grapheme-to-phoneme mapping can result in useful word alignments.

## introduction matching speech signal and its orthographic transcription is a necessary first step for many research questions in linguistics @xcite @xcite . for wellresourced languages, manually aligned corpora exist, providing an easy starting point for linguistic research. for under-resourced languages such corpora are rare, and for all languages new corpora are continuously studied. in these situations, the researcher needs to complete this task before any actual research can begin. forced alignment, i.e., automatically matching text to speech using automatic speech recognition (asr), is widely used, and tools that can accomplish this automatically exist, such as fave @xcite , prosodylab-aligner @xcite , maus @xcite , and montreal forced aligner (mfa) @xcite . if the researcher is studying a language that is supported by an existing tool for forced alignment, learning to use it will be beneficial, since manual segmentation is much more arduous than transcription @xcite . however, the effort for this necessary, but often uninteresting step increases tremendously if no suitable model exits. the reason may be that the target data is out-ofdomain of what the acoustic model was trained with, or the target language is under-resourced and there is no model available at all. some aligning tools do not support retraining models. for others, such as fave and prosodylab, the model has been trained with a known asr framework, here htk @xcite , and the researcher could use the framework to train their own models. however, at this point it would be more straightforward to use the asr framework itself. in addition to all of this, the technical knowledge required to train an acoustic model with minimal or difficult data is formidable. mfa provides ample documentation, and has a user friendly wrapper over kaldi @xcite , a popular speech recognition framework. it gives users the option to retrain the model to fit their own data, and add new languages. @xcite used mfa to experiment on iterative forced alignment, and how it compared to the traditional linear method. even though they used a ready-made tool, the effort to try two alignment methods on an under-resourced language was enough to qualify as a research paper on its own right. for a linguist, who might not have technical expertise on asr, this may be intimidating as the first step. an alternative solution to the task of training new models is cross-language forced alignment, in which an aligner trained with a different language than the speech and transcriptions to be aligned, is used. in this paper we introduce a new wordlevel forced alignment tool based on kaldi. we show that this very simple command line tool can align closely related languages, is robust against speaker variability without any fine-tuning, and can even adequately align linguistically very dissimilar languages. this paper shows the first results for cross-language forced alignment involving finnish. in addition, using the tool we force-aligned a northern sámi corpus without proper word alignments with very little expert knowledge of the language. ## experiments we evaluate our kaldi-based aligner on related and unrelated languages, with a small amount of expert knowledge added to grapheme-to-phoneme mapping. we also experiment on speaker variation. this is the first time either has been done in clfa literature. the code and tool used in this paper are publicly available. foot_0 ## results the finnish alignment results in ## future work most of the papers in related research use some tool to automatically generate a phoneme-based lexicon for the target language. these lexicons do contain errors, so we have evaluated our results with word boundaries, since the words can be extracted as is from the transcription. however, automatic phoneme mapping would be an interesting next step, and allow better comparison with previous research effort in this multidisciplinary field. ## conclusion we have demonstrated promising results for crosslanguage forced alignment using finnish acoustic model for related and unrelated languages. we have shown that its results for finnish in languagedependent use are comparable to state-of-the-art aligners for english data. in addition, we present promising results with related and unrelated languages. we also showed the effects of speaker variation in cross-language situations, demonstrating that retraining speaker dependent models is generally not necessary. we share our tool as an easy to use docker image.