On Pronunciations in W iktionary: Extraction and Experiments on Multilingual Syllabification and Stress Prediction (2021)

Abstract: We constructed parsers for five non-English editions of Wiktionary, which combined with pronunciations from the English edition, comprises over 5.3 million IPA pronunciations, the largest pronunciation lexicon of its kind. This dataset is a unique comparable corpus of IPA pronunciations annotated from multiple sources. We analyze the dataset, noting the presence of machine-generated pronunciations. We develop a novel visualization method to quantify syllabification. We experiment on the new combined task of multilingual IPA syllabification and stress prediction, finding that training a massively multilingual neural sequence-to-sequence model with copy attention can improve performance on both high- and low-resource languages, and multi-task training on stress prediction helps with syllabification.

## introduction wiktionary 1 is a free online multilingual dictionary containing a plethora of interesting information. in this paper, we focus on the pronunciation annotations in wiktionary, which are relatively understudied. for any given word, wiktionary may include data for ipa (both phonetic and phonemic), hyphenation, dialectical variation, and even audio files of speakers pronouncing the words. these types of data have been shown to be useful for tasks such as grapheme-to-phoneme transduction, e.g. in recent sigmorphon shared tasks @xcite . there are many existing parsing efforts that have extracted pronunciation information from wiktionary. recent extractions of data from wiktionary focus on obtaining high-quality pronunciations from a single edition of wiktionary, usually the english edition (e.g. wu and yarowsky, 1 www.wiktionary.org 2020a; @xcite . however, substantial increases in data can be obtained by parsing other editions of wiktionary, which have been shown to be helpful for downstream tasks. for example, @xcite extract pronunciations from the english, french, german, and spanish editions, and ? extract pronunciations from the english, german, greek, japanese, korean, and russian editions. in this paper, we build upon yawipa @xcite , a recent wiktionary parsing framework. targeting the larger wiktionaries for increased coverage and those not dealt with in previous work, we construct new pronunciation parsers for the french, spanish, malagasy, italian, and greek editions of wiktionary. combined with pronunciations from the english wiktionary, this totals to over 5.3 million words, which to our knowledge is the largest pronunciation lexicon to date and also a unique comparable corpora of pronunciations. in section 2, we show that our extracted pronunciations are a substantial increase in data, covering numerous pronunciations not in the english wiktionary. this is especially beneficial for low-resource languages. in section 3, we analyze this data and find that a small portion of these pronunciations may be low-quality and computer-generated. in section 4, we present a novel visualization technique for analyzing the use of stress in ipa pronunciations. in section 5, we experiment on the combined task of massively multilingual syllabification and stress detection. our neural sequence-to-sequence model with copy attention outperforms a sequence labeling baseline, especially in very low-resource scenarios, underscoring the contributions of additional languages to the task. in addition we find that a multi-task approach of predicting both stress and syllabification can improve the performance on syllabification alone. ## wiktionary pronunciation extraction as a multilingual resource, wiktionary exists as a set of numerous editions. that is, the english wiktionary is written in english by and for english speakers, while the french wiktionary is written in french by and for french speakers. any edition can contain entries for words in any language. for example, figure 1 shows a screenshot of the english wiktionary's pronunciation information for the french word chien. we use the @xmath0. ## analysis of the dataset for high-resource languages, the home language edition (e.g. english edition for the english language) usually supplies the most pronunciations, but this is not always the case (e.g. the french wiktionary provides more italian pronunciations than the italian edition). in terms of amount of data, two languages are outliers: malagasy (mg, an austronesian language spoken in madagascar) and volapük (vo, a constructed language). as relatively less spoken languages, these languages have a disproportionately large amount of data. why is this so? the data for these two languages come from the malagasy edition, which we parsed because of its high ranking in the list of wiktionaries. 4 both malagasy and volapük are inflected languages foot_2 whose ipa pronunciations seem to be entirely computer-generated using a regular transduction process from orthography to ipa, which was exploited to create a large set of pronunciations for these two languages. we also find that some latin pronunciations may be machine-generated. for example, the malagasy edition supplies /kontabulawit/ as the pronunciation for the latin contabulavit and [d "ẽ:onstrat] for demonstrat. these pronunciations lack stress and syllable markings, and in the case of demonstrat, do not agree with established pronunciations of latin. thus leading us to believe that these were machine-generated pronunciations. in contrast, the english edition contains both well-formed classical and ecclesiastical latin pronunciations with stress and syllable markers, but only for the dictionary forms contabulō /kon"ta.bu.lo:/ and dēmonstrō /de:"mon.stro:/. we must emphasize that we are not condemning the use of machine-generated pronunciations. for many languages, e.g. spanish and latin, the spelling of a word reflects its pronunciation, so generated pronunciations are likely to be accurate. indeed, the existence of pronunciation templates such as {{fr-ipa}}, mentioned above, are wellresearched additions to wiktionary that alleviate the need for humans to manually input ipa pronunciations, thus reducing the potential for human error. we fully support the use of these templates (though they make our parsing job harder), and we would love to see them standardized across all wiktionary editions, so that editions such as the malagasy edition can benefit from contributions to the english edition (or any other edition, for that matter). we do caution researchers that the data contained in crowd-sourced resources such as wiktionary may not be thoroughly vetted for accuracy, as we have discovered. fortunately, the openness of these crowdsourced data allows for community members to quickly intervene when problematic data is found. one especially poignant example in recent news is the scots wikipedia, a large portion of which was recently revealed to be written by an american teenager who is not a scots speaker. 6 essentially, this teenager translated english articles into "scots" by systematically rewriting english words to sound as if they were spoken with a scottish accent, in the same vein as some of the latin "ipa" pronunciations in the malagasy wiktionary. ## visualizing syllabification ipa has the ability to mark syllable boundaries (.) as well as primary (") and secondary () stress. words in some languages, e.g. malay, do not have stress, and sometimes stress can be double marked ("") for extra stress. we first quantify ipa stress and syllabification in our extracted dataset then present multilingual experiments on predicting syllabification and stress. we develop a visualization technique to understand the distribution of words in each language that contain syllable boundaries (figure 3 ). these bubble charts plot the number of characters in a word (x-axis), the percentage of words containing syllable markers (y-axis), and the number of words ideally, one should see that the longer the word, the higher the percentage of words that have syllables marked. french is a perfect example of this: once words reach 9-10 characters in length, they all contain syllable markers. by examining these plots, we can easily identify examples of problematic ipa syllabification in malagasy (mg) and latin (la) words. for malagasy words, syllable boundaries simply do not exist. for latin words, we see an unusual negative-slope curve, where words around 4-6 characters in length are more likely to have syllables marked, but longer words are less likely to have syllable boundaries marked. this analysis actually is consistent with our earlier finding in section 2: because latin is a highly inflected language, the dictionary forms contain high-quality ipa, but the overwhelming number of pronunciations are actually machine-generated for inflected forms, which may not have the syllables marked. english is a middle ground in terms of quality. while we see the expected upward slope as the length of the word increases, the percentage of words with syllable markers never approaches 100%. a manual review of several english pronunciations indicates that annotators simply did not include syllable boundaries for many english words. further analyses could shed light on the rea-sons for the negligence of the annotators, or other phenomena that might explain the lack of syllable markers. ## experiments in this section, we present experiments on multilingual syllable and stress prediction. in the linguistics literature, many studies have shown that awareness of syllable boundaries can improve word recognition performance in children (e.g. @xcite @xcite . speech syllabification is also a common step in a speech recognition pipeline. syllabification of text is not a new task, and has been explored via a variety of methods, including rulebased and grammar-based approaches (e.g. @xcite and data-driven approaches (e.g. @xcite @xcite . however, previous work has focused primarily on a handful of languages, and some focus on orthographic syllabification rather than phonemic segmentation. some use celex @xcite , a popular dataset containing syllabified text, but it only contains syllabified words in english, german, and dutch. in contrast, our extracted pronunciation lexicon is a unique multilingual resource that allows for developing and evaluating models and approaches on the new combined task of massively multilingual ipa syllabification and stress prediction across hundreds of languages. in this task, given unmarked ipa, a model must insert syllable markers or stress markers at the appropriate locations. data for our task, we filter our pronunciation dataset to keep only ipa containing syllable boundaries or stress markers, 7 so that we have ground truth for our model. this resulted in 93,206 ipa pronunciations across 174 languages, which we split into a 80-10-10 train-dev-test stratified split (same proportion of languages in each set). ## conclusion we extracted the largest dataset of ipa pronunciations to date, by combining ipa from the french, spanish, malagasy, italian, and greek editions of wiktionary along with existing pronunciations from the english edition, totaling to 5.3 million pronunciations. we developed a visualization method for examining syllabification in large datasets, which can give indications about the quality of ipa pronunciations. we experiment on the new combined task of massively multilingual prediction of syllabification and stress using a variety of models and approaches, showing success with a multitask multilingual sequence-to-sequence model. we hope our dataset and analysis methods will be useful for researchers in a variety of disciplines. we envision our newly extracted pronunciation dataset to be especially useful for researchers interested in lexicography and spoken language technologies. in terms of lexicography, this dataset is a unique comparable corpus containing annotations from several editions of wiktionary, each representing a distinct population of speakers. in several cases, the same pronunciation is supplied by multiple editions, and some editions use phonetic rather than phonemic ipa. future work can address questions such as: when and why might different editions disagree on a pronunciation? why do some words have pronunciations and others don't? in addition, we would like to investigate the use of our pronunciation dataset in language learning of core vocabulary of low-resource languages @xcite and modeling etymology relationships between words @xcite .