G _i 2 P _i Rule-based, index-preserving grapheme-to-phoneme transformations (2022)

Abstract: This paper describes the motivation and implementation details for a rule-based, index-preserving grapheme-to-phoneme engine ‘G _i 2P _i ' implemented in pure Python and released under the open source MIT license. The engine and interface have been designed to prioritize the developer experience of potential contributors without requiring a high level of programming knowledge. ‘G _i 2P _i ' already provides mappings for 30 (mostly Indigenous) languages, and the package is accompanied by a web-based interactive development environment, a RESTful API, and extensive documentation to encourage the addition of more mappings in the future. We also present three downstream applications of ‘G _i 2P _i ' and show results of a preliminary evaluation.

## introduction and motivation g i 2p i is a library 9 for grapheme-to-phoneme and orthographic transformation, with a particular focus on the needs of digital humanities projects. while libraries for general-purpose g2p exist, we found that our downstream projects had special needs that existing libraries did not entirely meet. in particular, 1. subject-matter experts for these languages are often teachers or linguists without a background in computer science, who are unfamiliar with the conventions of programming and need more intuitive interfaces to convert their knowledge into executable code ( §2.1). the need for such specialized knowledge presents a bottleneck in the development of g2p engines, particularly when we venture away from the nlp space and into the digital humanities space; experts of a particular language's sound patterns should not necessarily need experience in programming and compiling software in order to translate their knowledge of the language into a machine-readable format. meanwhile, however, the languages that we are concentrating on (in particular, indigenous languages spoken in canada) do not typically have extensive, publicly-available corpora of parallel orthographic and phonetic renderings, from which we could learn a weighted fst @xcite or neural model @xcite . for most of these languages, rule-based approaches based on expert knowledge will be the norm for the foreseeable future. fortunately, these are mostly languages with regular, linguistically-informed orthographies, such that rule-based approaches are adequate. in broad strokes, our library is most similar to epitran @xcite , which shares some of these design decisions; it prioritizes ease of installation and adopts a method for defining rule-based g2p mappings inspired by phonological re-write rule syntax that would be familiar to linguists. our work differs by allowing rules to be written in a spreadsheet format ( §2.1), by having a core engine that preserves the indices between inputs and output transductions ( §2.2), and by providing a bundled web interface for writing and running g2p mappings ( §2.4) with an accompanying restful api ( §2.6.1). ## g i 2p i this section briefly describes the process for writing rules ( §2.1), the motivation and implementation details for preserving indices between inputs and outputs ( §2.2), the automatic generation of cross-linguistic phoneme-to-phoneme mappings ( §2.3), the 'g2p studio' development environment ( §2.4), a list of currently supported languages ( §2.5), documentation information ( §2.6), and a description of various applications ( §2.7). ## evaluation as mentioned previously, this paper shares many similarities with epitran, however we cannot evaluate our system using the same method. epitran leverages baseline data available in some of the languages it supports to evaluate the system indirectly using the downstream task of developing asr systems. the word error rates (wer) of asr systems created using letter-to-sound rules from epitran are then compared against those created using the available baseline. the primary focus for this library is on extremely low resource languages, and we do not possess baseline data to recreate the evaluation procedure implemented by epitran. as a crude replacement, we evaluate two of our mappings by reporting the accuracy of a downstream forced alignment task using readalong studio ( §2.7.3). we manually annotated data from senćoŧen and kanyen'kéha with word-level alignments in praat. given the time-consuming nature of manual alignment, we were limited to a single document from each language; the senćoŧen document is 5:47 long and contains 417 words and the kanyen'kéha document is 5:07 long and contains 249 words. both documents are private materials owned by the language communities and shared with us by linguist timothy montler and kanyen'kéha educator owennatekha brian maracle respectively. we evaluate our hand-written mappings against a baseline zero-shot g2p method. the baseline we use is readalong studio's fallback method for 'und' (iso 639-3 for 'undetermined') text. this fallback method uses the text-unidecode foot_8 package to convert all characters to ascii equivalents, and then uses a rule-based mapping from ascii to ipa. for our hand-written senćoŧen and kanyen'kéha mappings, we use g i 2p i 's built-in automatic mapping functionality to map the ipa inventories to the closest english ipa equivalents ( §2.3). all methods are then mapped from ipa to the arpabet vocabulary used by the decoder. similar to @xcite , we evaluate the system by reporting the accuracy of the word boundaries predicted by the aligner within thresholds @xmath0, < 25, < 50, @xmath1, a result of 0.88 with a threshold @xmath2. as shown in ## conclusion in this paper we presented g i 2p i along with its motivations, and some descriptions of its use cases. the library is written in pure python to support (relatively) easy installation, with support for 30 different languages, index preservation between in19 kanyen'kéha contains fewer than half as many segmental phonemes as senćoŧen puts and outputs, an accompanying graphical web interface, a restful api, and extensive documentation to encourage the development of mappings for more languages in the future. we recognize that language experts are the best people suited to write mappings between a language's orthography and the ipa, and we hope that through a variety of features that such a contributor would "get for free" by contributing, that g i 2p i is an attractive option for rule-based g2p. to summarize, by contributing a mapping, a contributor will acquire the following: