OTEANN : Estimating the Transparency of Orthographies with an Artificial Neural Network (2021)

Abstract: To transcribe spoken language to written medium, most alphabets enable an unambiguous sound-to-letter rule. However, some writing systems have distanced themselves from this simple concept and little work exists in Natural Language Processing (NLP) on measuring such distance. In this study, we use an Artificial Neural Network (ANN) model to evaluate the transparency between written words and their pronunciation, hence its name Orthographic Transparency Estimation with an ANN (OTEANN). Based on datasets derived from Wikimedia dictionaries, we trained and tested this model to score the percentage of false predictions in phoneme-to-grapheme and grapheme-to-phoneme translation tasks. The scores obtained on 17 orthographies were in line with the estimations of other studies. Interestingly, the model also provided insight into typical mistakes made by learners who only consider the phonemic rule in reading and writing.

## introduction an alphabet is a standard set of letters that represent the basic significant sounds of the spoken language it is used to write. when a spelling system (also referred as orthography) systematically uses a oneto-one correspondence between its sounds and its letters, the encoding of a sound (also referred as phoneme) into a letter (also referred as grapheme) leads to a single possibility; similarly the decoding of a letter into a sound leads to a single possibility as well. such orthography is thus transparent with regards to phonemes with the advantage of offering no ambiguity when writing or reading the letters of a word, as illustrated in figure 1 . in real life, no existing orthography is fully transparent phonemically. one reason is that a word spoken alone is sometimes different from a word spoken in a sentence. an even more consequen-tial reason is that some orthographies like english foot_0 and french foot_1 have incorporated deeper depth rules that have moved them away from a transparent orthography @xcite ; this has created ambiguities when trying to write or read phonemically, as illustrated in figure 2 . many studies have discussed the degree of transparency of orthographies @xcite . these studies are mainly motivated by the estimation of the ease of reading and writing when learning a new language @xcite . finnish, korean, serbo-croatian and turkish orthographies are often referred as highly transparent @xcite ) @xcite , @xcite , @xcite , whereas english and french orthographies are referred as opaque @xcite . however, little work exists in nlp about measuring the level of transparency of an orthography. one noticeable exception is the work of van den @xcite who have created grapheme-to-phoneme scores and tested them on three orthographies @xcite . this study extends such work with a method called oteann, which models a word-based phoneme-to-grapheme task and a word-based grapheme-to-phoneme task using an ann. for the sake of simplicity, the former task is called a writing task while the latter task is called a reading task. the goal is not to build a perfect spelling translator or a spell checker. instead the goal is to build a translator which can indicate a degree of phonemic transparency and thus make it possible to rank orthographies according to this criterion. interestingly, recent years have seen tremendous progress regarding nlp with anns @xcite . @xcite proposed an ann /t/ <@xmath0: example of unambiguous correspondence during writing and reading tasks in esperanto. /t/ <@xmath1... <@xmath2: example of ambiguous correspondence during writing and reading tasks in french. the /t/ phoneme can correspond to multiple graphemes, depending on the nature of the word and also depending on the nature of neighboring words in the sentence or even in a previous sentence. similarly, @xmath3. called a sequence-to-sequence (seq2seq) model that has proven to be very successful on language translation tasks. more recently, anns based on as attention @xcite , @xcite and transformers like bidirectional encoder representations from transformers (bert), @xcite and generative pre-training (gpt) @xcite have again enhanced and outperformed seq2seqs. considering writing a word and reading a word as two translations tasks allows re-using the transformers for our work. to this purpose, we used a minimalist gpt implementation (karpathy, 2020) called mingpt. notice that since we don't aim at building a perfect spelling translator, we do not have to translate a sequence of words into another sequence of words; our model only requires translating a spoken word into a spelled word (writing task) and a spelled word into a spoken word (reading task). in other words, our ann operates at the character level within a sequence of characters of single words. the pronunciation and spelling of the word are both encoded as a sequence of utf-8 characters; a pronounced word is encoded with the characters belonging to the set of phonemes of the target language, whereas a spelled word is encoded with the characters belonging to the alphabet of the target orthography. we directly re-used mingpt code with no modification. the only differences were the training data and the code for extracting the prediction at inference time. we used oteann to test seventeen orthographies in order to evaluate their degree of phonemic transparency. sixteen of them are the official orthographies of their respective language (arabic, breton, chinese, dutch, english, esperanto, finnish, french, german, italian, korean, portuguese, russian, serbo-croatian, spanish, and turkish) while the seventeenth is a phonemic orthography proposed for french. a unique multi-orthography ann model instance was trained to learn the writing and reading tasks on all languages at the same time. in other words, we used a single dataset containing samples of all studied orthographies. the multi-orthography ann model was then tested for each orthography and each task with new samples, which allowed calculating an average percentage of correct translations. a score of 0% of correct translations represented a fully opaque orthography (no correlation between the input and the target), whereas a score close to 100% represented a fully transparent orthography (full correlation between the input and the target). our study first confirms that orthographies like arabic, finnish, korean, serbo-croatian and turkish are highly transparent whereas other ones like chinese, french and english are highly opaque. for example, when solely based on a phoneme-grapheme correspondence, we estimated the chances of correctly writing a french word at 28%; similarly, when solely based on a grapheme-phoneme correspondence, we estimated the chances of correctly pronouncing an english word at 31%. for dutch, english and french reading tasks, our obtained ranking is in line with the one of van den @xcite . one unexpected finding is that oteann also allows discovering orthography task input output en write dz6b job en read job dz6b ## methodology in order to evaluate a level of transparency of some orthographies two main steps were necessary: obtaining datasets and carrying out the training and testing experiments with the ann. ## results first, regarding the results of the two baseline orthographies, the 'eno' opaque orthography obtained a score of 0% in both writing and reading, which was in line with the expectations given that there was no correlation between its phonemes and its graphemes; on the other hand, the 'ent' transparent orthography scored above 99.6% on the writing and reading tasks, which indicated a high level of correlation between its phonemes and its graphemes. we thus considered our ann model satisfactory for our objective of comparing the performance of different orthographies. figure 3 and ## discussion and conclusion among the tested orthographies, some shared the grapheme inventory. given that they are all trained together, there might be an impact on performance. although some of our preliminary experiments with a single ann instance per orthography did not seem to lead to significant differences, it could be interesting to formally compare both approaches. the accuracy metric we used is all or nothing. additional work could also study alternative accuracy metrics and compare their results on the different orthographies. although wiktionary data may be inconsistent in quality and therefore positively or negatively impact the measured metric, the results obtained for dutch, english and french orthographies reasonably extended those of @xcite while the other results reflected the perception of several other studies. consequently, our oteann model showed that an ann can convincingly estimate a level of phonemic transparency for multiple orthographies both for the phoneme-to-grapheme and grapheme-to-phoneme directions. this method should be easily applicable to other orthographies beyond those tested in this study. however, since the superfluous ipa symbols slightly influence the score results, future work should closely examine and discuss the phonemes to use depending on the orthography to be tested. as oteann also points out some possible grapheme or phoneme errors when writing or reading phonemically, it could also be used to detect possible errors in the dictionaries of transparent orthographies; it could also be used to evaluate proposals for improving opaque orthographies. finally, it would be beneficial to investigate if our ann and its artificial neural units somehow imitate the way a beginner learns to write and read a language. if so, it might suggest that a transparent orthography would be easier and faster to learn than an opaque orthography.