L earning to P ronounce C hinese W ithout a P ronunciation D ictionary (2020)

Abstract: We demonstrate a program that learns to pronounce Chinese text in Mandarin, without a pronunciation dictionary. From non-parallel streams of Chinese characters and Chinese pinyin syllables, it establishes a many-to-many mapping between characters and pronunciations. Using unsupervised methods, the program effectively deciphers writing into speech. Its token-level character-to-syllable accuracy is 89%, which significantly exceeds the 22% accuracy of prior work.

## chinese writing the most-popular modern chinese writing system renders each spoken syllable token with a single character token (hanzi). there are over 400 syllable types in mandarin foot_0 and several thousand character types. the mapping is many-to-many: while most chinese words have two syllables, individual characters carry rough semantic meanings (eg, 中 = "middle", 重 = "weighty"). so it is no accident that the same character is used to write semantically-similar words: generally speaking, it is impossible for a student to correctly guess the pronunciation or meaning of a new character, though their guess may be better than chance. ## data preparation from a chinese wikipedia dump, foot_1 we remove all non-chinese characters, then convert to simplified characters. this forms our character corpus. for our pronunciation corpus, we could record and transcribe mandarin speech into pinyin syllables. instead, we simulate this. we take a large subset of the baidu baike encyclopedia, foot_2 but then immediately convert it to tone-marked pinyin syllables, by using a comprehensive dictionary foot_3 of 116,524 words and phrases. 99.97% of baike character tokens are covered by this dictionary. we substitute baike character sequences with pinyin sequences in left-to-right, longest-match fashion. this strategy works well most of the time. for example, it correctly pronounces 睡觉 as shui jiao, and 觉得 as jue de, despite the ambiguity of 觉. however, it incorrectly pronounces 想睡觉 because a dictionary entry 想睡 matches the phrase before 睡觉 = shui jiao can be applied; it also has trouble with single-character words like 还. using character sequences from chinese wikipedia and pinyin sequences from baidu baike is important. if we alternatively divided chinese wikipedia into two parts, unsupervised analysis could easily exploit high-frequency boilerplate expressions like 英 重定向：这是由 英 名 ，指向中文名 的重定向。它引 出英 至 遵循命名常 的合 名 ，能 助 者 作。 ("english redirection: this is a redirect from the english name to the chinese name. it guides the english title to a proper name that follows the naming convention and can assist the editor in writing.") we also pinyin-ize the first 100 lines (6059 characters) of our character corpus, as a gold-standard reference set, for later judging how well we phonetically decipher the character corpus. unless stated otherwise, all results are for token accuracy on this reference set. we also record internal structure of syllables and characters. for syllables, we separate onset and rime (eg, @xmath0. for characters, we employ the thorough graphical decompositions given in wikimedia commons, 6 which divide each character into (at most) two parts. this allows us to find, for example, 44 characters that include the second component 包 (咆, 孢, 狍, 炮, etc). we only use top-level decompositions in this work, forgoing any further recursive decompositions. ## supervised comparison points before turning to unsupervised methods, we briefly present two supervised comparison points. first, if we had a large parallel stream of character tokens and their pinyin pronunciations, we could train a simple pronouncer that memorizes the most-frequent pinyin for each character type. using the baike data as processed above as a putative parallel resource, we obtain 99.1% pronunciation accuracy on the test set. 7 the only errors involve ambiguous characters, showing that a deterministic character-to-pinyin mapping tablewhether obtained by memorization or by unsupervised methods-is sufficient to solve the bulk of the problem. second, to investigate whether written character components predict pronunciation, we use gold pronunciations of the most common 2000 characters to predict pronunciations of the next 1000. if a test character x has second (e.g., rightmost) written component y, then we use the pronunciation of y as a guess for the pronunciation of x. we find this works 25% of the time if we do not consider tones, and 17% of the time if we do. next we turn to unsupervised pronunciation, where we throw away pronunciation dictionaries and parallel data, working only from uncorrelated streams of characters and syllables. ## unsupervised vector method borrowing from unsupervised machine translation, which learns mappings between words in different languages @xcite , we attempt to learn a mapping between embeddings for characters and embeddings for pinyin symbols. we train fasttext @xcite vectors of dimension 300 and default settings on each of our corpora and use the muse system foot_4 to learn the relationship between the two vector spaces @xcite . there are two steps to this method: (1) map character vectors into pinyin space, (2) for each character type, find its nearest pinyin neighbor. this gives us a table that maps character types onto pinyin types. we apply this table to each character token of our 6059-character test set, obtaining token-level accuracy. unfortunately, this method does not work well. only 0.5% of type mappings are correct, and token-level accuracy is similarly small. reversing the mapping direction (pinyin embeddings into character embedding space) does not improve accuracy. it appears that the asymmetry of the mapping is difficult for the algorithm to capture. each pinyin syllable should, in reality, be the nearest neighbor of many different characters. moreover, the behavior of a pinyin syllable in running pinyin data may not be a good match for the behavior of any given character with that pronunciation. our next approach is to map words instead of characters. we break our long character sequence into a long word sequence, e.g., 竞争 很 激烈 by applying the jieba tokenizer foot_5 to wikipedia. we similarly break our long pinyin sequence into a long pinyin-word sequence, e.g., wo xihuan chi jiaozi, by applying the stanford tokenizer foot_6 to prepinyinized baidu. we build embeddings for types on both sides, and we again map them into a shared space. during the nearest-neighbor phase, we take each written-word and look for the closest pinyinword, giving preference to pinyin words with the source target tone? accuracy character pinyin no 0.20% character pinyin yes 0.05% pinyin character no 0.15% pinyin character yes 0.12% character word pinyin word yes 81.41% ## unsupervised noisy channel em method we next turn to a noisy-channel approach, following @xcite . we consider our character sequence @xmath1...c n as derived from a hidden (no tone) pinyin sequence @xmath2...p n : ## exploiting character components we next investigate whether character components can improve em results. instead of storing @xmath3 as em establishes a tentative high value for pr 1 (排 | pai), we hope to also create a high value for pr 4 (非 | pai), which will encourage pai to map to other characters with component 非 (such as 徘) in the following em iterations. unfortunately, while we do see compelling pr 4 entries, we do not see an overall improvement in test-set accuracy from this method. ## combining em and vector methods the em method gives 71% accuracy, while the vector method gives 81% accuracy. we find that the two methods agree 47% of time, and are 98.7% accurate in agreement cases, so in an unsupervised way, we distill out 261 high-confidence character/pinyin mappings. improved em results. we use the 261 highconfidence (c, p) mappings as our initial start point, by replacing each one's random initial p θ (c|p) value with a 1.0 weight. these weights bias the fractional counting in the first em iteration. improved vector-based results. we run the same initial procedure from section 5, giving us a vector space inhabited by both written words and pinyin words. however, we modify the nearestneighbor search that produces word/pronunciation mappings. our modified nearest-neighbor search takes a written word's vector and returns the nearest pronunciation vector that is consistent with the 261 high-confidence (c, p) mappings. for example, given the word 重 要, we prefer neighbors dang yao and zhong yao over dang pin, be-cause (要, yao) is one of the high-confidence mappings originally proposed by both em and vectorbased approaches. we also use high-confidence mappings to improve de sequences (section 5). this combination technique is also highly effective, raising accuracy from 81% to 89%. ## conclusion we implement and evaluate techniques to pronounce chinese text in mandarin, without the use of a pronunciation dictionary or parallel resource. the em method achieves a test-set accuracy of 71%, while the vector-based method achieves 81%. by combining the two methods, we obtain 89% accuracy, which significantly exceeds that of prior work. we also demonstrate that current methods for unsupervised matching of vector spaces are sensitive to the structure of the spaces. in the presence of one-to-many mappings between pinyin and characters, the mapping accuracy is severely downgraded, leaving open an opportunity to design more robust unsupervised vector mapping systems.