Deciphering Undersegmented Ancient Scripts Using Phonetic Prior (2021)

Abstract: Most undeciphered lost languages exhibit two characteristics that pose significant decipherment challenges: (1) the scripts are not fully segmented into words; (2) the closest known language is not determined. We propose a decipherment model that handles both of these challenges by building on rich linguistic constraints reflecting consistent patterns in historical sound change. We capture the natural phonological geometry by learning character embeddings based on the International Phonetic Alphabet (IPA). The resulting generative framework jointly models word segmentation and cognate alignment, informed by phonological constraints. We evaluate the model on both deciphered languages (Gothic, Ugaritic) and an undeciphered one (Iberian). The experiments show that incorporating phonetic geometry leads to clear and consistent gains. Additionally, we propose a measure for language closeness which correctly identifies related languages for Gothic and Ugaritic. For Iberian, the method does not show strong evidence supporting Basque as a related language, concurring with the favored position by the current scholarship.1

## introduction all the known cases of lost language decipherment have been accomplished by human experts, oftentimes over decades of painstaking efforts. at least a dozen languages are still undeciphered today. for some of those languages, even the most fundamental questions pertaining to their origins and connections to known languages are shrouded in mystery, igniting fierce scientific debate among humanities scholars. can nlp methods be helpful in bringing some clarity to these questions? recent work has already demonstrated that algorithms can successfully decipher lost languages like ugaritic and linear b @xcite , relying only on non-parallel data in known languages-hebrew and ancient greek, respectively. however, these methods are based on assumptions that are not applicable to many undeciphered scripts. the first assumption relates to the knowledge of language family of the lost language. this information enables us to identify the closest living language, which anchors the decipherment process. moreover, the models assume significant proximity between the two languages so that a significant portion of their vocabulary is matched. the second assumption presumes that word boundaries are provided that uniquely define the vocabulary of the lost language. one of the famous counterexamples to both of these assumptions is iberian. the iberian scripts are undersegmented with inconsistent use of word dividers. at the same time, there is no definitive consensus on its close known language-over the years, greek, latin, and basque were all considered as possibilities. in this paper, we introduce a decipherment approach that relaxes the above assumptions. the model is provided with undersegmented inscriptions in the lost language and the vocabulary in a known language. no assumptions are made about the proximity between the lost and the known languages and the goal is to match spans in the lost texts with known tokens. as a byproduct of this model, we propose a measure of language closeness that drives the selection of the best target language from the wealth of world languages. given the vast space of possible mappings and the scarcity of guiding signal in the input data, decipherment algorithms are commonly informed by linguistic constraints. these constraints reflect consistent patterns in language change and linguistic borrowings. examples of previously utilized constraints include skewness of vocabulary mapping, and monotonicity of character-level alignment within cognates. we further expand the linguistic foundations of decipherment models, and incorporate phonological regularities of sound change into the matching procedure. for instance, a velar consonant @xcite is unlikely to change into a labial @xcite . another important constraint in this class pertains to sound preservation, that is, the size of phonological inventories is largely preserved during language evolution. our approach is designed to encapsulate these constraints while addressing the segmentation issue. we devise a generative framework that jointly models word segmentation and cognate alignment. to capture the natural phonological geometry, we incorporate phonological features into character representations using the international phonetic alphabet (ipa). we introduce a regularization term to explicitly discourage the reduction of the phonological system and employ an edit distance-based formulation to model the monotonic alignment between cognates. the model is trained in an end-to-end fashion to optimize both the quality and the coverage of the matched tokens in the lost texts. the ultimate goal of this work is to evaluate the model on an undeciphered language, specifically iberian. given how little is known about the language, it is impossible to directly assess prediction accuracy. therefore, we adopt two complementary evaluation strategies to analyze model performance. first, we apply the model to deciphered ancient languages, ugaritic and gothic, which share some common challenges with iberian. second, we consider evaluation scenarios that capitalize on a few known facts about iberian, such as personal names, and report the model's accuracy against these ground truths. the results demonstrate that our model can robustly handle unsegmented or undersegmented scripts. in the iberian personal name experiment, our model achieves a top 10 accuracy score of 75.0%. across all the evaluation scenarios, incorporating phonological geometry leads to clear and consistent gains. for instance, the model informed by ipa obtains 12.8% increase in gothic-old norse experiments. we also demonstrate that the proposed unsupervised measure of language closeness is consistent with historical linguistics findings on known languages. decoding cipher texts man-made ciphers have been the focal point for most of the early work on decipherment. they usually use em algorithms, which are tailored towards these specific types of ciphers, most prominently substitution ciphers @xcite . later work by @xcite , and @xcite addresses the problem using a heuristic search procedure, guided by a pretrained language model. to the best of our knowledge, these methods developed for tackling man-made ciphers have so far not been successfully applied to archaeological data. one contributing factor could be the inherent complexity in the evolution of natural languages. deciphering ancient scripts our research is most closely aligned with computational decipherment of ancient scripts. prior work has already featured several successful instances of ancient language decipherment previously done by human experts @xcite @xcite . our work incorporates many linguistic insights about the structure of valid alignments introduced in prior work, such as monotonicity. we further expand the linguistic foundation by incorporating phonetic regularities that have been beneficial in early, pre-neural decipherment work @xcite . however, our model is designed to handle challenging cases not addressed by prior work, where segmentation of the ancient scripts is unknown. moreover, we are interested in dead languages without a known relative and introduce an unsupervised measure of language closeness that enables us to select an informative known language for decipherment. ## model we design a model for the automatic extraction of cognates foot_1 directly from unsegmented or undersegmented texts (detailed setting in section 3.1). in order to properly handle the uncertainties caused by the issue of segmentation, we devise a generative framework that composes the lost texts using smaller units-from characters to tokens, and from tokens to inscriptions. the model is trained in an end-to-end fashion to optimize both the quality and the coverage of the matched tokens. to help the model navigate the complex search space, we consider the following linguistic properties of sound change, including phonology and phonetics in our model design: to reason about phonetic proximity, we need to find character representation that explicitly reflects its phonetic properties. one such representation is provided by the ipa, where each character is represented by a vector of phonological features. as an example, consider ipa representation for two phonetically close characters @xcite (see figure 3 ), which share two identical coordinates. to further refine this representation, the model learns to embed these features into a new space, optimized for the decipherment task. ## experimental setup our ultimate goal is to evaluate the decipherment capacity for unsegmented lost languages, without information about a known counterpart. iberian fits both of these criteria. however, our ability to evaluate decipherment of iberian is limited because a full ground truth is not known. therefore, we supplement our evaluation on iberian with more complete evaluation on lost languages with known translation, such as gothic and ugaritic. ## conclusions we propose a decipherment model to extract cognates from undersegmented texts, without assuming proximity between lost and known languages. linguistics properties are incorporated into the model design, such as phonetic plausibility of sound change and preservation of sounds. our results on gothic, ugaritic, and iberian shows that our model can effectively handle undersegmented texts even when source and target languages are not related. additionally, we introduce a method for identifying close languages that correctly finds related languages for gothic and ugaritic. for iberian, the method does not show strong evidence supporting basque as a related language, concurring with the favored position by current scholarship. potential applications of our method are not limited to decipherment. the phonetic values of lost characters can be inferred by mapping them to the known cognates. these values can serve as the starting point for lost sound reconstruction and more investigation is needed to establish their efficacy. moreover, the effectiveness of incorporating phonological feature embeddings provides a path for future improvement for cognate detection in computational historical linguistics @xcite . currently our method operates on a pair of languages. to simultaneously process multiple languages as it is common in the cognate detection task, more work is needed to modify our current model and its inference procedure.