Deciphering Oracle Bone Language with Diffusion Models (2024)

Abstract: Originating from China’s Shang Dynasty approximately 3,000 years ago, the Oracle Bone Script (OBS) is a cornerstone in the annals of linguistic history, predating many established writing systems. Despite the discovery of thousands of inscriptions, a vast expanse of OBS remains undeciphered, casting a veil of mystery over this ancient language. The emergence of modern AI technologies presents a novel frontier for OBS decipherment, challenging traditional NLP methods that rely heavily on large textual corpora, a luxury not afforded by historical languages. This paper introduces a novel approach by adopting image generation techniques, specifically through the development of Oracle Bone Script Decipher (OBSD). Utilizing a conditional diffusion-based strategy, OBSD generates vital clues for decipherment, charting a new course for AI-assisted analysis of ancient languages. To validate its efficacy, extensive experiments were conducted on an oracle bone script dataset, with quantitative results demonstrating the effectiveness of OBSD.

## introduction oracle bone script (obs) represents an ancient language inscribed on turtle shells and animal bones, extensively utilized during china's shang dynasty, a feudal dynasty dating back 3,000 years. the script not only chronicled the human geography and daily activities of that period but also encapsulates invaluable historical significance, offering a unique window into the linguistic and cultural practices of early chinese civilization. however, despite the discovery of tens of thousands of fragments of oracle bones, a significant portion of the characters remain undeciphered @xcite , leaving the rest shrouded in mystery. to date, more than 4,500 oracle bone script (obs) characters have been discovered, but only about 1,600 of these have been deciphered and linked to their modern chinese counterparts. in modern chinese, unicode includes more than 90,000 chinese characters, though only approximately 3,500 characters are commonly used in contemporary chinese society. this challenge of understanding the remaining undeciphered obs characters and linking them to modern chinese has attracted significant research interest, with attempts being made to leverage modern ai technologies for the understanding of such an ancient language @xcite @xcite . however, the majority of existing methodologies primarily focus on the recognition and understanding of already deciphered obs @xcite @xcite , with the utilization of ai to assist in the decipherment of unknown inscriptions remaining an underexplored area. this is partly because, unlike modern languages that can be digitized and stored as text due to established encoding systems, obs lacks a standard input method or encoding scheme, resulting in its preservation predominantly in the form of images rather than digital text usually used in nlp methods. additionally, since obs was inscribed on turtle shells and animal bones, many of which have been damaged or fragmented upon discovery, there is essentially no complete corpus available. this absence of a comprehensive corpus severely limits the applicability of language models that require extensive datasets for training, such as bert @xcite , roberta @xcite , and gpt @xcite . to address the challenges inherent in the decipherment of obs using conventional nlp methodologies, this paper introduces a novel approach by employing image-based generative techniques for auxiliary decipherment of obs. specifically, we train a conditional diffusion model that utilizes un-seen categories of obs as a conditional input to generate corresponding images of its modern counterpart. this direct provision of modern representations or potential decipherment clues leverages the model's learned evolution from ancient scripts to contemporary fonts, circumventing the corpus construction and other challenges that traditional nlp methods face with ancient languages. notably, while our experiments focus on obs, this training paradigm holds the potential for extension to other ancient languages, such as cuneiform and hieroglyphics. in summary, this paper makes three key contributions: ## related works applying machine learning to the study of ancient languages represents a notable shift in linguistics and epigraphy. this area, distinct from the nlp tasks typically associated with modern languages, involves digitization, linguistic analysis, textual criticism, translation, and decipherment @xcite @xcite . for a comprehensive overview of this field, we direct interested readers to the survey by sommerschield et al. @xcite @xcite @xcite . due to space constraints, our review is limited to literature most pertinent to oracle bone language decipherment. the oracle bone language is considered a form of hieroglyphic that uses pictorial symbols to represent specific meanings. it originated around 1500 bc and has evolved over thousands of years into modern chinese characters. the evolution timeline can be summarized into seven periods as follows: oracle bone script (1500 bc), bronze inscriptions (1300 bc -221 bc), seal script (1100 bc -221 bc), spring & autumn characters (770 bc -476 bc), warring states characters (475 bc -221 bc), clerical script (221 bc -220 ad) and regular script (around 3rd century ad). the continuous evolutionary path makes obs a unique presence among ancient scripts. many of its character forms have been preserved in modern standard chinese characters. while these are significant overlaps in the forms and meanings of characters between adjacent periods, greater differences can be found between more distant periods. some characters disappeared and later reappeared across different periods, highlighting the dynamic nature of this ancient writing system. while the majority of work related to obs has focused on employing cv or nlp techniques to recognize @xcite @xcite or understand @xcite match known characters but also to uncover new forms that could elucidate the meanings of these ancient scripts. formally, the training set denoted as @xmath0, c i ) | s i is an obs instance and c @xmath1, pairs obs instances with their modern chinese counterparts from a set of known categories c. the model is designed to extend beyond the training set s, identifying modern equivalents for obs instances s ′ , and proposing new character forms where existing matches are absent. to achieve this, our approach utilizes a diffusionbased @xcite model, for transforming obs character images x into their modern chinese equivalents, as illustrated in figure 1 . the model operates in two phases: the forward process, the forward phase introduces noise to the modern chinese character images x 0 , transitioning them towards a state resembling pure noise via a controlled markov chain process, ultimately conforming to a gaussian distribution n (0, i). this is mathematically articulated as follows: the denoising phase employs a u-net architecture @xcite for the model f θ , trained to predict the noise ϵ and restore the image. the training objective minimizes the loss function: ## conclusion in this work, we presented obsd, an innovative approach leveraging conditional image generation for the decipherment of obs. our novel local structure sampling technique addresses the inherent challenges in learning modern chinese characters' structures from limited samples, enabling effective structural correspondence learning between obs and modern chinese characters. furthermore, the integration of a zero-shot refinement module significantly enhances the decipherment accuracy, a claim substantiated by promising results on the hust-obs dataset and evobc dataset. the potential of obsd extends beyond obs, offering prospects for deciphering other ancient scripts, such as hieroglyphs and maya glyphs. looking ahead, we aim to collaborate with epigraphy experts to further validate and refine the obsd, aspiring to advance ai's role in the decipherment of ancient languages.