H iero LM : E gyptian Hieroglyph Recovery with Next Word Prediction Language Model (2025)

Abstract: Egyptian hieroglyphs are found on numerous ancient Egyptian artifacts, but it is common that they are blurry or even missing due to erosion. Existing efforts to restore blurry hieroglyphs adopt computer vision techniques such as CNNs and model hieroglyph recovery as an image classification task, which suffers from two major limitations: (i) They cannot handle severely damaged or completely missing hieroglyphs. (ii) They make predictions based on a single hieroglyph without considering contextual and grammatical information. This paper proposes a novel approach to model hieroglyph recovery as a next word prediction task and use language models to address it. We compare the performance of different SOTA language models and choose LSTM as the architecture of our HieroLM due to the strong local affinity of semantics in Egyptian hieroglyph texts. Experiments show that HieroLM achieves over 44% accuracy and maintains notable performance on multi-shot predictions and scarce data, which makes it a pragmatic tool to assist scholars in inferring missing hieroglyphs. It can also complement CV-based models to significantly reduce perplexity in recognizing blurry hieroglyphs. Ourcode is available at https://github.com/Rick-Cai/HieroLM/.

## introduction egyptian hieroglyphs is the formal written language and an important medium for religious and funerary practices in ancient egypt. the process of decoding hieroglyphs involves first converting them into transliterations and then translating the transliterations into modern languages @xcite . due to natural erosion, it is common that the hieroglyphs on the surface of the unearthed artifacts are blurry or even missing. efforts have been as an example, the blurry hieroglyph a in the blue box in figure 1 would confuse a cv model, because it could be either 𓇑 (nḫb) or 𓇓 (sw) based on its vague shape, but from the surrounding words we know that this sentence describes an offering by the king to the god osiris, so it is likely that this blurry sign is 𓇓 (sw), which means "the king". moreover, for the red box in figure 1 , the signs are almost entirely missing, and the cv models will become useless, but from the words before it, we know that it should be a title of osiris, which indicates that the missing word is probably 𓊽 𓅱 (ḏdw), because 𓎠 𓊽 𓅱 (nb ḏdw; "lord of djedu") is a common title for osiris in the offering formula. in light of these limitations, we propose a novel approach where we model hieroglyph recovery as a next word prediction problem, which can be addressed effectively with language models. to select the best architecture for our task, we consider the following characteristics of egyptian hieroglyphs (allen, 2000): (i) it is a dead language whose corpora have ceased to grow, and thus the amount of data available for training is very limited. hence, our model must be comfortable with small-scale training data. (ii) in ancient egypt, hieroglyphs are mostly used in limited scenarios including funerals, religious rituals, and monumental inscriptions. the restrictive formats on the hieroglyphic sentences leads to a better hope of accurate language modeling with simpler architectures. (iii) due to its limited scope of usage, the hieroglyphic sentence structure has strong local affinity (e.g., a large portion of a sentence could be titles following names of gods or kings), suggesting that our model should have strong capability in capturing short-range dependencies. based on these characteristics, we build our hierolm with lstm @xcite . to validate our design choice, we compare the performance of hierolm with popular architectures such as rnn (medsker and jain, 1999) and transformers @xcite in section 4.3. our contributions are summarized as follows: ## methodology in this section, we describe in detail our hierolm model, which adopts the lstm architecture as illustrated in figure 2 . assume that the input sentence has t words. let x (t) ∈ {0, 1} |v | be the one-hot encoding of the t-th word (1 ≤ @xmath0. then, its embedding e (t) ∈ r s , where s is the embedding size, is computed as e (t) = ex (t) , where e is an embedding layer. the hidden state h (t) ∈ r d , where d is the hidden dimension size, at step t is computed as: the cell state c (t) ∈ r d at step t is computed as: the predicted output is calculated by: ## conclusion in this paper, we exclusively propose to model egyptian hieroglyph recovery as a next word prediction task addressed by language models. considering the data scale and the local semantic affinity, we propose hierolm with lstm architecture, which achieves significant accuracy in experiments. its notable performance on multi-shot predictions and short input contexts makes it practical in archaeological research to infer missing hieroglyphs and complement cv models. in the future, we plan to explore potential ways of integrating computer vision models and language models into a unified and effective hieroglyph recovery system.