Exploring Similarity Measures and Intertextuality in V edic S anskrit Literature (2024)

Abstract: This paper examines semantic similarity and intertextuality in selected texts from the Vedic Sanskrit corpus, specifically the Maitrāyaṇī Saṃhitā (MS) and Kāṭhaka-Saṃhitā (KS). Three computational methods are employed: Word2Vec for word embeddings, stylo package for stylometric analysis, and TRACER for text reuse detection. By comparing various sections of the texts at different granularities, patterns of similarity and structural alignment are uncovered, providing insights into textual relationships and chronology. Word embeddings capture semantic similarities, while stylometric analysis reveals clusters and components that differentiate the texts. TRACER identifies parallel passages, indicating probable instances of text reuse. The computational analysis corroborates previous philological studies, suggesting a shared period of composition between MS.1.9 and MS.1.7. This research highlights the potential of computational methods in studying ancient Sanskrit literature, complementing traditional approaches. The agreement among the methods strengthens the validity of the findings, and the visualizations offer a nuanced understanding of textual connections. The study demonstrates that smaller chunk sizes are more effective for detecting intertextual parallels, showcasing the power of these techniques in unraveling the complexities of ancient texts.

## introduction vedic sanskrit literature preserves invaluable cultural and historical information from ancient india. however, their study presents unique challenges due to linguistic characteristics, modes of composition, and transmission. computational methods offer promising avenues to analyze such texts on an unprecedented scale. in this paper, we explore similarity measures and intertextuality between selected texts from the vedic corpus -the maitrāyan . ī sam . hitā (ms) and kāt . haka sam . hitā (ks). these texts belong to different śākhās or schools, and are considered to exhibit mutual influence in their composition around 900-700 bce. 1 the main focus of this paper is to present reliable numerical data on the chapter-wise similarity between the ms and ks. while it is known that the ms and ks are parallel texts, the variations in similarity among chapters have not yet been confirmed through numerical data. since differences in chapter-wise similarity can contribute to estimating the relative chronology of each chapter, this similarity analysis holds significant importance for understanding the process of textual composition. in recent years, the editorial process has been increasingly elucidated through philological studies @xcite @xcite , suggesting variations in similarity between different sections in ms and ks depending on the time period. that is to say, sections edited in earlier periods exhibit lower similarity between ms and ks, whereas those edited in later times show higher similarity between ms and ks. similarity analysis using computational methods further advances this study. our analysis employs three approaches: ## related work computational methods have been increasingly applied to study various aspects of sanskrit literature in recent years. @xcite developed a neural network architecture for processing sanskrit texts. @xcite analyzed poetic style in sanskrit poetry using deep learning techniques. regarding vedic sanskrit specifically, @xcite developed a dependency parser for r ˚gvedic sanskrit. @xcite compiled a vedic treebank. these works provide nlp tools and resources for computational processing of vedic texts. stylometry has been widely used to study authorship and stylistic similarity in classical literature. @xcite applied stylometric analysis using the stylo r package @xcite to investigate the authenticity of an unknown classical latin text called the expositio. their study concluded that this work was probably written by the second-century african author apuleius of madauros. while stylometry focuses on authorship attribution and stylistic analysis on a macro-level, as demonstrated by the stylo package, text reuse detection tools offer a micro-level approach to detecting each text reuse such as quotations and allusions among texts. specifically, tracer @xcite @xcite is a text reuse detection tool that has been successfully applied to study intertextuality in ancient greek @xcite , latin @xcite , coptic texts @xcite @xcite , classical tibetan @xcite , german @xcite , etc. other programs are also available for historical text reuse analysis. for example, tesserae and passim are well-known tools in this field. tesserae @xcite is primarily used for latin texts, while passim @xcite has been adapted for western languages and arabic with promising results but has not yet been adapted for sanskrit. compared to these tools, tracer offers greater flexibility and customizability, making it possible to adapt it to vedic sanskrit using custom lemmatization, synonym, and cohyponym files. the corpus ms.1.1 includes ritual formulas for new and full moon sacrifice. ms.1.6 includes ritual explanation about establishment of sacred fires, whose parallel is ks.8. ms.1.7 includes ritual explanation about reestablishment of sacred fires, whose parallel is ks.9.1. ms.1.9 includes explanation of secret spells related to ritualistic communal life, whose parallel is ks.9.11. ms.1.6, 1.7, and 1.9 have been philologically studied by @xcite , and their parallels in ks have been accurately identified. ms and ks were composed in the same editorial policy, and have almost the same contents for the same rituals, but with some variants in details. ms and ks contain portions with different linguistic styles and content (sometimes irregularly inserted or arranged), which necessitates the exclusion of such portions to conduct linguistically and semantically accurate analyses. the corpora used in this analysis were created to ensure that different styles (formulas or explanations) and contents (rituals) are not mixed. the chapter numbers within the parentheses following each section name represent the exact chapter numbers included in the section. the size (word count) of each corpus is also provided above. using these corpora, we conduct comparisons (similarity analyses) between sections as follows: the first two comparisons, namely ms.1.1 ↔ ms.1.6 and ms.1.6 ↔ ms.1.7, serve as an evaluation of the proposed methods, as their similar or dissimilar relations are philologically demonstrated. ms.1.1 differs significantly in content from ms.1.6 and ms.1.7, while the latter two share similar contents. accordingly, if the proposed methods work well, the comparison of ms.1.1 ↔ ms.1.6 is expected to show a low similarity, whereas ms.1.6 ↔ ms.1.7 is expected to demonstrate a high similarity, compared to the former comparison. the following three comparisons are between the texts of ms and ks. each chapter is thought to have been edited in different periods and under different cultural influence, and therefore, the degree of similarity between ms and ks varies. the two comparisons, ms.1.6 ↔ ks.8 and ms.1.7 ↔ ks.9.1, were manually calculated @xcite . as a result, the comparison ms.1.6 ↔ ks.8 showed a low similarity, while the comparison ms.1.7 ↔ ks.9.1 showed a very high similarity. since it is philologically inferred that ms.1.6 is older than ms.1.7, the paper presented the perspective that chapters compiled in the earlier period have lower similarity with ks, whereas those from a later period have higher similarity with ks, indicating possible intertextual borrowing. from this, in our current analysis, the comparison of ms.1.6 ↔ ks.8 is anticipated to reveal a low similarity, while ms.1.7 ↔ ks.9.1 is expected to exhibit a high similarity. in contrast, the last comparison, ms.1.9 ↔ ks.9.11, was not examined in the previous studies, and serves as the main focus of our current analysis, aiming to demonstrate to what extent this comparison shows similarity. if the comparison of ms.1.9 ↔ ks.9.11 reveals a high similarity, akin to ms.1.7 ↔ ks.9.1, it strongly suggests that the intertextual contact between ms.1.9 and ks.9.11 occurred during a later period, characterized by a tendency for ms and ks to exhibit similarities, as argued in @xcite . the texts are procured from the digital corpus of sanskrit. 2 original sanskrit text undergoes phonetic fusion and changes at word boundaries, known as sandhi. these fusions and changes make it challenging to segment the text into individual words and perform morphological analysis. therefore, as a first step in processing the text, it is necessary to resolve the sandhi to create an "un-sandhied" text, which can then be used for lemmatization. the texts stored in the digital corpus of sanskrit are processed into un-sandhi-ed texts as well as lemmatized texts by the computational method of @xcite , verified through expert review and correction. for the purpose of comparing similarity, the lemmatized texts are used, which are manually divided into distinct chunks or paragraphs with attention to meaningful coherence: ## conclusion this paper presented an analysis of semantic similarity and text reuse in selected vedic sanskrit texts using word embedding, stylometric method and figure 9: tracer: ms.1.6 ↔ ks.8 (20 lemmas) tracer. the results from these approaches indicate patterns of similarity and clustering between different portions of the texts, which can be justified by previous literary studies. by analyzing the similarity of ms.1.9 ↔ ks.9.11, we inferred that ms.1.9 might share a similar historical period with ms.1.7. in conclusion, the computational analysis provides insights into the relationships between the texts and their sections in vedic literature, which can clarify the process of its composition. the general agreement between the word embedding, text reuse detection and stylometric approach enhances the validity of the findings, and various visualizations of various analyses complemented each other's weaknesses and contributed to a more accurate understanding. moreover, this study demonstrates that smaller chunk sizes are beneficial for finding parallels. on the other hand, documents with larger chunk sizes encompass various common topics. this means that documents with larger chunk sizes proportionally contain fewer topics that semantically distinguish them from each other, making it difficult to identify parallel relations with larger chunk sizes. therefore, a smaller chunk size is more suitable for our purpose of finding parallels or verifying parallel relations between texts. this research demonstrates the potential of computational methods in vedic sanskrit studies, and other ancient language corpora. future work can extend the analysis to more texts, explore other embedding models and stylometric techniques, and closely examine the nature of the parallels identified. we hope this encourages further collaborative