The Gemma Sutras: Fine-Tuning Gemma 3 for S anskrit Sandhi Splitting (2025)

Abstract: Sandhi, the phonological merging of morphemes, is a central feature of Sanskrit grammar. While Sandhi formation is well-defined by Pāṇini’s Aṣṭādhyāyī, the reverse task—Sandhi splitting—is substantially more complex due to inherent ambiguity and context-sensitive transformations. Accurate splitting is a critical precursor to tokenization in Sanskrit, which lacks explicit word boundaries and presents densely fused compounds. In this work, we present a data-driven approach, fine-tuning the Gemma-3 4B large language model on a dataset of over 49,000 training and 2,000 test examples of compound words and their morpheme-level decompositions. Leveraging the Unsloth framework with low-rank adaptation (LoRA) and 4-bit quantization, we train the model to predict these splits. Our work yields a scalable, Sandhi-aware system designed to enhance modern NLP pipelines for classical Sanskrit, demonstrating an effective application of LLMs to this linguistic challenge.

## introduction sanskrit, an ancient language with a vast literary corpus @xcite and a grammar codified by pāṇini @xcite that is a cornerstone of linguistics @xcite , features a key morphological process called sandhi (सं ध). this rule-governed merging of adjacent morphemes @xcite , illustrated in figure 1 , creates long, uninterrupted compound words. while sandhi formation is deterministic, the reverse process of splitting, or viccheda ( व े द), is significantly more complex due to inherent ambiguity @xcite . this complexity makes effective tokenization, a foundational nlp step, extremely challenging. naïve tokenizers fail on compounds like तदुपासनीयम् (which must be split to तत् + उपासनीयम् ) @xcite , and even modern subword algorithms like bpe @xcite or word-piece @xcite struggle because the transformations disrupt statistical regularities @xcite . to address this, we frame sandhi splitting as a data-driven, linguistically-informed pretokenization task. we fine-tune a large language model on an annotated dataset to accurately segment these compounds, with our overall approach depicted in figure 2 . ## related works automated sanskrit sandhi splitting has progressed through several computational paradigms, as surveyed by @xcite and more recently for deep learning techniques by s et al.. early approaches were rule-based, grounded in pāṇini's grammar @xcite and exemplified by tools like the jnu splitter @xcite and in-ria reader @xcite . these systems, however, often exhibit low perfor-mance on benchmarks like the sandhikosh corpus @xcite due to the inherent ambiguity of sandhi. a conceptual shift came with deep learning, which framed the task as a sequence-tosequence problem @xcite .models like the double decoder rnn (dd-rnn) learned transformations directly from character data using a two-stage process (locate split, then reconstruct), a paradigm also explored by @xcite . this two-stage neural approach was later refined by @xcite , whose model first identified a localized "sandhi window" before decoding, improving efficiency on a large dataset from the uoh corpus @xcite . building on these foundations, this work shifts to modern large language models (llms). while their application to this specific task is underexplored, we propose that fine-tuning an llm offers a more generalizable and simpler approach than specialized architectures. we leverage instruction tuning @xcite @xcite and parameter-efficient methods @xcite to adapt a model for this nuanced linguistic challenge. ## methodology to address sandhi splitting, we adopt a supervised fine-tuning approach using the gemma-3 4b instruction-tuned large language model (gemma @xcite . the goal is to adapt the model's generative capabilities to split compound sanskrit words into their morphemic components. our overall pipeline is summarized in figure 2 . we selected the gemma-3 4b variant as its instruction-tuned nature aligns well with our prompt-response task format @xcite , and its 4-billion parameter size offers a practical balance between performance and resource efficiency. the model's transformer architecture @xcite incorporates features like rotary position embeddings (rope) @xcite , and its sentencepiece tokenizer @xcite supports the devanagari script. ## conclusion in this paper, we have presented a data-driven approach for sanskrit sandhi splitting by finetuning the gemma-3 4b large language model. our method leverages parameter-efficient techniques (lora) and an instruction-based learning paradigm, achieving a competitive exact match accuracy of 87.7% on a curated dataset of over 50,000 examples. this result demonstrates that general-purpose pre-trained llms can be effectively adapted to handle complex, rule-governed morpho-phonological phenomena in sanskrit without requiring specialized architectures or full model fine-tuning. our methodology, which combines instruction following with lora, offers a scalable and resource-efficient path for tackling similar tasks. the findings affirm the potential of llms as powerful and adaptable tools for computational linguistics, especially for morphologically rich and low-resource languages. future work will focus on refining the instruction-tuning process, exploring more diverse and larger datasets, and investigating methods to integrate lexical or grammatical knowledge to further enhance performance and address the identified error categories.