Multilingual State Space Models for Structured Question Answering in I ndic Languages (2025)

Abstract: The diversity and complexity of Indic languages present unique challenges for natural language processing (NLP) tasks, particularly in the domain of question answering (QA).To address these challenges, this paper explores the application of State Space Models (SSMs) to build efficient and contextually aware QA systems tailored for Indic languages. SSMs are particularly suited for this task due to their ability to model long-term and short-term dependencies in sequential data, making them well-equipped to handle the rich morphology, complex syntax, and contextual intricacies characteristic of Indian languages. We evaluated multiple SSM architectures across diverse datasets representing various Indic languages and conducted a comparative analysis of their performance. Our results demonstrate that these models effectively capture linguistic subtleties, leading to significant improvements in question interpretation, context alignment, and answer generation. This work represents the first application of SSMs to question answering tasks in Indic languages, establishing a foundational benchmark for future research in this domain. Furthermore, we propose enhancements to existing SSM frameworks, optimizing their applicability to low-resource settings and multilingual scenarios prevalent in Indic languages.

## introduction in recent years, the field of natural language processing (nlp) has witnessed significant advancements with the development of models capable of handling complex linguistic structures and long-range dependencies. state space models (ssms) @xcite have emerged as a promising alternative to traditional architectures like transformers @xcite , particularly in tasks requiring efficient long-sequence modeling. models such as mamba @xcite and its variants have demonstrated the ability to manage long-context language modeling with constant memory usage, addressing some limitations inherent in transformer-based models @xcite . de-* equal contribution. spite these advancements, the development of questionanswering (qa) systems for low-resource languages, particularly indic languages, remains a challenging endeavor. india has a population of 1.4 billion and is home to 122 languages and 270 mother tongues. indian languages fall in the minority as far as nlp models are concerned. one of the indian languages-hindi-is spoken by 577.8 million people worldwide @xcite . likewise, marathi, another indian language, is ranked 11th on the list of most spoken languages. however, the scarcity of annotated datasets and neural models has hindered progress in building effective qa systems for these languages. recent efforts have been made to address this gap by introducing large-scale qa datasets for indic languages, such as the indic qa dataset @xcite , which provides a substantial resource for developing systems tailored to the linguistic nuances of indic languages. this paper explores the various ssm models to build efficient and contextually aware qa systems for indic languages. ssms are particularly suited for this task due to their ability to model both long-term and short-term dependencies in sequential data, making them well-equipped to handle the rich morphology, complex syntax, and contextual intricacies characteristic of indian languages. the main contributions of this paper are: the field of qa and nlp has seen rapid advancements, particularly with the introduction of models leveraging state-space representations and multi-modal learning. this section reviews key advancements related to ssms, indic language processing, and multi-modal approaches to contextual understanding. ## methodology our methodology focuses on developing a questionanswering framework tailored for indic languages by leveraging the capabilities of ssms. specifically, we experimented with various models, including mamba, mamba-2,falcon mamba, jamba, zamba, samba, and hymba, to address the linguistic diversity and complexity of indic languages. each model was evaluated for its ability to process questions, answers, and context efficiently while preserving the grammatical and semantic nuances of the respective languages. the overall framework for this system is illustrated in fig. 1 . ## experiments in our experimentation, we fine-tuned several models-falcon mamba, mamba, mamba-2, jamba, zamba, samba, and hymba-on a hindi and marathi squadstyle question-answering dataset specifically designed for these languages. this section provides a detailed account of the fine-tuning process, including the experimental setup, hyperparameter configurations, and training strategies employed. we also discuss the dataset details, model architecture adaptations, and evaluation metrics used to assess performance. ## conclusion this paper explores the development of efficient question-answering systems for indic languages. by applying ssms, this research demonstrates advancements in capturing the linguistic nuances of hindi and marathi. among the models evaluated, mamba-2 consistently delivered superior performance, showcasing its ability to handle both short-term and long-term dependencies effectively. fine-tuning proved critical in improving model accuracy, highlighting the importance of adapting pre-trained architectures to meet the specific needs of low-resource languages. the integration of robust preprocessing techniques, such as tokenization tailored for indic scripts, further contributed to the models' success. these findings lay a strong foundation for scalable, multilingual qa systems and underscore the potential of ssms in addressing language-specific challenges. future work will focus on expanding dataset diversity by including additional dialects of indic languages and developing a unified model for commonly used dialects.