OVQA : A Dataset for Visual Question Answering and Multimodal Research in O dia Language (2025)

Abstract: This paper introduces OVQA, the first multimodal dataset designed for visual question-answering (VQA), visual question elicitation (VQE), and multimodal research for the low-resource Odia language. The dataset was created by manually translating 6,149 English question-answer pairs, each associated with 6,149 unique images from the Visual Genome dataset. This effort resulted in 27,809 English-Odia parallel sentences, ensuring a semantic match with the corresponding visual information. Several baseline experiments were conducted on the dataset, including visual question answering and visual question elicitation. The dataset is the first VQA dataset for the low-resource Odia language and will be released for multimodal research purposes and also help researchers extend for other low-resource languages.

## introduction visual question answering (vqa) is a complex task at the intersection of computer vision and natural language processing, requiring models to understand and reason about visual content and formulate accurate responses to textual questions. despite significant advances in this field, the majority of vqa research has been focused on a handful of widely spoken languages, primarily english. this language bias limits the accessibility and applicability of vqa technologies to non-english speaking populations. to address this gap, we introduce ovqa, the first multimodal dataset specifically designed for vqa tasks in the odia language. odia, an official language of india, is currently spoken by approximately 50 million people. @xcite however, it has been largely underrepresented in the realm of natural language processing and vqa research. @xcite https://www.britannica.com/topic/ oriya-language by developing a vqa dataset in odia, we aim to broaden the inclusivity of ai technologies and foster advancements in multilingual and multimodal ai systems. the ovqa dataset was built by translating 6,149 english question-answer pairs from the widely used visual genome dataset into odia. each question-answer pair is associated with a unique image, resulting in a robust dataset of 27,809 english-odia parallel sentences. this ensures a strong semantic alignment between the visual content and the textual data in both languages. our contributions are threefold: the development of ovqa is a significant step towards bridging the linguistic divide in ai research. by making this dataset publicly available, we hope to inspire further research in multilingual vqa and contribute to the creation of more inclusive ai systems. for our work, the visual genome dataset introduced by krishna et al. (2016), has been used. it is a large-scale collection of images and associated descriptive data designed to facilitate research in computer vision and natural language processing. we explored the paligemma (beyer et al., 2024) model which can be used for various tasks such as vqa, detecting objects on images, or even generating segmentation masks. here, we have explored the capability of paligemma for lowresource language on the vqa task. although paligemma has zero-shot capabilities -meaning the model can identify objects without finetuning, google strongly recommends fine-tuning the model for optimal performance in specific domains. ## related work parida et al. (2023a) created havqa, a multimodal dataset for visual question answering for the low-resource hausa language. the dataset demonstrates several use cases utilizing text and images including multimodal machine translation, visual question answering, and visual question elicitation. romero et al. (2024) proposed a culturally diverse multilingual visual question answering (cvqa) benchmark which includes culturally driven images and questions from across 28 countries on four continents, covering 26 languages with 11 scripts, providing a total of 9k questions. gupta et al. (2020) proposed a framework for multilingual and code-mixed vqa for hindi and english. ## focused language odia is an indo-aryan language predominantly spoken in odisha, a state located in eastern india. it is part of the indo-aryan language family, which evolved in the indian subcontinent through three distinct phases: old indo-aryan (1500 bc to 600 bc), middle indo-aryan (600 bc to 1000 ad), and modern indo-aryan (after 1000 ad). languages that emerged during the modern indo-aryan period include odia, bangla, assamese, hindi, urdu, punjabi, gujarati, sindhi, bhojpuri, marathi, sinhali, and maithili. odia is thought to have developed around 1000 ad, and it serves as the official language of odisha, recognized as one of the 22 languages in the indian constitution. according to the 2011 census, approximately 42 million people speak odia. the language features several dialects, with mughalbandi (standard odia) recognized as the standard dialect used in education. the script employed for writing odia is called the oriya/odia script. ## odia vqa dataset in this section, we delve into the various stages involved in the odiavqa dataset creation process, including collection, annotation, validation, and data analysis. ## availability the ovqa dataset can be accessed via lindat at: http://hdl.handle.net/11234/1-5820 . additionally, the ovqa dataset, designed for multimodal llm training in an instruction set format, is available on hugging face: dataset: https://huggingface.co/datasets/ odiagenmllm/odia_vqa_en_odi_set ## conclusion in this work, we presented ovqa: a multimodal dataset suitable for various nlp tasks for the odia language. some examples of these tasks include vqa, vqe, and other research tasks based on multimodal analysis. the ovqa dataset is available for research and non-commercial use under the creative commons attribution-noncommercial-sharealike 4.0 license. @xcite our planned future work includes: i) extending the dataset with more images depicting regional, and cultural aspects and qa pairs ii) providing ground truth for all images for image captioning experiments, and iii) organizing a shared task using the ovqa dataset.