Pragyaan: Designing and Curating High-Quality Cultural Post-Training Datasets for I ndian Languages (2025)

Abstract: The effectiveness of Large Language Models (LLMs) depends heavily on the availability of high-quality post-training data, particularly instruction-tuning and preference-based examples. Existing open-source datasets, however, often lack multilingual coverage, cultural grounding, and suffer from task diversity gaps that are especially pronounced for Indian languages. We introduce a human-in-the-loop pipeline that combines translations with synthetic expansion to produce reliable and diverse Indic post-training data. Using this pipeline, we curate two datasets: Pragyaan-IT (22.5K) and Pragyaan-Align (100K) across 10 Indian languages covering 13 broad and 56 sub-categories, leveraging 57 diverse datasets. Our dataset protocol incorporates several often-overlooked dimensions and emphasize task diversity, multi-turn dialogue, instruction fidelity, safety alignment, and preservation of cultural nuance, providing a foundation for more inclusive and effective multilingual LLMs.

## introduction recent developments around large language models (llms) @xcite @xcite have demonstrated that post-training data, comprising both instruction-tuning and preference data, plays a critical role in enhancing model alignment, task generalization, and usability @xcite @xcite . particularly in a multilingual and multicultural landscape, like india, the availability of high-quality, culturally grounded post-training data is crucial to address performance gaps in low-resource languages that often arise from the scarcity of relevant and representative training data @xcite . while several open-source datasets for posttraining @xcite @xcite ) exist, they are predominantly english-centric and often suffer from limita-tions such as inconsistent quality, restricted coverage, insufficient task complexity, and limited multilingual coverage. these challenges extend to indic post-training data as well, focus of our work. direct translations of existing english posttraining datasets are prone to translation biases, errors @xcite @xcite and loss of cultural grounding @xcite . for instance, a prompt like "tell me about a small herb to plant in backyard" might yield western herbs such as thyme or rosemary, whereas indian users would expect culturally familiar options like tulsi (holy basil), pudina (mint), or curry leaves. similarly, when asked "what is a good comfort meal for a rainy day?", english-centric answers such as tomato soup or grilled cheese overlook indian preferences like masala chai with pakoras or khichdi with ghee. even in wellness contexts, "recommend a workout routine for beginners" may default to squats and push-ups, neglecting practices like surya namaskar or yoga exercises. such mismatches highlight the need for post-training data that reflects not just language, but also local traditions and cultural context. the recent popularity of llm-based synthetic data generation @xcite for creating post-training datasets, while promising, still suffers in quality due to linguistic inaccuracies, grammatical inconsistencies, and reduced fluency, especially in multilingual settings, that could degrade the performance of models trained on them. moreover, the lack of fine-grained control over output complexity and the potential for hallucination can lead to the generation of low-quality, unreliable data. with the aim of addressing these gaps, we present an approach to curate high-quality posttraining datasets, especially in multilingual settings. our curation approach combines the above techniques with post-hoc manual editing, leading to a scalable human-in-the-loop pipeline, with spe-cific focus on several aspects of quality, like task coverage, multilingual representation, task complexity, culture, multi-turns, reasoning, and others. we leverage our approach to curate high-quality indic post-training datasets: pragyaan-it comprising 22.5k instruction tuning examples and pragyaan-align, a dataset of 100k preference examples in 10 indian languages covering 56 task categories. our contributions could thus be summarized as follows: ## related work post-training is a key step in aligning large language models (llms) with human intent, commonly achieved through instruction tuning @xcite and preference tuning @xcite datasets, constructed in several ways. task template based resources such as flan 2021 @xcite , flan 2022 @xcite , and p3 @xcite adapt nlp datasets into instruction-response format. human-authored datasets like open assistant @xcite , dolly @xcite , and lima @xcite demonstrate the value of curated instructions but face scalability challenges. to overcome this, synthetic generation approaches leverage llms to expand from small human-annotated seeds, with efforts such as self-instruct @xcite , alpaca @xcite , and guanaco (joseph cheung, 2023), often distilling knowledge @xcite from stronger teacher llm models. advanced pipelines like evol-instruct @xcite iteratively increase instruction complexity, while later works extend these methods to reasoning and code generation @xcite . complementing these, user-contributed datasets such as instructionwild @xcite and sharegpt foot_0 provide naturally occurring conversational data, and unnatural instructions @xcite show how seed tasks can be scaled into diverse synthetic corpora. subsequent work expanded into specialized domains, including dialogue systems @xcite , structured knowledge grounding @xcite , and chain-of-thought reasoning @xcite . more recently, the magpie dataset @xcite introduced a fine-grained taxonomy spanning creative writing, math, role-playing, planning, and data analysis, emphasizing the importance of broad coverage in post-training resources. preference datasets such as ultrafeedback @xcite and tulu3 @xcite comprises human and synthetic preference pairs for llm alignment. building on these advances, we construct our approach and dataset tailored to indian languages and cultural contexts leveraging manual annotations in complement with synthetic generation. while large-scale post-training datasets have become increasingly available, they remain predominantly english-centric, with limited coverage for other languages. a few exceptions incorporate some proportions of multilingual data @xcite @xcite @xcite , but they remain limited in cultural and linguistic diversity compared to english resources. prior efforts to extend post-training resources beyond english have typically followed three strategies: (1) translating english datasets into additional languages @xcite , (2) generating template-based datasets @xcite , and (3) manually curating instruction datasets in non-english languages @xcite . amongst these, template-based efforts such as xp3 @xcite extend the p3 taxonomy with 28 multilingual datasets. however, xp3 relies on uniform templates across languages, leading to limited task diversity and frequent repetition. translation-based figure 1 : workflow for building indian language post-training data: english prompts are either translated or expanded via modified self-instruct pipeline to generate synthetic prompts. in both cases, responses are then produced with an llm, translated into one of the 10 indian languages, and manually refined (section 3). ## methodology we present our multi-stage post-training dataset creation process that encompasses a variety of task categories at both broad and fine-grained levels, critical settings (complexity, interaction depth, constraints, safety, indian context, thinking trails) and leverages human annotators alongside curated data sources to ensure quality and coverage. while our approach itself is generic, we discuss how we used it to create high-quality indic post-training datasets that we collectively refer to as pragyaan-it and pragyaan-align. ## pragyaan: indic post-training datasets as part of this work, we construct high-quality post-training datasets that explicitly target 10 indian languages (bengali, gujarati, hindi, kannada, malayalam, marathi, oriya, punjabi, tamil, and telugu), yielding two complementary resources: i) pragyaan-it (22.5k): an instruction-tuning dataset designed to enhance a model's ability to follow diverse prompts across multiple domains, ensuring that models can generalize well to everyday user interactions. ## analysis we begin with an assessment of raw synthetic generations refined through human annotation, followed by a broader dataset-level analysis. ## conclusion this work addresses the scarcity of high-quality post-training data for multilingual llms by developing a human-in-the-loop pipeline to ensure diversity, quality and cultural grounding. through this approach, we construct two datasets: pragyaan-it and pragyaan-align covering 10 indian languages and multiple task categories. the datasets highlight inclusion of local cultural context, task diversity, multi-turn dialogue, and safety alignment, overcoming the limitations of naive translations and low-quality synthetic resources. although designed for indian languages, the pipeline is readily adaptable to other multilingual contexts. we present a comprehensive analysis of the dataset's characteristics, covering language distribution and domain coverage, and further demonstrate its effectiveness for alignment through a small-scale pilot study. future efforts will expand language coverage and further annotation quality refinement. we aim for this work to support broader efforts in building culturally inclusive resources that strengthen llm applicability in multilingual contexts.