Title: IndicGenBench: A Multilingual Benchmark to Evaluate Generation Capabilities of LLMs on Indic Languages
Benchmark: IndicGenBench
Summary: This benchmark introduces INDICGENBENCH; a benchmark for evaluating LLMs
on user-facing generation tasks across a diverse set 29 of Indic languages covering 13
scripts and 4 language families. INDICGENBENCH is composed of diverse generation
tasks like cross-lingual summarization, machine translation, and cross-lingual question
answering. INDICGENBENCH extends existing benchmarks to many Indic languages through
human curation providing multi-way parallel evaluation data for many under-represented Indic languages.
Phenomenon type: General Capability (A broadly useful ability, which could be relevant to multiple applications)
Target phenomenon: Indic language generation capabilities
Phenomenon defined by authors: Yes
Phenomenon definition: The phenomenon is defined is the ability to generate text in Indic languages.
Definition scope: Subset
Definition integrity: Composite phenomenon
Definition integrity detail: Yes
Phenomenon contested: Widely-agreed
Task definition: Generation tasks are  cross-lingual summarization, machine translation, and cross-lingual question answering.
Task item definition: Summarization and translation: A text sentence
Question-answering: A sentence as a question and multiple choices 
Task source: Modified from another benchmark (e.g. translation into another language)
Task ecology: Constructed task (e.g. predicting medical diagnoses from clinicians' notes)
Dataset size: train:~10k, test:~100k, dev:~60k
Dataset metadata detail: language
Sampling method: Random sample (creators defined a task space and sampled from it)
Split reviewed: Test, Train
Response format: Multiple choice, Free response (e.g. summary paragraph, executable code)
Metric definition: Exact Match (accuracy, F1, precision, recall), n-gram (BLEU, ROUGE, chrF)
Metric aggregation: Simple Mean
Metric statistics: Simple mean
Metric subscores: Yes
Metric subscores detail: language, and sub-task
Metric access: Outputs alone
Few-shot evaluation: Yes
Authorship: Industry
Benchmark availability: Yes
Task face validity: Yes
Metric face validity: Yes
Results interpreted by authors: Yes
Results compared: No
Results comparison explanation: No comparisons made
Results realism: No
Human baseline: No
Authors discuss validity: No
Taxonomy: NLP / 
Alternate taxonomy: Multilinguality
New taxonomy: General Capability / Multilinguality