N usa C rowd: Open Source Initiative for I ndonesian NLP Resources (2023)

Abstract: We present NusaCrowd, a collaborative initiative to collect and unify existing resources for Indonesian languages, including opening access to previously non-public resources. Through this initiative, we have brought together 137 datasets and 118 standardized data loaders. The quality of the datasets has been assessed manually and automatically, and their value is demonstrated through multiple experiments.NusaCrowd’s data collection enables the creation of the first zero-shot benchmarks for natural language understanding and generation in Indonesian and the local languages of Indonesia. Furthermore, NusaCrowd brings the creation of the first multilingual automatic speech recognition benchmark in Indonesian and the local languages of Indonesia. Our work strives to advance natural language processing (NLP) research for languages that are under-represented despite being widely spoken.

## introduction indonesia is one of the most linguistically diverse and populous countries in the world, with over 270 million people living across 18,000+ islands. it covers more than 700 spoken languages, making up ∼10% of all languages in the world @xcite figure 1 : system architecture of nusacrowd. open access to the datasheets is provided through nusacatalogue, while dataloader scripts to access the resources are implemented in nusacrowd data hub. @xcite . however, the progress of nlp research in indonesian languages has been held back by factors including language diversity @xcite @xcite , orthographic variation (soeparno, 2015), resource limitation @xcite , and other societal challenges @xcite @xcite . existing nlp research mainly focuses on highresource languages @xcite @xcite , while the vast majority of languages with limited data-including most languages spoken in indonesia-are neglected @xcite . specifically, many indonesian nlp resources are scattered, undocumented, and not publicly available. these issues cause a severe data scarcity problem, which hinders nlp research in indonesian and other local languages spoken in indonesia from progressing. in this work, we introduce nusacrowd, 1 an open collaborative effort to gather and unify existing resources in indonesian languages for public use, and liberate non-public resources. this initiative has successfully collected a total of 137 datasheets with 118 standardized data loaders in nusacrowd data hub 2 . the datasets were manually assessed for data quality by multiple native speakers and experts in nlp. utilizing the datasets collected in nusacrowd, we introduce the first zero-shot nlu benchmark (nusanlu), zero-shot nlg benchmark (nusanlg), and multilingual asr benchmark (nusaasr) for indonesian languages. we evaluate various indonesian and multilingual models on the benchmarks. our contributions can be summarized as follows: ## nusacrowd in this section, we provide an overview of nusacrowd, a detailed description of the nusacrowd framework, the dataset curation process, as well as a detailed summary and statistics of the datasets contained in nusacrowd. ## nusacrowd benchmarks to showcase the benefit of nusacrowd, we develop three different benchmarks from subsets of the datasets. specifically, we develop benchmarks for indonesian and other local languages including a zero-shot nlu benchmark (nusanlu), a zero-shot nlg benchmark (nusanlg), and a multilingual asr benchmark (nusaasr). ## conclusion we have introduced nusacrowd, a combined resource for indonesian and regional languages, covering 137 datasets, 118 of which have a standardized loader. nusacrowd covers indonesian and 18 regional languages, encompassing 3 different data modalities. manual and automatic curation processes were conducted to verify the quality of the collected datasets. the effectiveness of nusacrowd is shown in three use cases: zero-shot nlu (nu-sanlu), zero-shot nlg (nusanlg), and multilingual asr (nusaasr) benchmarks. our experiments provide evidence regarding the efficiency of cross-tasks method over prompting for zero-shot nlu, the limited capabilities of existing large lms for handling nlg tasks in local languages, and the potential of joint multilingual multi-task learning for indonesian asr. we hope nusacrowd will benefit the research community as a data hub for indonesian and regional languages by facilitating easy access to datasets, as well as accelerating research and development.