  A benchmark provides an ecosystem to measure the advancement of models with
standard datasets and automatic and human evaluation metrics. We introduce
IndoNLG, the first such benchmark for the Indonesian language for natural
language generation (NLG). It covers six tasks: summarization, question
answering, open chitchat, as well as three different language-pairs of machine
translation tasks. We provide a vast and clean pre-training corpus of
Indonesian, Sundanese, and Javanese datasets called Indo4B-Plus, which is used
to train our pre-trained NLG model, IndoBART. We evaluate the effectiveness and
efficiency of IndoBART by conducting extensive evaluation on all IndoNLG tasks.
Our findings show that IndoBART achieves competitive performance on Indonesian
tasks with five times fewer parameters compared to the largest multilingual
model in our benchmark, mBART-LARGE (Liu et al., 2020), and an almost 4x and
2.5x faster inference time on the CPU and GPU respectively. We additionally
demonstrate the ability of IndoBART to learn Javanese and Sundanese, and it
achieves decent performance on machine translation tasks.
