Massively Multilingual Natural Language Understanding 2022 ( MMNLU -22) Workshop and Competition (2022)

Abstract: To be writen (workshop summary paper)

## introduction according to a 2020 study by juniper research @xcite it is expected that by 2024 there will be over 8 billion virtual assistants worldwide, the majority of which will be on smartphones. additionally, over 100 million smart speakers have been sold, and virtual assistants continue to be integrated into new products. these devices have in common that humans interact with them via natural language interfaces. this development has significantly boosted research to advance natural language understanding. however, most natural language understanding work focuses on only a few of the more than 4,000 written languages in the world. the limitation is driven by the lack of labeled data, the expense associated with humanbased quality assurance, model maintenance, update costs, and more. to overcome these hurdles, further research in the field of multilingual natural language understanding is needed to enable natural language understanding for currently not-or under-served languages. @xcite and related work, we have seen progress in recent years on the expansion of machine translation into the domain of under-served languages both by advancing science as well as creation of corpora in machine translation. however, in areas as nlu modeling for virtual assistants, many of these limitations still remain. the vision of this workshop is to address the limitations in nlu and help propel nlu technology into the 50-language, 100-language, and even the 1,000-language regime, both for production systems and for research endeavors, succinctly captured by our slogan, let's scale natural language understanding technology to every language on earth!. we do this via a threepronged approach. first, we created and released the multilingual amazon slu resource package (slurp) for slot-filling, intent classification, and virtual assistant evaluation, or massive dataset @xcite , containing 1 million realistic, parallel, labeled virtual assistant text utterances spanning 51 languages. second, we hosted the massively multilingual nlu (mmnlu) 2022 challenge, a competition designed to advance massively multilingual nlu modeling. finally, we organized the first mmnlu workshop to bring together researchers working in the field of nlu. by providing much needed labelled data, motivating multilingual nlu exploration and bringing nlu researchers together to share findings and spark further collaboration, we hope to push the state-of-the art in multilingual natural language understanding technology. ## workshop overview the first mmnlu 2022 workshop is co-located with emnlp 2022 in abu dhabi. in our call for papers we asked for submissions relevant to the advancement of the field of multilingual nlp. we were particularly interested in submissions related to the shared tasks part of this workshops competition on the recently published massive @xcite dataset (see 4.1) or other multilingual data-sets. we sought work exploring multilingual representations, augmentation and preprocessing techniques, and more efficient models. ## paper submissions in total we received 12 submissions for the venue, of which 8 were accepted for presentation at the workshop. the papers being part of the proceedings for this workshop showed a wide variety of approaches to deal with the problem of massive multilingual systems. the investigations ranged from the evaluation and investigation of tokenization across languages, towards large language model or translation model based data augmentation as well as direct use of translation systems as a natural language understanding solution. also, investigations how to minimize degradation on other languages when trained only on a small set of languages, as well as how to use consistency regularization as a way to boost performance were part of the submissions to this workshop. another topic investigated by more than one submission was the investigation of code mixing and other cross lingual effects on natural language understanding performance. furthermore we also received a paper describing an investigation of how to design templates when seq2seq generation is used as a solution for zeroshot cross-lingual tagging. overall we are very grateful for the diverse set of research directions proposed in the submissions to this workshop. ## conclusions this paper presents an overview of the first mmnlu workshop collocated with emnlp 2022. the paper submissions and competition entries showed encouraging progress in the field of multilingual nlu. we hope that the findings presented as well as the collaborations initiated at this workshop, drives more progress in the field.