Everything you need to know about Multilingual LLM s: Towards fair, performant and reliable models for languages of the world (2023)

Abstract: This tutorial will describe various aspects of scaling up language technologies to many of the world’s languages by describing the latest research in Massively Multilingual Language Models (MMLMs). We will cover topics such as data collection, training and fine-tuning of models, Responsible AI issues such as fairness, bias and toxicity, linguistic diversity and evaluation in the context of MMLMs, specifically focusing on issues in non-English and low-resource languages. Further, we will also talk about some of the real-world challenges in deploying these models in language communities in the field. With the performance of MMLMs improving in the zero-shot setting for many languages, it is now becoming feasible to use them for building language technologies in many languages of the world, and this tutorial will provide the computational linguistics community with unique insights from the latest research in multilingual models.

## tutorial content this tutorial will describe various aspects of scaling up language technologies to many of the world's languages by presenting the latest research in massively multilingual language models (mmlms). we will cover topics such as data collection, training and fine-tuning of models, responsible ai issues such as fairness, bias and toxicity, linguistic diversity and evaluation in the context of mmlms, specifically focusing on issues in non-english and low-resource languages. further, we will also talk about some of the real-world challenges in deploying these models in language communities in the field. with the performance of mmlms improving in the zero-shot setting for many languages, it is now becoming feasible to use them for building language technologies in many languages of the world, and this tutorial will provide the computational linguistics community with unique insights from the latest research in multilingual models. although past tutorials have covered some of these topics (such as linguistic diversity, data and training of models), there has been a lot of interesting research in the recent past that the cl community will benefit from knowing about. further, this will be the first tutorial (as per our knowledge) that will discuss issues of deployment in language communities and responsible ai in the context of multilingual models. this tutorial will present a broad survey covering work done by several research groups (as indicated in the references), including work done by the authors. type of the tutorial: cutting-edge target audience and pre-requisites: the target audience for this tutorial are researchers from in-dustry and academia who work on large language models, and are interested in learning about the latest research in multilingual models to build systems for non-english languages, low-resource languages and multilingual speakers. we will not be covering the basics of llms, so we expect that the audience will be familiar with (at least the english versions of) models such as bert. ## other estimate of audience size: 50 venues: we would prefer acl 2023 to be the venue for the tutorial, but emnlp and eacl are also acceptable. we do not forsee any special requirements for technical equipment.