DialettiBot: a Telegram Bot for Crowdsourcing Recordings of Italian Dialects
Authors: Federico Sangati, Ekaterina Abramova, Johanna Monti (Università di Napoli L'Orientale, Nijmegen University, 2018)
Topic: Chatbots and Dialogue Systems

Abstract
In this paper we describe DialetEnglish. In this paper we describe DialettiBot, a Telegram based chatbot for crowdsourcing geo-referenced voice recordings of Italian dialects. The system enables people to listen to previously recorded audio and encourages them to contribute to building a collective linguistic resource by sending voice recordings of their own spoken dialects. The project aims at collecting a large sample of voice recordings in order to promote knowledge of linguistic variation and preserve proverbs or idioms typical for different local dialects. Moreover, the collected data can contribute to several voice-based Natural Language Processing (NLP) applications in helping them understand utterances in non-standard Italian.

Introduction
It is commonly known that Italy has an abundance of different dialects, such as Florentine, Venetian, and Neapolitan. These dialects are not only characterized by simple phonetic variation as it is usually meant by this term, but they are proper Romance languages, with a fully developed grammar and lexicon. As Repetti puts it: The Italian ‘dialects’ [...] are daughter languages of Latin and sister languages of each other, of standard Italian, and of other Romance languages, and they may be as different from each other and from standard Italian as French is from Portuguese. (Repetti, 2000) This dialectical variety is a resource that deserves to be studied and preserved for both cultural and applied reasons. The former, because it is quickly disappearing with less and less people who regularly use dialect at home and in public places. According to UNESCO “Atlas of the World’s Languages in Danger”,1 there are about 2,500 endangered languages worldwide. In Italy, thirty dialects are at risk of extinction, such as friulano, ladino and veneciano.2 The applied motivation is that in recent years we have witnessed a signiﬁcant growth in the number of voice-based NLP applications (such as virtual assistants), which are currently not trained on local dialects and therefore perform poorly with a number of Italian speakers. In this paper we present a freely available tool that enables geo-referenced recording of Italian dialects: DialettiBot, a Telegram based chatbot, whose aim is to collect a large sample of voice recordings, promoting preservation of linguistic variation and its use in NLP applications. The rest of the paper is organized as follows: in section 2 we describe related work, in section 3 the implemented system and in section 4 the collected data.

Conclusion
We have presented DialettiBot, a chatbot system based on Telegram for crowdsourcing georeferenced recordings of Italian dialects. 17https://leaﬂetjs.com 18As of 31st of September 2018. 19Created via https://mapmakerapp.com. Figure 3: Frequency of approved recordings collected over time. Figure 4: Heat map of the approved recordings. Preliminary tests show that the system can be easily used by anyone who wishes to collect data in the ﬁeld as well as the dialect speakers themselves. The recording quality is good and the data is easily exportable to be used for further processing in the service of linguistic research or NLP applications. At the same time, the current state of the project suffers from a number of limitations that need to be addressed in future work and that we discuss next. First, the preliminary tests have not been informed by a detailed linguistic study of dialectical variation nor have we implemented a methodology for data collection. This is because the tests have been carried out as a proof-of-concept for the technology used to collect linguistic resources rather than a full-ﬂedged linguistic project. Future tests will require a more careful consideration for dialect characteristics in the Italian language, the type of data that would be most valuable (spontaneous speech vs a set of set sentences etc.) and a construction of precise, reproducible instructions for the contributors. Second, as described in section 3, we make use of a centralized validation procedure to approve a subset of recordings. However, since we have no complete knowledge of all Italian dialects we may end up accepting recordings which are not mapped to the correct location. In the future, we would like to decentralize the procedure, by delegating the approval to a higher number of volunteers spread out in all the regions, so that each new recording will get validated by the closest volunteer. Finally, the number of users and recordings collected so far is relatively modest. This is due to the fact that no effort has been undertaken so far to promote its use by researchers or the general public. Accordingly, the current goal of the project is to get support from cultural institutions (both at a local and at a national level) to help us engage the citizens in this crowdsourcing effort, as well as academic partners to further reﬁne the methodology and extend the chatbot capabilities. We believe this project could contribute to help safeguard the Italian dialectic richness and collect useful resources for NLP applications, as we intend to make all recordings openly available for other researchers to use.20 20We are planning to upload the data to the Common Language Resources Infrastructure (CLARIN).