Playing with NeMo for Building an Automatic Speech Recogniser for Italian
Authors: Fabio Tamburini (Università di Bologna, 2021)
Topic: Multimodal

Abstract
This paper presents work in progress for the creation of a Large Vocabulary Automatic Speech Recogniser for Italian using NVIDIA NeMo. Thanks to this package, we were able to build a reliable recogniser for adults' speech by fine tuning the English model provided by NVIDIA and rescoring it with powerful neural language models, obtaining very good performances. The lack of a standard, reliable and publicy available baseline for Italian motivated this work.

Introduction
The advent of the “Deep Learning Revolution” introduced astonishing changes also in the field of speech processing allowing for the development of brand new tools and devices able to recognise and synthesise speech exhibiting performances never seen before. It is sufficient to think to the new virtual assistants that populates our houses and mobile phones for getting an immediate idea about the improvements in this research field. Most big IT companies developed, in the past 3/4 years, solutions well integrated with various devices that include high performance tools for speech processing. However, these solutions very often are not released freely, sometimes they require registrations and fees and, in the best situations, codes are free, but the models for a specific language are not available. A notable exception regards NVIDIA NeMo[Other exceptions providing also multilingual models including Italian are Facebook Wav2Vec and SpeechBrain.], a conversational AI toolkit built for researchers working on Automatic Speech Recognition (ASR), Natural Language Processing (NLP), and Text-To-Speech synthesis (TTS). The primary objective of NeMo is to help researchers from industry and academia to reuse prior work, namely code and pretrained models for various languages, and make it easier to create new conversational AI models, maybe adapting tools and models to specific languages or particular domains. This paper reports an attempt to build a high performance Large Vocabulary ASR system for Italian adults' speech by exploiting all the features available in NeMo and most of the largest Italian spoken corpora available to the community. Section <ref> describes the various speech datasets used for developing the model, followed by Section <ref> that describes the state of the art; in Section <ref> we will describe the NeMo ASR model used in the experiments and Section <ref> will discuss the experiments and the obtained results. Section <ref> draws some provisional conclusions about our work.

Conclusion
This paper presented work in progress for the construction of a reliable and performing ASR system for Italian adults' speech. Thanks to the NVIDIA NeMo package, we were able to produce a very strong baseline reaching a WER = 9.67% over the MCV7 test set. This is only the beginning of our work, as any change in the kind of speech used to train the system could degrade the whole performance, but, having used a collection of four different datasets containing thousands of different speakers and speech utterances for setting up such ASR system, we believe that the result should be robust enough. Unfortunately, the lack of a standardised benchmark for Italian does not allow for a quantitative and objective evaluation of this statement. End-to-end character ASR model, and its improvement on WER, is only part of the game: the work on decoding and rescoring procedures produced much more improvements. Thus, the most important “take home lesson” is certainly to focus on the development of high performance LM specifically tuned for ASR. All the models presented in this paper as well as the scripts and additional codes for using NeMo and generating the results will be made available[<https://github.com/ftamburin/ItaNeMoASR>].