Proteno: Text Normalization with Limited Data for Fast Deployment in Text to Speech Systems (2021)

Abstract: Developing Text Normalization (TN) systems for Text-to-Speech (TTS) on new languages is hard. We propose a novel architecture to facilitate it for multiple languages while using data less than 3% of the size of the data used by the state of the art results on English. We treat TN as a sequence classification problem and propose a granular tokenization mechanism that enables the system to learn majority of the classes and their normalizations from the training data itself. This is further combined with minimal precoded linguistic knowledge for other classes. We publish the first results on TN for TTS in Spanish and Tamil and also demonstrate that the performance of the approach is comparable with the previous work done on English. All annotated datasets used for experimentation will be released.

## introduction text-to-speech synthesis (tts) consists of a number of processing steps that control the conversion of input text to output speech. text normalization (tn) is usually the first step for any tts system. it is defined as the process of mapping of written text to its spoken form. as per @xcite , semiotic class denotes things like numbers, dates, times, etc. that are written differently from the way they are verbalized. tn is the process of verbalizing instances of such classes. most systems are entirely based on hard coded rules which are neither scalable across languages nor easy to maintain. many machine learning based techniques have been proposed for tn but they still have heavy dependency on encoded linguistic knowledge or require considerable amount of annotated data making it difficult to scale. the contributions of this paper are as follows: i) presenting proteno, a novel architecture for tn * work done while at amazon with a granular tokenization mechanism, which requires minimal language specific rules, curtails unacceptable errors and is transferable to a large extent to multiple languages, ii) establishing an architecture which can be used to benchmark tn baselines for multiple languages with limited annotated data, iii) release of annotated tn datasets for tamil and spanish suitable for tts systems. as no benchmark datasets or baselines exist for tn for tts in spanish and tamil, we curated datasets for both and evaluated proteno on them. we also use the best performing system for tn in english and compare its results with previous work. ## related work in spite of the success of deep learning approaches in other natural language processing tasks, the problem of tn for tts systems still remains a challenging one @xcite . work has been done to solve tn by pure encoder-decoder methods particularly recurrent neural networks @xcite . however, authors have shown that even though such models can perform well overall, occasionally they can make "unacceptable errors" like reading "$2" as "two pounds" and thus rendering the system unsuitable for industrial tts applications. to curtail such unacceptable errors, previous work based on semiotic classification @xcite @xcite , are encoded with measures like weighted finitestate transducers (fsts) introduced by @xcite . fsts revolve around creating a weak covering grammar which encompasses language specific lexical information. although such grammars are easier to create as compared to a full blown grammar, they still need prior knowledge of the language and the language specific rules need to be coded in the system @xcite . to completely induce fst from training data, as suggested by @xcite , diverse and large amount of data is required. the data should represent all the forms a particular token can appear in a given language. such requirements for all semiotic classes limit the reproducibility of such models for a new language with limited annotated data. other language-agnostic approaches (a. @xcite also need large amounts of data (5m sentences for each language) as parallel corpus and can also result in unacceptable errors. our approach curtails such errors by breaking down complex entities like dates into multiple tokens by a granular tokenization mechanism and also by limiting which tokens can be accepted into a class. this mechanism, we will see, also enables the system to rely more on data and disambiguate context for normalizations without requiring the knowledge to be specifically coded in the system. ## proposed approach the target normalization can be directly predicted from unnormalized text with a seq2seq architecture @xcite by treating tn as a machine translation task @xcite with the previously mentioned limitations. a way to limit the unacceptable errors in such systems would be to limit the kind of normalizations the network can generate for a token @xcite . on the other hand, solutions based on semiotic classification convert tn into a sequence tagging problem, where each class has associated mechanisms for normalizing the corresponding unnormalized token(s). it produces verbalizations by first suitably tokenizing the input, then classifying the tokens, and then verbalizing each token according to its corresponding class. these approaches often have a complex tokenization mechanism which is not easily transferable across languages and also need all the possible classes to be exhaustively defined manually. we solve both these problems by a granular tokenization mechanism which extends the concept of semiotic classification to a granular level wherein each unique unnormalized token to normalized token mapping can have a class of its own. the majority of the classes and their appropriate normalizations are automatically learnt from data. our classes represent whether a particular token is of a certain type and convert unnormalized tokens into their normalized form. the goal is to manually define the minimum possible set of classes and all the other classes will be automatically learnt from the data. the system learns when each class should be applied. the solution is divided into 4 stages: i) tokenization of unnormalized data, ii) data preparation, iii) classifying unnormalized tokens into correct classes, iv) normalizing tokens using the corresponding class. ## conclusions we propose a novel architecture suitable for scaling text normalization for tts across languages using minimal language specific rules, limited annotated dataset and while curbing unacceptable errors which makes it suitable for fast deployment in industry applications. we treat text normalization as a sequence classification problem while proposing a granular tokenizer which enables majority of normalizations to be automatically learnt from data. we experiment across 3 languages: spanish, tamil and english, while pre-coding maximum 5 classes with language specific logic. we also demonstrate that datasets of the order of 135k-500k tokens can give competitive performance while still being of a size practical for hand annotation. proteno consists of i) a granular tokenizer based on unicode classes, ii) a classifier of tokens into classes, either predefined or added based on the tokenized data, and iii) the class verbalizers, either defined by linguists for predefined classes or automatically learnt from the data. bilstms give the best performance with @xmath0.89 for spanish, @xmath1.3 for tamil and @xmath2.6 for english. in english, 88.2% of the normalizations were learnt automatically from data while using less than 3% of the data used in previous work @xcite and still showed comparable performance. given the simplicity of this architecture, we believe that proteno can be used to benchmark tn for many languages with limited annotated data. however, languages which are not separated by space or highly inflected languages will be a challenge for the proposed system @xcite . we leave the adaptation of proteno to more challenging languages for future work.