Compress Polyphone Pronunciation Prediction Model with Shared Labels (2020)

Abstract: It is well known that deep learning model has huge parameters and is computationally expensive, especially for embedded and mobile devices. Polyphone pronunciations selection is a basic function for Chinese Text-to-Speech (TTS) application. Recurrent neural network (RNN) is a good sequence labeling solution for polyphone pronunciation selection. However, huge parameters and computation make compression needed to alleviate its disadvantage. In contrast to existing quantization with low precision data format and projection layer, we propose a novel method based on shared labels, which focuses on compressing the fully-connected layer before Softmax for models with a huge number of labels in TTS polyphone selection. The basic idea is to compress large number of target labels into a few label clusters, which will share the parameters of fully-connected layer. Furthermore, we combine it with other methods to further compress the polyphone pronunciation selection model. The experimental result shows that for Bi-LSTM (Bidirectional Long Short Term Memory) based polyphone selection, shared labels model decreases about 52% of original model size and accelerates prediction by 44% almost without performance loss. It is worth mentioning that the proposed method can be applied for other tasks to compress the model and accelerate the calculation.

## introduction polyphone pronunciation prediction is a basic module of g2p @xcite in chinese textto-speech (tts) system, which provides the right pronunciation for chinese character. the algorithms of polyphone pronunciation prediction include dictionary-based algorithm, statistical machine learningbased algorithm like conditional random field (crf) in @xcite , and deep learning-based algorithm like rnn. dictionary-based method may fail for polyphone words problem, such as "朝阳" can be read as "chao2yang2" and "zhao1yang2", and out of vocabulary(oov) problem. crf and rnn perform well for polyphone pronunciation selection with context features. however, crf needs manually designed context features. neural network always has more parameters, larger model size and more expensive computation cost. in the application for embedded device, small model size and quick computation are necessary. so compression and acceleration of neural network is a hot research field in recent years. there are several methods to compress deep learning model: low precision of data format, quantification, pruning, low rank factorization, and knowledge distillation. low precision of data format replaces double or float with 16-bit float, which can reduce model size by quarter or half, but it cannot reduce running time when corresponding computations are not supported by existing instruction sets. quantification and pruning methods pack some weights into one and prune the weights close to zero. it can reduce the model size but cannot accelerate the computation. low rank factorization changes the network structure by factorizing large matrix into small matrixes. it can reduce the model size and accelerate the computation. our interest focuses on how to compress model that contains huge number of labels, and each input has a fixed label set. for example, each polyphone chinese character has several (less than 10) fixed pronunciations. if we take all pronunciations of polyphone characters as target labels, there are too many parameters in fully-connected layer before softmax. therefore, the computation of fully-connected layer and softmax will be costly. one idea is to use 1 character-1 model method, which can reduce the network complexity and computation cost, but it needs larger memory because of too many polyphone characters. this inspired us how we can share parameters in a single model which has comparable size to one-character model. we propose a method to share parameters by means of different characters sharing pronunciation labels. we randomly assign labels for each character's pronunciation under conditions of avoiding label conflicts and assuring that the target label set is small. this yields smaller model size and less computation cost. then we train our bi-lstm model with newly tagged corpus, and compare it with other compression methods in model size, memory usage and decoding time. the experimental results show our method can compress the model to half size and accelerate computation speed while maintaining a comparable performance compared with original model, overcoming the problem of too many labels. the digital labels for different polyphone characters can be the same but their real pronunciations may be different. for example, in the above sentence, "好hao4" and "中zhong4" share the digital label "9". we can translate the digital shared labels into real pronunciations with a dictionary based on the method of the following section. ## conclusion we propose a novel shared labels method to compress polyphone pronunciation selection model. it decreases size of models consisting of huge number of labels by mapping labels to small shared labels. our proposed method reduces the model size and memory usage remarkably, and accelerate decoding speed without performance loss compared with other methods. in the future, we will verify the compressed model on embedded device and investigate other tasks which can apply this method.