# Effects of Sinusoidal Model on Non-Parallel Voice Conversion with Adversarial Learning
###### Abstract  
Voice conversion (VC) transforms the speaking style of a source speaker to the speaking style of a target speaker by keeping linguistic information unchanged. Traditional VC techniques rely on parallel recordings of multiple speakers uttering the same sentences. Earlier approaches mainly find a mapping between the given source-target speakers, which contain pairs of similar utterances spoken by different speakers. However, parallel data are computationally expensive and difficult to collect. Non-parallel VC remains an interesting but challenging speech processing task. To address this limitation, we propose a method that allows a non-parallel many-to-many voice conversion by using a generative adversarial network. To the best of the authors' knowledge, our study is the first one that employs a sinusoidal model with continuous parameters to generate converted speech signals. Our method involves only several minutes of training examples without parallel utterances or time alignment procedures, where the source-target speakers are entirely unseen by the training dataset. Moreover, empirical study is carried out on the publicly available CSTR VCTK corpus. Our conclusions indicate that the proposed method reached the state-of-the-art results in speaker similarity to the utterance produced by the target speaker, while suggesting important structural ones to be further analyzed by experts. sinusoidal model; non-parallel voice conversion; generative adversarial networks; continuous parameters +
Footnote †: 2021: 12 August 2021  
of vocoding algorithms are prone to generating robotic-sounding results. Several non-linear spectral mapping techniques address these constraints: frequency warping [10], non-negative matrix factorization (NMF) [11], restricted Boltzmann machine (RBM) [12], artificial neural networks (ANNs) [13], feed-forward deep neural networks (FF-DNNs) [14], and long short-term memory (LSTM) [15]. With the novelty of WaveNet architecture, it was applied in [16] as a vocoder and the quality of synthesized speech was greatly increased. Nevertheless, most of these conventional VC methods require accurately aligned parallel data for training, where the source speaker and target speaker should read the same sentence, and their pairs of frames must be aligned. On the other hand, it is not always feasible to collect parallel utterances. Even when parallel data are accessible, the required alignment procedures introduce artifacts and lead to speech-quality degradation. Furthermore, these methods may need some sort of automatic time alignment between the speech data of speakers if they were not accurately time aligned. This can lead to a trick and influence the training part of the VC task [17]. In addition, there is another major problem: if both source and target speakers speak different languages or have different accents, building parallel data would become an unviable task. To overcome these limitations, numerous attempts have been made to use non-parallel voice conversion methods. The concept of this paradigm is that voice examples of many speakers are provided, but these examples do not appear in the same sentence. In other words, linguistic features are not shared between speakers. Hence, the VC model can generalize to different voices in the same dataset or even outside it. This type of voice conversion technique can be reliably applied as one-to-many [18] and many-to-many VC [19]. The converted speech quality of non-parallel VC systems is still likely not better than that available from systems using parallel data. Thus, suggesting a non-parallel VC technique can be very challenging to estimate natural voices. Over the years, a well-designed VC system has typically consisted of an analysis of the input waveform into acoustic features, a conversion function to map the source-target features, and synthesizing the converted features to a target speaker. Note that these phases may degrade the performance of the entire voice conversion system.