# Enrichment of Oesophageal Speech: Voice Conversion with Duration-Matched Synthetic Speech as Target
## 1 Introduction  
Laryngectomy is the surgical procedure of removing the larynx [1]. In addition to several functional disorders and lifestyle changes [2], this results in the loss of vocal folds and the patient's pre-surgery speech [3]. One of the several alternative ways that a laryngectome can communicate [3,4] is to speak using the vibrations of the pharyngoesophageal segment [5], known as Oesophageal Speech (OS). Generating OS introduces acoustic artefacts [6] and makes OS less intelligible [7,8], which affects communication, social activities and quality of life [2,9]. OS is less intelligible and more effortful to listen to compared to healthy speech (HS). This is evident from previous listening experiments [10,11] as well as acoustic characteristics and challenges of OS [12]. Prolonged exposure to effortful speech causes fatigue in listeners [13]. Therefore, there is a strong motivation to make OS more intelligible and pleasant to listen to. We aim to enrich OS by closing the OS-HS gaps in intelligibility, quality and listening effort (LE). Modern speech technologies and machine learning have great potential for use in the healthcare sector, be it for improvement of healthcare services [14] or to aid patients with speech impairments [15]. One such application is transforming pathological speech with the aim of making it more intelligible, pleasant and easier to process. This can reduce the load on the listeners and improve communication for people with speech pathologies. One of the possible approaches to enrich OS is to use a voice conversion (VC) system. The goal of a VC system is to convert the utterances of a source speaker to sound like thoseof a target speaker [16]. In the OS enrichment context, utterances of an OS speaker can be mapped to those of a healthy speaker, thereby having OS acquire characteristics of HS. Some OS enrichment has been done using statistical VC methods such as Gaussian Mixture models (GMMs) [17; 18; 19]. In these methods, OS and HS are modelled by a linear combination of Gaussian distributions. In the training process, the Gaussian distributions of OS are mapped to those of HS. The output of such a training session is a conversion function mapping OS to HS. This conversion function can then be used to convert new OS samples, thereby getting OS speech that has characteristics of HS. In recent times, Deep Neural Networks (DNN) are more popular and effective compared to GMM based methods for enhancement of alaryngeal speech [20; 21; 22; 23] and other types of pathological speech [24; 25]. Another attempt to enrich OS was by using the eigenvoices concept [26], which was inspired by the eigenfaces concept [27]. Some studies have used filtering approaches [28], formant synthesis [29] and increasing the harmonics to noise ratio of OS [30]. Like our previous approaches [31; 32], the proposed method is also based on VC. The bidirectional long short-term memory (BLSTM) based transformation [31] had better Automatic Speech Recognition (ASR) scores compared to OS. The method used by [PERSON] et al. [32] was inspired by a Phonetic Posteriorgrams (PPG) based system [33] which had good results for HS-HS VC. When applied for OS-HS VC, there was no improvement in ASR. Mel Cepstral Distortion (MCD) was reduced by both systems. Unprocessed OS was preferred over both of the systems in preference tests. VC systems may be parallel (requires temporally aligned source-target utterance pairs) or non-parallel (requires many hours of speech data). Due to data limitations (100 sentences per speaker), parallel VC is best suited for our purposes. A parallel VC requires the parallel source and target sentences to be aligned for training. This is primarily done by Dynamic Time Warping (DTW) alignment which finds an optimal match based on similarities in the two sequences. [PERSON] et al. [34] describe some challenges of DTW in the context of VC. One of them is the presence of silences or extra sounds in the source and not in the target. Another one is the poor estimation of end points of silences and phonemes.