Automatic Speech Recognition and Error
Analyses of Dutch Oral Cancer Speech
Kirsten Wildenburg1, Bence M. Halpern2,3,5, Teja Rebernik1,3, Thomas Tienkamp1, Rob J.J.H. van Son2,3, Vass Verkhodanova1, Max J.H. Witjes1,4 and Martijn Wieling1
1University of Groningen, 2University of Amsterdam, 3Netherlands Cancer Institute, 4University Medical Center Groningen, 5Delft University of Technology

MOTIVATION• ASR has significantly improved due to theintroduction of deep learning [1]•••

RQs AND HYPOTHESESMETHODS
1. NKI-UMCG-RUG oral cancer speech corpus:• RQ1 ‘What phonemes in oral cancer speech cause
N = 11 (oral cancer: 6; control: 5)higher recognition error rates in a standard ASR
However, oral cancer speech is recognized poorly
a. 3 mandibulectomy patients
system compared to healthy speech in Dutch?’by ASR systems [2]b. 3 (partial) glossectomy patients→ Plosives, alveolar sibilants and certain vowels
It is important to develop ASR systems specificallyare expected to cause higher error rates [2,4,5] 2. Data preprocessed with librosa Python library [8]for oral cancer speech to ameliorate patients’
3. Data run through ESPnet [9] with a Dutch pre• RQ2 ‘Does the surgical treatment of oral cancer
quality of life [3]trained Conformer [10] model
patients influence the ASR performance on oral
A phoneme-level error analysis could potentially
4. Extensive error analyses:
cancer speech?’guide future ASR development
1. WER→ Mandibulectomy is expected to impact ASR
𝐼𝑛𝑠𝑒𝑟𝑡𝑖𝑜𝑛+𝑆𝑢𝑏𝑠𝑡𝑖𝑡𝑢𝑡𝑖𝑜𝑛+𝐷𝑒𝑙𝑒𝑡𝑖𝑜𝑛
performance more than for glossectomy
2. PER𝑁[4,6,7]
3. AFERRESULTS: WER• Healthy speech > oral cancer speech (see Table 1)• Mann Whitney U test: W=0, p=0.0043 → significant
Table 1. Overview of the word recognition errors in percentages. Blue bold numbers indicate for which participant in eachspeaker group the ASR system achieved the best performance per column. Orange bold numbers represent the worst ASR
performance per column for both speaker groups.

RESULTS: PER• Threshold healthy speech: 10%• Phonemes with PER over threshold are /i(ː), y, ɛ, ŋ, h, j/ (see Figure 2)• Threshold oral cancer speech: 45%• Phonemes with PER over threshold are /i(ː), k, ŋ, j/ (see Figure 2)

Figure 2. PER (%) results for healthy speakers and oral cancer patients.

••

Generally, patients with a glossectomy > patients with a mandibulectomy• Except for /i(ː), k, ŋ/ (see Figure 3)

Patients with a glossectomy >patients with a mandibulectomy(see Figure 1)• Independent Samples t-test:
t(4)=1.03, p=0.36,
95% CI [-34.44, 75.18]→ failed to reach significanceFigure 1. WER (%) results for oral cancer
patients grouped by surgical treatment.
Figure 3. PER (%) results for oral cancer patients with mandible and tongue surgery.

DISCUSSION• Particularly /k/ is challenging to capture for oral cancer speech, which is supported by AFER analysis, as plosives and velars elicit highest and second highest
recognition error rates in oral cancer speech• In contrast with our expectations, sibilants were relatively well captured• Speech of patients who underwent a mandibulectomy obtains higher recognition error rates than the speech of patient who underwent a (partial) glossectomy,
although the difference fails to reach significance• Amount of data in this study was limited, and caution should be taken regarding the generality of our results• Thus, plosives, especially /k/, elicit higher recognition error rates in Dutch oral cancer speech and the type of surgical treatment slightly affects ASR performance
REFERENCES
4.
1.
Graves, A. and Jaitly, N. (2014). Towards end-to-end speech recognition with recurrent neural networks. In International conferenceon machine learning, pages 1764–1772. PMLR.
2.
Halpern, B. M., Feng, S., van Son, R., van den Brekel, M., and Scharenborg, O. (2022). Low-resource automatic speech recognition
5.
and error analyses of oral cancer speech. Speech Communication.
3.
Epstein, J. B., Emerton, S., Kolbinson, D. A., Le, N. D., Phillips, N., Stevenson-Moore, P., and Osoba, D. (1999). Quality of life and oral 6.
function following radiotherapy for head and neck cancer. Head Neck, 21(1):1–11.

Take a picture to
download the fullpaper

Borggreven, P. A., Verdonck-de Leeuw, I., Langendijk, J. A., Doornaert, P., Koster, M. N., de Bree, R., and Leemans, C. R. (2005).
7.
Speech outcome after surgical treatment for oral and oropharyngeal cancer: A longitudinal assessment of patients reconstructed by 8.
a microvascular flap. Head Neck, 27(9):785–793.
Laaksonen, J.-P., Rieger, J., Harris, J., and Seikaly, H. (2011). A longitudinal acoustic study of the effects of the radial forearm free flap 9.
reconstruction on sibilants produced by tongue cancer patients. Clinical Linguistics Phonetics, 25(4):253–264.
Matsui, Y., Ohno, K., Yamashita, Y., and Takahashi, K. (2007). Factors influencing postoperative speech function of tongue cancer
10.
patients following reconstruction with fasciocutaneous/myocutaneous flaps—a multicenter study. International Journal of Oral andMaxillofacial Surgery, 36(7):601–609.

Mooshammer, C., Hoole, P., and Geumann, A. (2007). Jaw and Order. Language and Speech, 50(2):145–176.
McFee, B., Raffel, C., Liang, D., Ellis, D. P. W., McVicar, M., Battenberg, E., and Nieto, O. (2015). librosa: Audio and music signal
analysis in python. In Proceedings of the 14th python in science conference, pages 18–25.
Watanabe, S., Hori, T., Karita, S., Hayashi, T., Nishitoba, J., Unno, Y., Soplin, N. E. Y., Heymann, J., Wiesner, M., Chen, N.,Renduchintala, A., and Ochiai, T. (2018). ESPnet: End-to-End Speech Processing Toolkit. In INTERSPEECH, pages 2207–2211.
Gulati, A., Qin, J., Chiu, C.-C., Parmar, N., Zhang, Y., Yu, J., Han, W., Wang, S., Zhang, Z., Wu, Y., et al. (2020). Conformer: Convolutionaugmented transformer for speech recognition. arXiv preprint arXiv:2005.08100.