# Effectiveness of automatic phoneme recognition on a pseudoword production task after oral or oropharyngeal cancer

Maëlys Leiçarrague, Tuan Nguyen, Alain Ghio, Julien Pinquier, Muriel Lalain, Virginie Woisard, Clémence Devoucoux, Corinne Fredouille, Mathieu Balaguer

<figure><img src="image_1.png" /></figure>

## To cite this version:

Maëlys Leiçarrague, Tuan Nguyen, Alain Ghio, Julien Pinquier, Muriel Lalain, et al.. Effectiveness of automatic phoneme recognition on a pseudoword production task after oral or oropharyngeal cancer. IALP 2025 33rd World Congress, Aug 2025, San Giljan, Malta. hal-05413235

HAL Id: hal-05413235

https://hal.science/hal-05413235v1

Submitted on 12 Dec 2025

HAL is a multi-disciplinary open access archive for the deposit and dissemination of scientific research documents, whether they are published or not. The documents may come from teaching and research institutions in France or abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est destinée au dépôt et à la diffusion de documents scientifiques de niveau recherche, publiés ou non, émanant des établissements d'enseignement et de recherche français ou étrangers, des laboratoires publics ou privés.

<figure><img src="image_2.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# Effectiveness of automatic phoneme recognition on a pseudoword production task after oral or oropharyngeal cancer

<figure><img src="image_3.png" /></figure>

<figure><img src="image_4.png" /></figure>

<figure><img src="image_5.png" /></figure>

## BACKGROUND

* Oral / oropharyngeal cancer treatments induce **anatomical changes**

* These alterations affect speech production at the **segmental level** [1]

* They motivate analysis of the links between **anatomy and phoneme recognition** [2]

* Automatic acoustic-phonetic decoding systems make this analysis now possible

* Their **effectiveness** needs to be evaluated in the context of pathological speech, i.e. in speech after ENT cancer in our case [3]

## OBJECTIVE

To study the **effectiveness*** of automatic phonemic recognition on a pseudoword production task after oral / oropharyngeal cancer

*Effectiveness* = recognition in relation to a target, despite production / perception errors, and not in relation to a human reference

33RD
WORLD
CONGRESS
OF THE IALP

Innovation and Inspiration
in Communication
Sciences and Disorders

<figure><img src="image_6.png" /></figure>

AUGUST 24-28
MALTA 2025

<figure><img src="image_7.png" /></figure>

<figure><img src="image_8.png" /></figure>

ISHLA
The International Society for
Hearing, Language and
Communication Sciences

<figure><img src="image_9.png" /></figure>

1924-2024

<figure><img src="image_10.png" /></figure>

Low metrics
(averages close to 0)
=
better recognition effectiveness

## METHODS

**80 subjects** treated for oral / oropharyngeal cancer

Playback recordings
(list of **52 pseudowords**)

Automatic phonemic
transcription

* TDNN-HMM

* Transformer CTC

* Transformer
  CTC fine-tuned

Real-time perceptive
phonemic transcription

+ Speech impairment
severity score

<figure><img src="image_11.png" /></figure>

**Statistics** : Friedman test, then pairwise comparison
(Durbin-Conover)

1. **Comparison of the effectiveness of the 4 systems**
   (3 automatic and 1 perceptive)

2. **Comparison of the effectiveness of the 4 systems**
   according to severity groups

**Recognition effectiveness metrics**
(Conditions : isolated, unduplicated pseudowords)

* PER (Phone Error Rate)
  => Proportion of errors

$$ PER = \frac{\text{Deletions} + \text{Substitutions} + \text{Insertions}}{\text{Total number of phonemes in the target pseudoword}} $$

* PPD (Perceived Phonological Deviation [4])
  => Severity of errors

$$ PPD = \frac{\text{Average number of phonological features of deviation per phoneme between recognition and target}}{\text{Total number of phonemes}} $$

## RESULTS

Final corpus:
**79 subjects**
**4 107 pseudowords**
(exclusions)

1. **Comparison of the effectiveness of the 4 systems**

<table><thead><tr><th>Systems</th><th>PER</th><th>PPD</th></tr></thead><tbody><tr><td>TDNN-HMM</td><td>0.59</td><td><strong>1.54</strong></td></tr><tr><td>Transformer CTC</td><td>0.70</td><td>1.83</td></tr><tr><td>Transformer CTC fine-tuned</td><td><strong>0.52</strong></td><td>1.76</td></tr><tr><td>Human transcription</td><td><strong>0.24</strong></td><td><strong>0.44</strong></td></tr></tbody></table>

**Better recognition effectiveness :**

* Lowest PER and PPD : **Human transcription**

Among automatic systems :

* Lowest PER : **Transformer CTC fine-tuned**

* Lowest PPD : **TDNN-HMM**

## CONCLUSION

* **Automatic acoustic-phonetic decoding systems** applied to speech after cancer are **less effective than human experts**

* Possible causes : **systems architecture → frequent insertions; training on "healthy" speech; sound events in the signal; no phonemic restoration effect** (unlike in humans); ...

* Prospects : further research into the **link between the errors found and the patients analytical deficits**

2. **Comparison of the effectiveness of the 4 systems**
   according to severity groups

<table><thead><tr><th>Severity</th><th>Systems</th><th>PER</th><th>PPD</th></tr></thead><tbody><tr><td rowspan="4">0-6<br>(n = 30)</td><td>TDNN-HMM</td><td>0.77</td><td><strong>2.15</strong></td></tr><tr><td>Transformer CTC</td><td>0.86</td><td>2.28</td></tr><tr><td>Transformer CTC fine-tuned</td><td><strong>0.68</strong></td><td>2.36</td></tr><tr><td>Human transcription</td><td><strong>0.36</strong></td><td><strong>0.63</strong></td></tr><tr><td rowspan="4">7-10<br>(n = 36)</td><td>TDNN-HMM</td><td>0.42</td><td><strong>1.01</strong></td></tr><tr><td>Transformer CTC</td><td>0.57</td><td>1.43</td></tr><tr><td>Transformer CTC fine-tuned</td><td><strong>0.37</strong></td><td>1.21</td></tr><tr><td>Human transcription</td><td><strong>0.15</strong></td><td><strong>0.25</strong></td></tr><tr><td rowspan="4">0-9<br>(n = 18)</td><td>TDNN-HMM</td><td>0.69</td><td><strong>1.87</strong></td></tr><tr><td>Transformer CTC</td><td>0.79</td><td>2.07</td></tr><tr><td>Transformer CTC fine-tuned</td><td><strong>0.61</strong></td><td>2.09</td></tr><tr><td>Human transcription</td><td><strong>0.29</strong></td><td><strong>0.51</strong></td></tr><tr><td rowspan="4">10<br>(n = 48)</td><td>TDNN-HMM</td><td>0.28</td><td><strong>0.63</strong></td></tr><tr><td>Transformer CTC</td><td>0.47</td><td>1.13</td></tr><tr><td>Transformer CTC fine-tuned</td><td><strong>0.24</strong></td><td>0.77</td></tr><tr><td>Human transcription</td><td><strong>0.12</strong></td><td><strong>0.20</strong></td></tr></tbody></table>

**Better recognition effectiveness according to severity :**

* Lowest PER and PPD : **less severe subjects (7-10)** for automatic and human

* Automatic **converges towards human when we isolate the less severe subjects (10)**

## REFERENCES

[1] Mlynarek, A., Rieger, J., Harris, J., O'Connell, D., Al-Qahtani, K., Ansari, K., Chau, J., & Seikaly, H. (2008). Methods of functional outcomes assessment following treatment of oral and oropharyngeal cancer: review of the literature. *Journal of Otolaryngology - Head & Neck Surgery*, 37(1), 2–10. https://doi.org/10.2310/7070.2008.1001

[2] Lazarus, C. L., Husaini, H., Anand, S. M., Jacobson, A. S., Mojica, J. K., Buchbinder, D., & Urken, M. L. (2013). Tongue Strength as a Predictor of Functional Outcomes and Quality of Life after Tongue Cancer Surgery. *Annals of Otology, Rhinology & Laryngology*, 122(6), 386–397. https://doi.org/10.1177/000348941312200608

[3] Halpern, B. M., Feng, S., van Son, R., van den Brekel, M., & Scharenborg, O. (2022). Low-resource automatic speech recognition and error analyses of oral cancer speech. *Speech Communication*, 141, 14–27. https://doi.org/10.1016/j.specom.2022.04.006

[4] Ghio, A., Lalain, M., Rebourg, M., Marczyk, A., Fredouille, C., & Woisard, V. (2022). Validation of an Intelligibility Test Based on Acoustic-Phonetic Decoding of Pseudo-Words: Overall Results from Patients with Cancer of the Oral Cavity and the Oropharynx. *Folia Phoniatrica et Logopaedica*, 74(3), 209–222. https://doi.org/10.1159/000519427

## Contact Details and QR Code

Maëlys Leiçarrague, First author : maelys64240@gmail.com

Mathieu Balaguer, Attending author (QR code) : mathieu.balaguer@irit.fr

<figure><img src="image_12.png" /></figure>