<figure><img src="image_1.png" /></figure>

HAL
open science

# Iterative LLM-based improvement for French Clinical Interview Transcription and Speaker Diarization

Ambre Marie, Thomas Bertin, Guillaume Dardenne, Gwenolé Quellec

► To cite this version:

Ambre Marie, Thomas Bertin, Guillaume Dardenne, Gwenolé Quellec. Iterative LLM-based improvement for French Clinical Interview Transcription and Speaker Diarization. 2026. hal-05512777

HAL Id: hal-05512777

https://hal.science/hal-05512777v1

Preprint submitted on 16 Feb 2026

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_2.png" /></figure>

Distributed under a Creative Commons CC BY 4.0 - Attribution - International License
Iterative LLM-based improvement for French Clinical Interview
Transcription and Speaker Diarization

Ambre Marie 1,2,**, Thomas Bertin³, Guillaume Dardenne¹, Gwenolé Quellec 1

¹ LaTIM UMR 1101 INSERM, Brest, France

² University of Western Brittany, Brest, France

³ University of Rouen Normandy, Rouen, France

ambre.marie@univ-brest.fr, thomas.bertin@univ-rouen.fr, guillaume.dardenne@inserm.fr,
gwenole.quellec@inserm.fr

**Abstract**

Automatic speech recognition for French medical conversations remains challenging, with word error rates often exceeding 30% in spontaneous clinical speech. This study proposes a multi-pass LLM post-processing architecture alternating between Speaker Recognition and Word Recognition passes to improve transcription accuracy and speaker attribution. Ablation studies on two French clinical datasets (suicide prevention telephone counseling and preoperative awake neurosurgery consultations) investigate four design choices: model selection, prompting strategy, pass ordering, and iteration depth. Using Gwen3-Next-80B, Wilcoxon signed-rank tests confirm significant WDER reductions on suicide prevention conversations (p < 0.05, n=18), while maintaining stability on awake neurosurgery consultations (n=10), with zero output failures and acceptable computational cost (RTF 0.32), suggesting feasibility for offline clinical deployment.

**Index Terms:** automatic speech recognition, clinical interview transcription, LLM post-processing, speaker attribution, French medical conversations

**1. Introduction**

Automatic speech recognition (ASR) still presents major limitations for concrete medical applications in French, where transcription errors can directly affect clinical analysis and downstream uses [1]. In clinical interviews, ASR performance remains far below that observed in controlled dictation settings. While dictation tasks often achieve Word Error Rates (WER) below 10%, general-purpose ASR systems typically reach WERs between 30% and 65% on spontaneous clinical interviews. Even domain-trained or medically adapted systems only reduce these error rates to approximately 18-35%, often failing to capture clinically important details under real-world conditions [2, 3].

Medical conversations in non-English languages remain substantially harder for ASR than English dictation, mainly due to the limited availability of in-domain training data and language-specific challenges [4]. In French, rich morphology and frequent homophones make lexical disambiguation particularly difficult: even state-of-the-art systems systematically confuse common homophones in conversational speech [5]. Moreover, high-quality labeled medical speech data in French and other non-English languages is much rarer than in English, especially for real clinical interviews rather than read or scripted speech [6].

Our baseline system (WhisperX large-v3 + Pyannote 3.1) achieves WDER of approximately 43% on both datasets. This high error rate reflects both ASR limitations: lexical errors on medical terminology, and speaker diarization errors during rapid turn-taking and overlaps. Also, the strictness of our linguist-produced reference transcriptions also preserves fine-grained phenomena of spontaneous speech production that are often suppressed by conventional ASR normalization pipelines, such as filled pauses, partial lexical initiations with restart, repetitions, and self-repairs.

Recent work has explored the use of large language models (LLMs) for post-processing ASR outputs. Adedeji et al. [7] investigated LLM-based post-processing for English primary care consultations using commercial models such as GPT-4, Claude, and Gemini, reporting reductions in WER as well as improvements in speaker-specific transcription quality. However, diarization improvements were mainly assessed qualitatively, and the study was limited to English data.

These limitations motivate the need for systematic evaluation of LLM post-processing approaches in non-English medical contexts. This work proposes a multi-pass LLM post-processing architecture using Gwen3-Next-80B (80B parameters) for French clinical interview transcriptions. The method iteratively alternates between Speaker Recognition (SR) and Word Recognition (WR) passes, and is validated in two distinct clinical domains: suicide prevention telephone counselling and preoperative awake neurosurgery consultations. Ablation studies demonstrate that this configuration achieves significant improvements in speaker attribution in our medical conversations while maintaining stability over two separate datasets.

**2. Material and Methods**

The complete source code, including the transcription pipeline, LLM post-processing implementation, and domain-adapted prompts, is publicly available at https://github.com/amarie-research/iterative-llm-clinical-transcription.

**2.1. Datasets**

Experiments are conducted on two French medical speech datasets drawn from distinct clinical contexts. Both datasets consist of non-scripted, real-world interactions and exhibit the variability typical of routine medical conversations. All recordings are in French and were manually transcribed by a linguist specialized in verbal interactions, to produce a high-quality ground truth. The transcriptions include fine-grained annotations of hesitations, pauses, overlapping speech, and speaker

<footnote>** indicates the corresponding author.</footnote>
changes, providing a strict reference for evaluating both trans-cription and diarization performance. Consistent with French interactional linguistics conventions [1, 8], reference transcrip-tions omit capitalization and punctuation, replacing them with prosodic markers (micropauses, intonation contours).

Table 1 summarizes the scale and structure of the two datasets.

### 2.1.1. Suicide Prevention (SP) dataset

The Suicide Prevention (SP) dataset consists of telephone calls conducted as part of post-suicide-attempt follow-up. Each call begins with an automatic answering system, followed by an in-terview between a psychiatric nurse and a patient. In some cases, a second psychiatric nurse also participates in the ex-change. Speech is spontaneous and conversational, with vari-able turn lengths, and high lexical diversity (reflecting the diver-sity of the life stories told). These conversations are explicitly focused on emotional assessment and therapeutic support, and the protocol prioritizes empathetic listening and patient elabo-ration.

### 2.1.2. Awake Neurosurgery (AN) dataset

This dataset includes audio recordings collected during preop-erative preparation for language testing in awake neurosurgery (AN). Conversations always involve a patient and at least one medical professional, such as a neurosurgeon, neuropsycholo-gist, psychologist, or anaesthesiologist. Additional participants, including a nurse or a patient's family member, may also be present. These interviews follow a format with clearer turn-taking organization than SP. When excluding SP's automated voice system, AN involves more speakers on average, and ex-hibits higher conversational density with shorter median turn duration and more frequent very short turns, reflecting the struc-tured question-answer dynamic typical of in person clinical as-sessments. The medical context being about brain tumor resec-tion, the consultation is explanatory while the conversation ob-jective is cognitive and linguistic assessment, following a stan-dardized neuropsychological protocol.

<table><caption>Table 1: Dataset statistics</caption><thead><tr><th></th><th>AN</th><th>SP</th></tr></thead><tbody><tr><td><strong>Dataset size</strong></td><td></td><td></td></tr><tr><td>Total duration</td><td>5h49m</td><td>6h32m</td></tr><tr><td>Total words</td><td>57,668</td><td>66,144</td></tr><tr><td>Conversations</td><td>10</td><td>18</td></tr><tr><td>Unique speakers (corpus)</td><td>15</td><td>13</td></tr><tr><td><strong>Conversational structure</strong></td><td></td><td></td></tr><tr><td>Speakers per conversation</td><td>2–4</td><td>2–3 *</td></tr><tr><td>Avg. duration (min)</td><td>35.0 ± 12.7</td><td>21.8 ± 5.1</td></tr><tr><td>Segments per minute</td><td>16.9 ± 4.2</td><td>14.7 ± 2.6</td></tr><tr><td>Median turn duration (s)</td><td>1.6 ± 0.6</td><td>2.2 ± 0.5</td></tr><tr><td>Type-token ratio</td><td>0.30 ± 0.07</td><td>0.49 ± 0.06</td></tr></tbody></table>

AN: Awake Neurosurgery, SP: Suicide Prevention

* does not include the automatic answering system

## 2.2. Pipeline

The initial transcription is performed using the WhisperX framework [9]. The large-v3 model is used, which offers a robust balance between accuracy and computational cost. To ensure precise alignment of transcription and diarization, the au-

dio is first processed with a VAD (Voice Activity Detection) fil-ter followed by the Whisper model to generate initial segments. A specialized French alignment model is applied to provide word-level timestamps for synchronizing speaker changes with the text. Speaker identification is performed using the Pyan-note 3.1 architecture [10]. This step assigns generic labels (e.g., SPEAKER_00, SPEAKER_01) to each segment. This output, obtained without any LLM-based correction or post-processing, is referred to as the Baseline during the rest of this paper.

This Baseline serves as the input for the LLM post-processing stage. In this context, a segment is defined as a struc-tured unit containing:

* Timestamps: The start and end times of the utterance.

* Speaker Label: The identity assigned by the diarization model.

* Textual Content: The transcribed speech corresponding to that specific turn.

To handle large-scale clinical interviews, a chunking strategy has been implemented. Files exceeding 500 segments are split into sequential chunks of 500 segments to fit the LLM's context limits. Each chunk is processed independently, and the results are concatenated in order. The pipeline is designed to be model-agnostic and supports both commercial APIs and open-source models.

All experiments were conducted on an NVIDIA RTX A5000 GPU (24 GB VRAM) with 64 GB system RAM. Real-Time Factor (RTF) measurements (Section 2.6.3) include only LLM inference time and exclude initial WhisperX/Pyannote processing (baseline RTF 0.09), allowing direct evaluation of post-processing computational overhead.

## 2.3. Proposed Method: N-pass Architecture

The proposed pipeline consists of sequential LLM processing passes using the Qwen3-Next-80B model (see Section 2.5.1) in zero-shot prompting (see Section 2.5.2). Passes alternate between SR and WR improvement steps. The optimal order and number of passes are empirically determined by ablation studies (Section 3).

1. **SR Improvement:** Maps generic speaker labels (SPEAKER_00, SPEAKER_01) to clinical roles (Patient, Neurosurgeon, etc.) based on conversational patterns and medical terminology.

2. **WR Improvement:** Corrects ASR errors, resolves homophones using the identified clinical context, and anonymizes personal names while preserving spontaneous speech markers.

3. **SR Refinement:** Re-evaluates speaker attributions using the corrected transcript to resolve ambiguities that were obscured by initial ASR errors.

4. **WR Refinement:** Re-evaluates lexical corrections using the refined speaker attributions to resolve remaining transcription ambiguities that were obscured by initial speaker errors.

This architecture is designed based on three key principles validated through systematic ablation studies presented in Section 3. All experiments use Qwen3-Next-80B, a recent large-scale open-weight model optimized for high-reasoning tasks [11].

## 2.4. Prompts

The performance of the LLM post-processing stage relies on specialized system prompts designed to provide the models with
terminology before speaker attribution. In the Three-pass architecture, this alternates between: Pass 1 (WR Improvement) → Pass 2 (SR Improvement) → Pass 3 (WR Refinement).

This comparison evaluates whether it is more effective to first identify speaker roles and then use this context to guide lexical correction (SR-led) or to first clean the transcript and then use the corrected text to improve speaker attribution (WR-led).

### 2.5.4. Iteration Depth

To identify the optimal number of SR-WR cycles before diminishing returns or instability, configurations ranging from one to seven passes are evaluated using Qwen80B in zero-shot SR-first mode.

The Single-pass configuration performs joint WR and SR in a single inference, testing whether Qwen80B can improve upon the baseline without iterative refinement. The Two-pass approach splits the task into sequential stages: an initial SR Pass maps speaker labels to clinical roles, followed by a WR Pass that influences this role information to guide lexical modifications. The Three-pass architecture adds a SR refinement stage, re-evaluating speaker attributions using the now-corrected transcript to resolve ambiguities that were obscured by initial ASR errors. Configurations with four to nine passes extend this alternating pattern, adding successive WR and SR stages to test whether further iteration continues to improve performance or introduces instability, until time taken to compute exceeds real audio time with Real-Time-Factor>1 (see Section 2.6.3).

## 2.6. Evaluation Metrics

### 2.6.1. Word Diarization Error Rate (WDER)

Given the dual objectives of accurate lexical transcription and reliable speaker attribution, Word Diarization Error Rate (WDER) is selected as the primary evaluation metric. WDER is a word-level speaker attribution metric that measures the proportion of words with incorrect speaker tags among aligned words. In our implementation [14, 15], lexical and speaker errors are combined as:

$$ WDER = WER + \frac{W_{\text{correct but wrong speaker}}}{N} \quad (1) $$

where

* $W_{correct but wrong speaker}$ = number of correctly transcribed words assigned to the wrong speaker,

* $N$ = total number of words in the reference transcript.

Unlike pure diarization metrics (DER) that penalize temporal boundary shifts regardless of semantic correctness, or pure transcription metrics (WER) that ignore speaker attribution, WDER captures both dimensions simultaneously. Moreover, LLMs can improve semantic speaker attribution while slightly altering temporal boundaries. WDER rewards correct word-level assignments even when segment boundaries are re-distributed based on conversational logic. Pure speaker-level WDER (e.g. proportion of misattributed words among aligned words) is also used in the literature [16, 17] but this version allows direct comparison with WER.

### 2.6.2. Format Errors

Files that cannot be parsed as valid output due to format violations are defined as “outliers” (also referred to as parsing failures).

### 2.6.3. Real-Time Factor (RTF)

The ratio of processing time to audio duration is called Real-Time Factor (RTF), and is used specifically in the iteration depth analysis (Section 3.4) to quantify the computational cost of multi-pass strategies. RTF is defined as:

$$ RTF = \frac{T_{\text{processing}}}{T_{\text{audio}}} \quad (2) $$

where $T_{processing}$ is the total wall-clock time required to process a recording and $T_{audio}$ is the duration of the corresponding audio file. An RTF below 1 indicates faster-than-real-time processing.

### 2.6.4. Statistical significance

Statistical significance is assessed using the Wilcoxon signed-rank test [18], a non-parametric test for paired comparisons. For each pairwise comparison (e.g., 3P-S vs Baseline), the WDER difference for each individual recording is computed. The test then evaluates whether the median of these paired differences is significantly different from zero, indicating that one configuration consistently outperforms the other across recordings. Comparisons are conducted separately for each dataset (AN and SP) due to their distinct conversational characteristics. Results are considered statistically significant when $p \le 0.05$.

## 3. Results

### 3.1. Model selection

To identify the optimal model for our application, three LLM candidates are first compared in a single-pass configuration where the model performs joint SR and WR in a single inference. Table 2 presents the comparative performance on both clinical datasets.

<table><caption>Table 2: Model comparison in single-pass configuration</caption><thead><tr><th rowspan="2">Model</th><th colspan="3">WDER</th><th rowspan="2">Outliers</th></tr><tr><th>AN</th><th>SP</th><th>Avg</th></tr></thead><tbody><tr><td>Baseline</td><td>43.03</td><td>42.45</td><td>42.74</td><td>0</td></tr><tr><td>GPT4omini</td><td>42.93</td><td>42.44</td><td>42.69</td><td>0</td></tr><tr><td>QwenVL</td><td>45.46</td><td>42.55</td><td>44.01</td><td>3</td></tr><tr><td>Qwen80B</td><td>43.02</td><td>42.44</td><td>42.73</td><td>0</td></tr></tbody></table>

Avg: arithmetic mean across both datasets.

### 3.2. Zero vs Few-shot prompting

The effect of augmenting prompts with in-context examples on performance and stability is evaluated with Qwen80B. Table 3 compares zero-shot and few-shot prompting strategies in a two-stage workflow (SR → WR).

<table><caption>Table 3: Zero-shot vs Few-shot prompting with Qwen80B</caption><thead><tr><th rowspan="2">Strategy</th><th colspan="3">WDER</th><th rowspan="2">Outliers</th></tr><tr><th>AN</th><th>SP</th><th>Avg</th></tr></thead><tbody><tr><td>Baseline</td><td>43.03</td><td>42.45</td><td>42.74</td><td>0</td></tr><tr><td>Zero-shot</td><td>43.76</td><td>39.96</td><td>41.86</td><td>0</td></tr><tr><td>Few-shot</td><td>42.12</td><td>40.44</td><td>41.28</td><td>1</td></tr></tbody></table>
### 3.3. SR vs WR-led prompting

The impact of pass ordering is evaluated to assess if SR should precede WR (as in our proposed method), or if lexical cleaning should occur first. Table 4 compares both orderings across two-pass and three-pass configurations. All configurations produced zero outliers.

<table><caption>Table 4: SR-led vs WR-led pass ordering with *Qwen80B*</caption><thead><tr><th rowspan="2">Strategy</th><th rowspan="2">N pass</th><th colspan="3">WDER</th><th rowspan="2">Outliers</th></tr><tr><th>AN</th><th>SP</th><th>Avg</th></tr></thead><tbody><tr><td>Baseline</td><td>0</td><td>43.03</td><td>42.45</td><td>42.74</td><td>0</td></tr><tr><td>SR-led</td><td>2</td><td>43.76</td><td>39.96</td><td>41.86</td><td>0</td></tr><tr><td>WR-led</td><td>2</td><td>44.62</td><td>39.72</td><td>42.17</td><td>0</td></tr><tr><td>SR-led</td><td>3</td><td>43.79</td><td>39.57</td><td>41.68</td><td>0</td></tr><tr><td>WR-led</td><td>3</td><td>43.96</td><td>41.04</td><td>42.50</td><td>0</td></tr></tbody></table>

### 3.4. Iteration depth

The optimal number of iterative refinement cycles is determined with Table 5, which evaluates configurations ranging from one to seven passes, measuring WDER improvement, output stability (outliers) and computational cost (Real-Time Factor).

<table><caption>Table 5: Iteration depth analysis with *Qwen80B*</caption><thead><tr><th rowspan="2">N pass</th><th colspan="3">WDER</th><th rowspan="2">Outliers</th><th rowspan="2">RTF</th></tr><tr><th>AN</th><th>SP</th><th>Avg</th></tr></thead><tbody><tr><td>0</td><td>43.03</td><td>42.45</td><td>42.74</td><td>0</td><td>0.09</td></tr><tr><td>1</td><td>43.02</td><td>42.44</td><td>42.73</td><td>0</td><td>0.12</td></tr><tr><td>2</td><td>43.76</td><td>39.96</td><td>41.86</td><td>0</td><td>0.22</td></tr><tr><td>3</td><td>43.79</td><td>39.57</td><td>41.68</td><td>0</td><td>0.32</td></tr><tr><td>4</td><td>43.96</td><td>39.33</td><td>41.65</td><td>0</td><td>0.45</td></tr><tr><td>5</td><td>44.38</td><td>40.75</td><td>42.57</td><td>1</td><td>0.57</td></tr><tr><td>6</td><td>43.32</td><td>39.76</td><td>41.54</td><td>0</td><td>0.68</td></tr><tr><td>7</td><td>42.65</td><td>39.89</td><td>41.27</td><td>0</td><td>0.78</td></tr><tr><td>8</td><td>43.94</td><td>39.73</td><td>41.84</td><td>0</td><td>0.89</td></tr><tr><td>9</td><td>44.12</td><td>40.81</td><td>42.47</td><td>0</td><td>1.04</td></tr></tbody></table>

RTF: Real-Time Factor

N pass: 0 corresponds to the Baseline.

### 3.5. Statistical significance

To quantify the robustness of our chosen method for our datasets, pairwise statistical comparisons are conducted between the Three-pass SR-led method (3P-SR) and all alternative configurations evaluated in the preceding ablation studies. Table 6 presents Wilcoxon signed-rank test results comparing 3P-S to baseline, single-pass models, alternative prompting strategies, alternative pass orderings, and alternative iteration depths. Negative ΔWDER values indicate improvements by the proposed method, and significant comparisons (p < 0.05) along with improvements are highlighted in bold.

### 3.6. Qualitative analysis

Beyond quantitative metrics, systematic improvements in clinical usability were observed in both datasets. The proposed 3P-S method presents semantic speaker labels replacing generic

<table><caption>Table 6: Statistical comparison of 3P-S vs alternatives.</caption><thead><tr><th rowspan="2">vs 3P-S</th><th colspan="2">AN</th><th colspan="2">SP</th></tr><tr><th>ΔWDER</th><th>p-value</th><th>ΔWDER</th><th>p-value</th></tr></thead><tbody><tr><td>Baseline</td><td>1.70</td><td>1.00</td><td><strong>-7.30*</strong></td><td><strong>0.02</strong></td></tr><tr><td>1P-S<br>(GPT4omini)</td><td>2.00</td><td>0.77</td><td><strong>-7.20*</strong></td><td><strong>0.02</strong></td></tr><tr><td>1P-S<br>(QwenVL)</td><td><strong>-2.20</strong></td><td>0.13</td><td><strong>-10.40*</strong></td><td><strong>0.02</strong></td></tr><tr><td>1P-S</td><td>1.80</td><td>1.00</td><td><strong>-7.30*</strong></td><td><strong>0.02</strong></td></tr><tr><td>2P-S</td><td>0.10</td><td>0.31</td><td><strong>-1.00</strong></td><td>0.40</td></tr><tr><td>2P-S-FS</td><td>3.80</td><td>0.16</td><td><strong>-2.20</strong></td><td>0.82</td></tr><tr><td>3P-W</td><td><strong>-0.40</strong></td><td>1.00</td><td><strong>-3.70</strong></td><td>0.44</td></tr><tr><td>4P-S</td><td><strong>-0.40</strong></td><td>0.69</td><td>0.60</td><td>0.11</td></tr><tr><td>5P-S</td><td>0.20</td><td>0.20</td><td><strong>-3.00</strong></td><td>0.51</td></tr><tr><td>6P-S</td><td>1.10</td><td>0.22</td><td><strong>-0.50</strong></td><td>0.83</td></tr><tr><td>7P-S</td><td>2.60</td><td>0.22</td><td><strong>-0.80</strong></td><td>0.61</td></tr><tr><td>8P-S</td><td><strong>-0.30</strong></td><td>0.94</td><td><strong>-0.40</strong></td><td>0.88</td></tr><tr><td>9P-S</td><td><strong>-0.80</strong></td><td>0.84</td><td><strong>-3.10</strong></td><td>0.28</td></tr></tbody></table>

$$ \text{nP-X} = \text{n passes with strategy X (S=SR-led, W=WR-led, FS=Few-Shot)}. $$

Models in parentheses indicate single-pass with that specific model; no parentheses = *Qwen80B*.

$$ \Delta WDER = \text{mean paired difference (negative} = 3P-S \text{ improvement)}. $$

* significant ($p \le 0.05$) improvements.

tags and preservation of anonymization tokens. Representative examples illustrate these improvements in Figure 1. Baseline outputs use generic SPEAKER_XX tags, while 3P-S provides role-specific labels and distinguishes neurosurgeon from family member (*Proche*), a critical distinction absent in Baseline's generic labels. 3P-S preserves anonymization tokens (*name*) that Baseline replaces with real heard names.

## 4. Discussion

### 4.1. Model selection

Table 2 presents the comparative performance of three LLM candidates in both clinical datasets. *QwenVL* shows systematic degradation relative to baseline on AN (WDER +2.43 points) and produces three unparseable outputs. This was expected, as *QwenVL* is optimized for vision-language tasks and the 8B parameter scale is insufficient for complex medical dialogue reasoning [19, 20]. This result demonstrates that compact models cannot replace larger instruction-tuned LLMs for medical transcription post-processing.

*GPT4omini* achieves improvements on both datasets (avg WDER reduction of 0.1% relative) with zero outliers, demonstrating robust baseline maintenance. However, its commercial nature limits deployment flexibility and incurs per-inference costs.

*Qwen80B* matches baseline performance within measurement precision (avg WDER reduction of 0.02% relative) while offering three decisive advantages: (1) open-weights licensing enabling local deployment without per-inference costs, (2) stability with zero outliers across both datasets, and (3) cross-domain consistency comparable to *GPT4omini* (0.58 WDER difference between datasets vs 0.49 for *GPT4omini*).

Based on these results, *Qwen80B* is selected for all subsequent experiments. While single-pass improvements are min-
Reference

34.44 - 37.66 Psychiatric nurse
je suis name l'infirmière

Baseline

34.44 - 37.66 SPEAKER_02
Je suis Sophie, l'infirmière.

3P-S

34.44 - 37.66 Psychiatric nurse
Je suis name, l'infirmière.

Reference

97.52 - 99.16 Anaesthesiologist
pas de problème de thyroïde
99.18 - 99.26 Patient
non
107.29 - 112.74 Anaesthesiologist
vous n'avez jamais eu de saignements anormaux d'hémorragies

Baseline

97.52 - 99.16 SPEAKER_01
Pas de problème de thyroïde ?
99.18 - 99.26 SPEAKER_01
Non.
107.29 - 113.52 SPEAKER_01
Vous n'avez jamais eu de saluements anormaux, d'hémorragie ?

3P-S

97.52 - 99.16 Anaesthesiologist
Pas de problème de thyroïde ?
99.18 - 99.26 Patient
Non.
107.29 - 113.52 Anaesthesiologist
Vous n'avez jamais eu de saignements anormaux, d'hémorragie ?

<caption>Figure 1: Representative examples of qualitative improvements.</caption>

imal, this establishes a stable foundation for iterative refinement strategies explored in subsequent sections. The comparable performance between GPT4omini and Qwen80B suggests that open-source models have reached sufficient capability for medical transcription post-processing tasks, provided that sufficient clinical context is given with structured prompting and iterative refinement.

4.2. Zero vs Few-shot prompting

Table 3 compares zero-shot and few-shot prompting strategies for Qwen80B in a Two-pass SR-led configuration. Few-shot prompting achieves lower average WDER (41.28 vs 41.86 for zero-shot, -1.4% relative improvement), with particularly strong gains in AN (WDER 42.12 vs 43.76, -3.7% relative).

However, this marginal improvement comes at the cost of one unparsable output (outlier) in the few-shot condition, representing a production stability risk. Although few-shot prompting can improve domain adaptation, it also risks introducing output format instability or over-fitting to specific conversational patterns [21].

Given that zero-shot already delivers substantial improve-

ments over baseline (avg WDER 41.86 vs 42.74, -2.1% relative) with perfect output parseability, the additional complexity and instability introduced by few-shot examples is not justified. Zero-shot prompting is therefore retained for the proposed method, prioritizing robustness for clinical deployment over marginal WDER gains.

4.3. SR vs WR-led prompting

Table 4 evaluates the impact of pass ordering across both Two-pass and Three-pass architectures. In the Two-pass setting, WR-led achieves marginally lower WDER on SP (39.72 vs 39.96), but this advantage reverses in the Three-pass setting where SR-led outperforms (39.57 vs 41.04, -3.6% relative).

More important, SR-led ordering demonstrates superior cross-dataset stability. The performance gap between datasets is 3.80 WDER points for 2P SR-led versus 4.90 for 2P WR-led. This pattern intensifies in Three-pass, where SR-led maintains 4.22 point gap versus 2.92 for WR-led, but the absolute performance on SP is substantially better (39.57 vs 41.04).

The advantage of SR-led likely comes from LLMs' limited impact on pure lexical correction. Models pre-trained on written text excel at speaker role attribution when provided with conversational structure, but struggle to outperform domain-tuned ASR on phonetic disambiguation without first understanding who is speaking.

SR-led ordering is therefore preferred, as it provides the best compromise between per-dataset performance and cross-domain generalization, achieving consistent improvements in SP while maintaining stability in AN.

4.4. Iteration depth

Table 5 reveals a non-monotonic relationship between iteration depth and performance. Single-pass processing maintains Baseline quality (avg WDER 42.73 vs 42.74) with minimal computational overhead (RTF 0.12 vs 0.09), confirming that Qwen80B's instruction-following capacity is sufficient for joint WR-SR but does not improve upon the pipeline without iterative refinement.

Two-pass and Three-pass configurations deliver consistent improvements on SP (WDER 39.96 and 39.57 respectively, vs baseline 42.45), with Three-pass achieving the best WDER on this dataset. The marginal gain from Four-pass (39.33) is offset by a 40% longer processing time (RTF 0.45 vs 0.32) for only 0.24 WDER reduction. This cost-benefit ratio worsens further at higher depths.

Beyond four passes, stability degrades: Five-pass produces one outlier and shows WDER regression on both datasets relative to Three-pass. Six-pass and Seven-pass show high variance, with Seven-pass achieving competitive average WDER (41.27) but at 2.4x baseline computational cost (RTF 0.78 vs 0.32 for Three-pass).

Three-pass represents the optimal balance for our datasets. It achieves 97.6% of Four-pass WDER performance (41.68 vs 41.65) while maintaining perfect output stability with zero outliers, and 29% lower computational cost (RTF 0.32 vs 0.45). Although Four-pass shows marginally better WDER on SP (39.33 vs 39.57), this 0.24-point difference is not statistically significant (p=0.11, Table 6) and does not justify the increased computational overhead for offline batch processing scenarios. For applications requiring maximum accuracy regardless of cost, Four-pass may be preferred.

These observations are further supported by statistical comparisons presented in the following section.
## 4.5. Statistical significance

Table 6 presents pairwise statistical comparisons of the proposed Three-pass SR-led method (3P-S) compared to alternative configurations. Given the exploratory nature of these comparisons across multiple configurations, uncorrected p-values are reported. Borderline significant results ($0.01 < p < 0.05$) should be interpreted with appropriate caution. On SP, 3P-S achieves statistically significant improvements ($p < 0.05$) with Wilcoxon signed-rank test) over baseline (mean ΔWDER = -7.30 points) and all Single-pass configurations regardless of model choice (GPT4omini: -7.20, GwenVL: -10.40, Gwen80B: -7.30).

SP conversations involve higher lexical diversity (type-token ratio 0.49 vs 0.30) than the more structured AN consultations. This likely makes SP more sensitive to SR refinement, as LLMs can leverage contextual emotional cues and conversational dynamics to disambiguate speaker attribution. In contrast, AN's more formal consultation structure provides clearer speaker boundaries even in the baseline, leaving less room for improvement through iterative refinement.

Moreover, no configuration shows statistically significant degradation relative to 3P-S on either dataset, confirming that the proposed method does not introduce regressions. In AN, 3P-S maintains performance within statistical equivalence of Baseline (ΔWDER = +1.70, $p = 1.00$), indicating dataset-specific effectiveness rather than universal improvement.

Comparisons between 3P-S and other multi-pass depths (2P, 4P, 5P, 6P, 7P) show no significant differences (all $p > 0.10$), suggesting that beyond the initial two-pass threshold, performance gains stabilize within the statistical noise of our sample size. The lack of significance for 4P-S vs 3P-S (ΔWDER = +0.60 on SP, $p = 0.11$) reinforces that the marginal 0.60-point improvement observed in Table 5 does not justify the 40% increase in computational cost (RTF 0.45 vs 0.32).

The statistically significant improvements on SP, combined with stability on AN and zero production failures (outliers), support the adoption of 3P-S for clinical deployment across both our conversational contexts, with larger gains expected in emotionally charged interactions with higher lexical diversity.

## 5. Perspectives

Several research directions could extend this work. Recent work suggests that Chain-of-Thought (CoT) prompting, where the LLM is instructed to explicitly verbalize its reasoning process before making corrections, can improve transcription accuracy on English medical consultations [7]. Integrating CoT reasoning into the SR pass could help the model justify speaker role assignments based on conversational cues, potentially reducing attribution errors in ambiguous cases.

The combination of outputs from multiple LLMs by majority vote or confidence-weighted averaging can reduce individual model hallucinations and improve robustness [22]. Applying ensemble techniques to the proposed pipeline could reduce outlier risk and improve performance on both datasets simultaneously.

A key limitation of this study is the modest sample size (10 AN and 18 SP conversations), which limits statistical power and generalizability claims. Future work should validate the proposed architecture on larger corpora across additional French medical applications (e.g., emergency medicine, general practice) to assess robustness beyond the two domains studied here. Multi-site validation would also help establish whether the observed dataset-specific effects (improvements on SP but not AN) reflect fundamental differences in conversation structure or artifacts of our specific recording conditions.

Future work should assess whether improved speaker attribution reduces manual correction effort for clinical linguists. A question that remains open is also whether better transcriptions improve automatic classification performance in tasks related to datasets, such as suicide risk assessment or surgical outcome prediction. Moreover, tighter ASR-LLM integration, such as feeding n-best hypotheses to enable contextual re-ranking, could further improve correction quality, with clinical utility as the ultimate validation criterion.

## 6. Conclusion

This work proposes a N-pass LLM post-processing architecture for French clinical interview transcription, validated through systematic ablation studies on two distinct clinical datasets: suicide prevention telephone counseling and preoperative awake neurosurgery consultations.

Large-scale open-source models (Qwen3-Next-80B, 80B parameters) match commercial API performance while enabling local deployment, addressing privacy and cost constraints critical for clinical applications. This approach proposes a reproducible framework for other under-resourced languages in medical context, reducing dependence on English-centric commercial platforms. Moreover, zero-shot prompting ensures production stability (zero output failures), SR-led ordering provides superior cross-dataset generalization, and three iterative passes represent the optimal balance between accuracy gains and computational cost. The proposed method achieves statistically significant improvements in word-level speaker attribution in emotionally charged conversations (relative WDER reduction of 6.8%, $p < 0.05$ on SP) while maintaining stability in structured medical consultations.

These results are achieved with acceptable computational cost (RTF 0.32, approximately a third of real-time) and high output stability, suggesting feasibility for offline clinical deployment in conversation analysis, clinical documentation, and research applications requiring speaker-attributed transcriptions. Clinical validation with end-users and integration into existing workflows remain necessary steps before operational deployment.

Future work should explore advanced prompting strategies (chain-of-thought reasoning), LLM ensemble methods, and tighter ASR-LLM integration to further improve robustness and extend applicability to real-time scenarios and various clinical contexts.

## 7. Acknowledgments

This research was approved by an Ethic Committee and the institutional review boards of participating hospitals. All participants provided informed consent for their conversations to be used for research purposes. Audio recordings were pseudonymized prior to processing, and all personally identifiable information was replaced with generic tokens in the final transcripts.

This work was funded by the Brittany Region (France) through the doctoral program "Allocations de Recherche Doctorale" (ARED).

Ambre Marie designed the experiments, conducted the analyses, and drafted the manuscript. Thomas Bertin created the reference transcriptions. Gwenolé Quellec, Guillaume Dard-
enne and Thomas Bertin supervised the research and critically revised the manuscript.

## 8. References

[1] T. Bertin and G. Quellec, "Transcription automatique des interactions verbales. limites observées et perspectives envisagées à partir d'un corpus de consultations médicales," *Corpus*, no. 26, 2025.

[2] M. V. van Buchem, H. Boosman, M. P. Bauer, I. Kant, S. Cammel, and E. Steyerberg, "The digital scribe in clinical practice: a scoping review and research agenda," *NPJ Digital Medicine*, vol. 4, 2021.

[3] A. Renato, D. Luna, and S. Benitez, "Development of an asr system for medical conversations," *Studies in health technology and informatics*, vol. 310, pp. 664–668, 2024.

[4] J. Linke, J. Winkler, and B. Schuppler, "Context is all you need? low-resource conversational asr profits from context, coming from the same or from the other speaker," in *Interspeech 2025*, 2025.

[5] H. B. Pasandi and H. B. Pasandi, "Evaluation of asr systems for conversational speech: A linguistic perspective," *Proceedings of the 20th ACM Conference on Embedded Networked Sensor Systems*, 2022.

[6] T. Reitmaier, E. Wallington, D. Raju, O. Klejch, J. Pearson, M. Jones, P. Bell, and S. Robinson, "Opportunities and challenges of automatic speech recognition systems for low-resource language speakers," *Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems*, 2022.

[7] A. Adedeji, S. Joshi, and B. Doohan, "The sound of healthcare: Improving medical transcription asr accuracy with large language models," *arXiv preprint arXiv:2402.07658*, 2024.

[8] I. Groupe, "Convention icor," *Lyon: université de Lyon*. URL: http://icar.cnrs.fr/projets/corinte/documents/2013_Conv_ICOR_250313.pdf, 2013.

[9] M. Bain, J. Huh, T. Han, and A. Zisserman, "Whisperx: Time-accurate speech transcription of long-form audio," *INTER-SPEECH 2023*, 2023.

[10] H. Bredin, R. Yin, J. M. Coria, G. Gelly, P. Korshunov, M. Lavechin, D. Fustes, H. Titeux, W. Bouaziz, and M.-P. Gill, "Pyannote. audio: neural building blocks for speaker diarization," in *ICASSP 2020-2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)*. IEEE, 2020, pp. 7124–7128.

[11] Q. Team, "Qwen3 technical report," 2025. [Online]. Available: https://arxiv.org/abs/2505.09388

[12] N. Cummins, S. Scherer, J. Krajewski, S. Schnieder, J. Epps, and T. F. Quatieri, "A review of depression and suicide risk assessment using speech analysis," *Speech Communication*, vol. 71, pp. 10–49, 2015.

[13] A. Marie, M. Garnier, T. Bertin, L. Machart, G. Dardenne, G. Quellec, and S. Berrouiguet, "Acoustic and machine learning methods for speech-based suicide risk assessment: A systematic review," *arXiv preprint arXiv:2505.18195*, 2025.

[14] L. E. Shafey, H. Soltau, and I. Shafran, "Joint speech recognition and speaker diarization via sequence transduction," pp. 396–400, 2019.

[15] B. D. Tran, M. Tai-Seale, R. Mangu, J. Lafata, and K. Zheng, "Automatic speech recognition performance for digital scribes: a performance comparison between general-purpose and specialized models tuned for patient-clinician conversations," *AMIA ... Annual Symposium proceedings. AMIA Symposium*, vol. 2022, pp. 1072–1080, 2022.

[16] R. Paturi, S. Srinivasan, and X. Li, "Lexical speaker error correction: Leveraging language models for speaker diarization error correction," pp. 3567–3571, 2023.

[17] R. Paturi, X. Li, and S. Srinivasan, "Ag-Isec: Audio grounded lexical speaker error correction," *ArXiv*, vol. abs/2406.17266, 2024.

[18] R. F. Woolson, "Wilcoxon signed-rank test," *Wiley encyclopedia of clinical trials*, pp. 1–3, 2007.

[19] J. Bai, S. Bai, S. Yang, S. Wang, S. Tan, P. Wang, J. Lin, C. Zhou, and J. Zhou, "Qwen-vl: A frontier large vision-language model with versatile abilities," *ArXiv*, vol. abs/2308.12966, 2023.

[20] K. Singhal, S. Azizi, T. Tu, S. Mahdavi, J. Wei, H. W. Chung, N. Scales, A. Tanwani, H. Cole-Lewis, S. Pfohl, P. Payne, M. G. Seneviratne, P. Gamble, C. Kelly, N. Scharli, A. Chowdhery, P. A. Mansfield, B. A. Y. Arcas, D. Webster, G. S. Corrado, Y. Matias, K. Chou, J. Gottweis, N. Tomašev, Y. Liu, A. Rajkomar, J. Barral, C. Semturs, A. Karthikesalingam, and V. Natarajan, "Large language models encode clinical knowledge," *Nature*, vol. 620, pp. 172–180, 2022.

[21] T. Zhao, E. Wallace, S. Feng, D. Klein, and S. Singh, "Calibrate before use: Improving few-shot performance of language models," pp. 12 697–12 706, 2021.

[22] G. Efstathiadis, V. Yadav, and A. Abbas, "Llm-based speaker diarization correction: A generalizable approach," *ArXiv*, vol. abs/2406.04927, 2024.