# SIMSAMU - A French medical dispatch dialog open dataset

Aimé Nun, Olivier Birot, Gaël Guibon, Frédéric Lapostolle, Ivan Lerner

► To cite this version:

Aimé Nun, Olivier Birot, Gaël Guibon, Frédéric Lapostolle, Ivan Lerner. SIMSAMU - A French medical dispatch dialog open dataset. Computer Methods and Programs in Biomedicine, 2025, 268, pp.108857. 10.1016/j.cmpb.2025.108857. hal-05325611

HAL Id: hal-05325611

https://inria.hal.science/hal-05325611v1

Submitted on 22 Oct 2025

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# SIMSAMU - A French medical dispatch dialog open dataset

Aimé Nun<sup>a,b,c,*</sup> <sup> ID </sup>, Olivier Birot<sup>a,b</sup>, Gaël Guibon<sup>d,g ID</sup>, Frédéric Lapostolle<sup>e,f</sup>,
Ivan Lerner<sup>a,b,c ID</sup>

## ARTICLE INFO

**Keywords:**

Communication

Medical dialogues

Medical informatics computing

Cluster analysis

Dialogue modeling

Medical emergencies dispatch

## ABSTRACT

**Background:** Dispatch Services (DS) are essential to Emergency Medical Services (EMS). Dispatchers enable patients to access medical assistance in emergencies, anytime and anywhere, within limited time and resources. Al-based decision-support tools hold great promise for dispatchers. Developing these tools requires medical field-specific data. Medical dispatch dialogue is unique: it is a brief phone exchange in an emergency, within a limited time frame, without a physical examination.

**Objective:** Our main objective was to (i) create an open French dataset of medical dispatch dialogues. Our sec-ondary objectives were to (ii) develop a detailed medical dispatch scheme from this dataset using an unsuper-vised method, and (iii) provide a baseline evaluation of diarization and speech recognition models for this domain in French.

**Methods:** From 2022 to 2023, emergency medicine junior doctors simulated real-life medical dispatch calls. These calls were recorded and transcribed to form the SIMSAMU corpus. We developed a dispatch scheme based on (i) recording analysis, (ii) data-driven utterance typology, and (iii) domain expertise. Utterance typology was derived via hierarchical clustering of representations learned by finetuning BERT embeddings on SIMSAMU. Clusters were mapped to the Roter Interaction Analysis System (RIAS) and included in our dispatch scheme. SIMSAMU was used to train and evaluate state-of-the-art neural network models for diarization and speech recognition. Diarization used the PyaNet model, fine-tuned on the ESLO2 dataset. Speech recognition used a CTC model with pre-trained wav2vec 2.0 embedding, compared to the multilingual Whisper model. The CTC-wav2vec model was further fine-tuned on SIMSAMU and evaluated by leave-one-speaker-out cross-validation.

**Results:** The dataset consists of 61 audio recordings totaling 3 h 14 min. Four clusters were identified for callers and 3 for dispatchers. Two main dialogue phases were identified: interrogation and contractualization. The diarization model achieved a 10.4 % error rate. Speech recognition word error rates were 35.8 % for Whisper, 24.8 % for the CTC-wav2vec model fine-tuned on ESLO2, and 16.1 % after in-domain fine-tuning.

**Conclusion:** We propose a French open medical dispatch dialogue dataset and an expert-validated schema of the medical dispatch dialogue based on unsupervised analysis. Notable gaps in how well speech recognition models generalize underscore the need for targeted, in-domain fine-tuning in this specialized application. SIMSAMU is designed to support this effort by serving as a benchmark for evaluating domain-adapted speech recognition and dialogue modeling strategies.

## 1. Introduction

Dispatch Services (DS) are an integral part of Emergency Medical

Services (EMS). The role of dispatchers is to respond to a specific need: enable patients to have access to medical assistance in critical emergency situations, at any time and in any place, within a limited time and

<footnote><sup>a</sup> Université Paris Cité, Inserm, Centre de Recherche des Cordeliers, Sorbonne Université, Paris F-75006, France</footnote>

<footnote><sup>b</sup> HeKA, Inria Paris, Paris F-75012, France</footnote>

<footnote><sup>c</sup> Assistance Publique Hôpitaux de Paris (AP-HP), Department of Medical Informatics, Georges Pompidou European Hospital, Paris, France</footnote>

<footnote><sup>d</sup> LORIA, Université de Lorraine, CNRS, 54600, France</footnote>

<footnote><sup>e</sup> SAMU 93, UF Recherche-Enseignement-Qualité, Avicenne Hospital-APHP, Bobigny, France</footnote>

<footnote><sup>f</sup> Université Paris 13, INSERM Unit 942, Sorbonne Paris Cité, Bobigny, France</footnote>

<footnote><sup>g</sup> Université Sorbonne Paris Nord, CNRS, Laboratoire d'Informatique de Paris Nord, LIPN, Villetaneuse F-93430, France</footnote>

<footnote>* Corresponding author.
E-mail address: aime.nun14@gmail.com (A. Nun).</footnote>
with limited resources [1]. In France, medical regulation involves a systematic medical assessment carried out as quickly as possible by phone by an emergency doctor, which is a specific feature of the French system [2]. Then one or more diagnostic and prognostic hypotheses and prognosis are established, and the risks of the situation are assessed in relation to the expected benefits of the possible course of action: simple medical advice, first aid assistance, prescription of medication, medical intervention with the patient, referral, and transport of the patient. Medical dispatch aims to prescribe the right care at the right time [3]. In other countries such as the United States and the United Kingdom, DS are insured by paramedics [4].

According to a 2020 report in France: 105 EMS responded to 40,000 calls a day, and 456 MICU (Mobile Intensive Care Unit) performed 787,000 interventions. In terms of decisions, medical advice was given in 36 % of cases, victim assistance, and rescue vehicles were dispatched in 32 % of cases, a private ambulance in 20 % of cases, a home visit from a doctor in 8 % of cases, and a MICU in 4 % of cases [5].

As DS represents substantial activity across the world, it requires an accurate dispatching system to maintain patient safety. However, the overall level of evidence for the accuracy of medical dispatching systems is very low to low [1]. Decision-support tools based on Artificial Intelligence (AI) hold great promises for DS. Specific tools for certain emergencies, such as strokes in Denmark are reported to increase stroke detection by 61 % [6]. Decision support tools for cardiac arrest show heterogeneous results, with no significant change in the detection of cardiovascular events [7,8]. In Spain, a decision support tool for grading the severity of the situation, based on deep learning, has been developed with satisfying performance (12.5 % increase in the detection of life-threatening situations) [9]. A decision support tool based on simple questions improved the sensitivity of acute coronary syndrome detection from 82.6 % to 90.2 % among patients calling for chest pain [10].

The development of AI-based decision support tools requires the collection of data specific to the medical field. Several previous studies have proposed open datasets with medical dialogues: consultations, calls, and simulations [11–13].

However, medical dispatch dialogue has its specific features: it's a short exchange by telephone, in an emergency situation, within a limited time frame, and without a physical clinical examination [3]. To our knowledge, there are no AI-based decision-support tools for medical dispatch in France. No French medical dispatch dataset has yet been published to develop this type of tool which can help Medical Dispatchers (MD) make the optimal decision by predicting the right type of response at the right time [14].

Our main objective was to (i) create an open French dataset of medical dispatch dialogues. Our secondary objectives were to (ii) develop a detailed medical dispatch scheme from this dataset using an unsupervised method, and (iii) provide a baseline evaluation of diarization and speech recognition models for this domain in French.

## 2. Materials and methods

### 2.1. Data source

The dataset consists of audio files recorded during the training of junior doctors in emergency medicine, designed to enhance their communication skills. The training involved simulating medical dispatch calls, with junior doctors alternating between the roles of the caller and the medical dispatcher (MD). Each scenario was pre-defined and randomly assigned using a custom card system defining: (i) the caller's communication style, (ii) the relationship between the caller and the patient, (iii) the patient's age, and (iv) the reason for the call (Table 1) (Supplementary S1) [15]. With this card system, the total number of possible scenarios is up to 54.131.

* **Caller's communication style:** Seven communication (7 cards, corresponding to the seven dwarfs from Snow White®) modes were

<table><caption>**Table 1**
Card deck used to determine call simulation scenarios.</caption><thead><tr><th>Categories</th><th>Cards</th><th>Content</th></tr></thead><tbody><tr><td>Caller's role</td><td>19</td><td>for himself/herself, his/her father/mother (2 cards), his/her brother/sister (2 cards), his/her son/daughter (2 cards), his/her spouse/husband (2 cards), his/her grandfather/grandmother (2 cards), his/her friend (2 cards*), his/her colleague (2 cards), his/her neighbor (2 cards*) & a stranger in the street (2 cards*) (* one card for a male, one for a female patient)</td></tr><tr><td>Communication style</td><td>7</td><td>shy, procedural, angry, detached, frightened, impassive, incomprehensible</td></tr><tr><td>Patient's age</td><td>11</td><td>6 months, 3 years, 10 years, 21 years, 32 years, 43 years, 54 years, 65 years, 76 years, 87 years & 98 years</td></tr><tr><td>Reason for the call</td><td>37</td><td>Abdominal pain, assaulted, bladed weapon injury, burn, chest pain, childbirth, confusion, deterioration of general condition, difficulty in maintaining home care, dizziness, electric shock, fall from standing height, fall from the second floor, fever, fire on the third floor, flu-like syndrome, hanging, headaches, hematemesis, household product poisoning, loss of consciousness, malaise, medical device vigilance, motorcycle accident, neurological deficit, no longer answering his phone, palpitations / tachycardia, poisoning, request involuntary hospitalization, seizures, shortness of breath, suicidal ideation, tetany, vomiting, 3 blank cards</td></tr><tr><td>Total</td><td>74</td><td>Possible combinations: 54.131</td></tr></tbody></table>

defined: shy (shy), procedural (doc), angry (grumpy), detached (happy), frightened (bashful), impassive (sneezy), incomprehensible (dopey). These modes, reflective of problematic communication that can affect emergency regulation, were specifically chosen, excluding healthcare professionals, as their communication was considered similar to other callers.

* **Caller's role:** Fourteen cards. This set categorized the caller's relationship with the patient (e.g., spouse, friend, sibling, parent, bystander). Although the literature lacks precise data on the distribution of caller-patient relationships, this categorization was based on practical emergency communication considerations.

* **Patient's age:** Eleven cards. The age was categorized by decade, ranging from newborns to 100 years old, to ensure diverse age representation in the simulations.

* **Reason for the call:** Twenty-one cards. In addition to common emergency reasons (e.g., road accidents, chest pain), rare or exceptional cases, such as fires or hangings, were included to introduce specific challenges in emergency regulation.

In each simulation, five key pieces of information were collected: the patient's age and gender, the caller's relationship with the patient (from a list of 13 options), the caller's communication characteristics (the seven-communication mode, "normal" communication, or other), and the reason for the call. Table 2

Before these simulations, participants underwent theoretical communication training, which included listening to emergency call recordings and discussing key strategies such as empathy, reformulation, and effective questioning. During simulations, roles were assigned randomly and blind to the MD participants, while the simulating students were isolated, and their peers observed the conversations in real time. The MD's voice was recorded directly through a microphone, while the caller's voice was transmitted through the phone network and re-emitted by a speaker before being recorded, mimicking the real-world call setup.

This dataset was generated through a bicentric study conducted at CHU Avicenne, involving 5 junior doctors in the first round and 7 in the second, as well as CHU La Réunion, which included 5 junior doctors, bringing the total to 17 junior doctors.
<table><caption>**Table 2**
Example of an audio transcript translated from French.</caption><tr><td>Medical Doctor (MD):</td><td>hello madam, I'm doctor French the regulating doctor uh your daughter called us because you have chest pain</td></tr><tr><td>Caller (C):</td><td>that's right</td></tr><tr><td>MD:</td><td>which started fifteen or twenty minutes ago, um, do you have any health problems?</td></tr><tr><td>C:</td><td>you know, a bit of cholesterol, a bit of hypertension, that's it, just basic stuff.</td></tr><tr><td>MD:</td><td>Have you ever had a heart attack?</td></tr><tr><td>C:</td><td>I don't think so. What's a heart attack?</td></tr></table>

## 2.2. Audio transcription

The audio files were transcribed by two annotators. The transcription guide for this project was based on two guides produced for two other studies: the ESTER2 corpus (Evaluation des Systèmes de Transcription Enrichie d'Émissions Radiophoniques 2), transcription of radio and television broadcasts and ESLO2 corpus (Enquêtes SocioLinguistiques à Orléans 2), which consisted in transcribing recordings of street interviews or daily conversations on a variety of subjects with residents of the city of Orléans [16,17]. The main rules of the transcription guide (S.1) were to be as close as possible to what was said, not to use capital letters at the beginning and end of utterances or sentences, except for proper nouns; dictionary's non-existing words should be constantly transcribed with the same orthography and with the same spelling; each segment should not exceed 15 s. The transcription was done by AN according to the guide, then it was proofread by OB and the guide was modified according to the transcription errors.

## 2.3. Unsupervised description

To extract embeddings of our sentences, we used a transfer learning method using the CamemBERT language model with 110 M parameters [18]. The choice of using CamemBERT model rather than BioBERT or DrBERT is justified by the need for a generalist model adapted to dialogue but not specifically to medical topics as the goal is to extract clusters of utterances with an unsupervised method [19,20].

Due to limited data, we reduced the model's output dimensions to avoid overfitting by adding a pooling layer. This pooling extracts the value of the first token, typically the CLS token, reducing the input from 3D (batch, sequence, dimension) to 2D (batch, dimension). Originating from the BERT model, this "pooler" retrieves the CLS token's value, unlike other pooling methods like Average or MaxPooling that reduce all tokens. In our model, multiple CLS tokens maintain the 3D structure, but the first layer reduces the 768-dimensional space to 128 through learned matrix multiplication and normalization.

$$ \text{Pooling} = \tanh(W.h + b) $$

For each training session, we used the *huggingface* trainer with a Masked Language Modeling (MLM) task with fixed token masks. MLM involves masking tokens at random. The model must then predict which token best fits the masked ones. We compared two different fine-tuning procedures: fine-tuning the full model and only the last layers of the model. To obtain the optimal hyperparameters that minimize the validation dataset cross-entropy loss, we used the Optuna tuner with the following parameters: learning rate, batch size, pooler dimension, and fine-tuning procedure [21]. Then we selected the model with the lowest perplexity where:

$$ \text{Perplexity} = \exp(\text{loss}) $$

The clustering of utterances aimed at identifying the speech acts. We then extracted the embeddings of each CLS token (which represents the entire turn of speech) and performed a two-level hierarchical clustering, to have clusters of sufficient size according to the dendrograms. For each first-level cluster, we assigned a label corresponding to a RIAS (Roter Interaction Analysis System) category. This label was chosen based on sentences corresponding to one of the categories in each cluster. Clustering was performed separately for MDs and patients/callers (referred

to as "callers" in the following sections of the document) (Fig. 1). Each cluster was then described and visualized in the space of utterances projected in 2 dimensions using the UMAP method (Uniform Manifold Approximation and Projection) [22]. Then we used TF-IDF (Term Frequency * Inverse Document Frequency) to describe clusters with terms specific to each cluster. Utterances are translated from French to English for the reader in the main figures and tables of the manuscript.

## 2.4. Medical dispatch scheme

Based on the unsupervised description results, we proposed to a group of medical dispatch experts and emergency physicians a dialog scheme specific to medical dispatch dialog. They compared it to their daily experiences and validated it based on clinical insights.

## 2.5. Automatic diarization and speech recognition models

To perform diarization, we use the pyannote-audio library [23,24], which relies on the PyaNet model and agglomerative clustering. A full recording is split into overlapping chunks of 5 s, and the PyaNet model performs multi-speaker voice activity detection on each chunk. This gives us local speaker segments, but identical speakers may have been assigned different labels by the PyaNet model across different chunks. To identify the speakers, numerical representations (ECAPA-TDNN embeddings) are computed for each segment and regrouped using agglomerative clustering [25]. We used the pre-trained models of Pyannote 3.0 and fine-tuned them on the ESLO2 dataset (preprocessed to match the acoustic characteristics of recorded phone calls). For the speech recognition task, we use a wav2vec-CTC (Connectionist Temporal Classification) model implemented by the SpeechBrain library [26]. In the wav2vec model, the audio signal is encoded into frames of 25 milliseconds through temporal convolutions, and for each frame, a contextualized embedding is computed through a transformer architecture [27]. This is then fed into a feed-forward-CTC neural network that outputs the most likely token for each time frame. We use a pre-trained wav2vec model and train the layers of the CTC network using the CommonVoice dataset as well as the preprocessed ESLO2 dataset and the PxSLU dataset [28,29]. We also use a KenLM language model to improve the final output. We compared the performance of these models with Whisper (large V2), a multilingual model [30,31].

## 2.6. Ethics

The junior doctors involved in creating the dataset were fully informed about the study and its aims. They were assured that refusing to participate would not affect their training. They were then asked to sign a consent form indicating their willingness to participate in the data collection. The study was approved by the local ethics committee of the Avicenne Hospital with the number CLEA-2024-n°384.

# 3. Results

## 3.1. Dataset description

The dataset is available at the following URL: https://hf.co/datasets/medkit/simsamu. It is composed of 61 audio recordings with a median duration of 3 min 9 s, a minimum of 58 s, and a maximum of 7 min 23 s.
<figure><img src="image_2.png" /><figcaption>**Fig. 1.** Model architecture</figcaption></figure>

Note: * 128 dimensions was the most optimal pooler (see Supplementary S.3).

The total duration of the recordings is 3 h 14 min and 28 s. Audio files have an 8 KHz sample rate and a 128 Kbps bitrate, which is like actual dispatch services call recordings. It has a total of 2997 utterances, 1484 turns for the caller, and 1593 for the MD. For the caller, this represents 17,623 words, and for the MD, 17,536 words. For the caller, the shortest turn is 1 word, and the longest is 84 words, with an average of 12 words per turn. For the MD, the shortest turn is 1 word, and the longest is 61 words, with an average of 11 words per turn (Fig. 2).

According to the scenario definition methodology, the card statistics involved for each case are described in the following table (Fig. 3).

## 3.2. Unsupervised description

### 3.2.1. Hierarchical clustering

Perplexity on our dataset was 89.74 before training and 6.77 after training for our best model. Training details for the other models can be found in Supplementary S3. For the caller utterances, 4 clusters were identified: *backchannel*,<fnref n="1" /> *questions*, *informative*, and *persuasion*. In the MD utterances, two clusters referred to backchannel, thus they were merged. 3 clusters were identified: backchannel, questions, and advice/orders. The utterances with the highest average number of words were *persuasion* for the caller and *directions* for the MD (Fig. 3, Table 4).

Note: *Agglomerative clustering was used, the high-level threshold was set to 10 clusters to have enough granularity and information. The low-level threshold was determined qualitatively according to RIAS labels.*

To visualize the different clusters and their proximities, we have used a UMAP representation for each corpus: MD and caller (Supplementary SF1). An interactive version of the UMAP figures is also available online [32,33].

### 3.2.2. TF-IDF

We describe each cluster by extracting specific terms using TF-IDF of bigrams. The results are displayed in Table 3.

## 3.3. Medical dispatch scheme

Based on the transcriptions, a certain pattern of medical dispatch dialogues was repeated for each exchange. The dialogue could be divided into two phases: medical interrogation and contractualization. The medical interrogation consists of asking medical or situation questions (Fig. 5) and answering the caller's questions and requests. If the

caller is different from the patient, the MD will systematically ask to speak directly to the patient, so there may be a change of interlocutor. This phase ends with the MD recapitulating the situation. Then, in the contractualization phase, the MD issues a decision (as described in the figures), and usually, a negotiation between the MD and the caller takes place, with a request for clarification or even a request for a different course of treatment. Fig. 4

This negotiation may last several cycles before a common agreement is reached and the call is ended (Fig. 5).

## 3.4. Experiments: automatic diarization and speech recognition

The DER (Diarization Error Rate) metric corresponds to the proportion of wrongly labeled audio segments (wrong voice activity status or wrong speaker). The fine-tuned Pyannote pipeline gave a DER of 10.4 % with a margin of ± 0.25 s before and after each segment. For speech recognition, the WER (Word Error Rate) metric obtained on our dataset using the trained CTC model is 24.8 %, (and 16.6 % for the CER, Character Error Rate). These results are visible in Table 4. Furthermore, fine-tuning our CTC model on our dataset using “leave-one speaker-out” cross-validation gave an average WER of 16.1 %. The fine-tuned diarization result pipeline and the trained speech recognition models are available at the following URL: https://huggingface.co/medkit

# 4. Discussion

## 4.1. Dataset and experiments

In this study, we developed the SIMSAMU dataset which comprises 61 audio recordings, each offering a glimpse into the dynamics of medical dispatch conversations. These recordings cover approximately two-thirds of real emergency scenarios [15], from brief exchanges to more detailed dialogues lasting up to 7 min and 23 s. Leveraging unsupervised representation of utterances, we found distinct types of functions for callers and MD utterances, which we summarized in a concise medical dispatch scheme. Finally, we were able to provide the first diarization and speech recognition model evaluation in this domain. Our analysis showed that the generalization gap was large, from 24.8 % to 16.1 % WER after in-domain fine-tuning, supporting the importance of collecting data in this specific domain.

A detailed analysis of the dataset reveals a total of 2997 utterances, with callers contributing 1484 turns and MDs contributing 1593 turns. Interestingly, the average word count per turn for both callers and MDs hovers around 12 and 11 words, respectively, highlighting the brevity and efficiency characteristic of emergency communications. Unexpectedly, callers and MD both contributed equally to the length of the call,

<footnote n="1">¹ Backchannel = phatic: refers to the function of language when it is not used to communicate a message, but to maintain contact between the speaker and the addressee, for example: allò, ouais, eh bien.</footnote>
<figure><img src="image_3.png" /></figure>

<figure><img src="image_4.png" /><figcaption>Fig. 2. Dataset details.</figcaption></figure>

both in terms of turns and number of words.

Employing unsupervised clustering techniques, we identified distinct utterance clusters within both the caller and MD corpus. We used this static analysis of utterances (Fig. 3, Table 4) to support our qualitative analysis, conducted by medical experts (AN, FL, IL), of the dynamics of dispatch dialogues, grounding it in previous work from the general medical domain (RIAS). We identified a structured pattern underlying medical dispatch dialogues, characterized by two distinct phases: medical interrogation and contractualization. Medical interrogation phase involves addressing the caller's queries and concerns, often necessitating a direct conversation with the caller to ascertain relevant medical information. Subsequently, the contractualization phase entails the MD issuing a decision, initiating a negotiation process with the caller, and iteratively refining the course of action until a consensus is reached. This iterative negotiation highlights the nuanced decision-making process inherent in emergency medical dispatch scenarios, emphasizing the

importance of effective communication in reaching optimal outcomes within constrained time frames. This formalization of medical dispatch dialogue should be put in perspective with practices in other countries if the analysis is replicated. Table 5

Our study allowed the evaluation of state-of-the-art diarization and speech recognition models in this specific domain in French. We found a Diarization Error Rate (DER) of 10.4%, when the pipeline was evaluated with a margin of ± 0.25 s before and after each segment. We found a Word Error Rate (WER) of 24.8% for our CTC-based model fine-tuned on ESLO2, a French open dialogue corpus. Interestingly, this model surpassed the multilingual state-of-the-art Whisper model (large V2) [29]. Whisper was found at a WER of 35.8% on our dataset, whereas it was reported at 13.9% on CommonVoice. Finally, fine-tuning the CTC-based model on SIMSAMU, which required estimating the WER through cross-validation due to the small dataset size, showed an average WER of 16.1% across folds. Together, the combination of rare
<figure><img src="image_5.png" /><figcaption>Fig. 3. Card statistics.</figcaption></figure>

<table><caption>**Table 3**
TF-IDF of caller and MD cluster bigrams.</caption><thead><tr><th>Cluster caller</th><th>Example (french)</th><th>Example (english)</th></tr></thead><tbody><tr><td>backchannel</td><td>oui allo bonjour ?</td><td>yes allo?</td></tr><tr><td>information</td><td>j'en peux plus là</td><td>I can't take it anymore</td></tr><tr><td>giving</td><td>elle bouge pas son bras</td><td>she won't move her arm</td></tr><tr><td>persuasion</td><td>il faut faire quelque chose là</td><td>we have to do something</td></tr><tr><td></td><td>je suis très très inquiète hein</td><td>I'm very, very worried</td></tr><tr><td></td><td>il faut l'amener à l'hôpital</td><td>we've got to bring him to</td></tr><tr><td></td><td>c'est pas la première fois</td><td>hospital</td></tr><tr><td></td><td>il faut envoyer quelqu'un tout de suite</td><td>it's not the first time</td></tr><tr><td>question asking</td><td>je sais pas quoi</td><td>we have to send someone</td></tr><tr><td></td><td>j'en sais rien, sais rien</td><td>right away</td></tr><tr><td></td><td>et peut être qu'on va l'amener à l'hôpital ?</td><td>I don't know what</td></tr><tr><td>Cluster MD</td><td>Example (french)</td><td>I don't know, I know</td></tr><tr><td>backchannel</td><td>merci, au revoir</td><td>nothing</td></tr><tr><td>question asking</td><td>passez la moi s'il vous plait</td><td>and maybe we'll take him to the hospital?</td></tr><tr><td></td><td>elle a des problèmes de santé</td><td>Example (english)</td></tr><tr><td></td><td>votre mère?</td><td>thank you. bye</td></tr><tr><td></td><td>c'est la première fois qu'il ne se réveille pas?</td><td>please pass it to me</td></tr><tr><td></td><td>depuis combien de temps?</td><td>does your mother have</td></tr><tr><td></td><td>elle prend des médicaments?</td><td>health problems?</td></tr><tr><td>directions</td><td>on va en- envoyer une ambulance</td><td>is this the first time he hasn't woken up?</td></tr><tr><td></td><td>on va envoyer les pompiers</td><td>since how?</td></tr><tr><td></td><td>emmenez le heu mettre son bras sous l'eau</td><td>does she take medication?</td></tr><tr><td></td><td></td><td>we'll send an ambulance</td></tr><tr><td></td><td></td><td>we'll send the firemen</td></tr><tr><td></td><td></td><td>take him to put his arm under water</td></tr></tbody></table>

medical terms, overlapping speech, and stressful environment makes it difficult to reach good performance without in-domain finetuning. Although the dataset is too small to support a fully supervised approach, this highlights the importance of building such datasets to enable the evaluation of generalist models or domain adaptation strategies in zero-shot or few-shot learning settings within this specific domain.

## 4.2. Comparison with other medical dialog datasets

SIMSAMU is closely comparable to the open medical dialogue datasets of Fareez et al. and PriMock57 in that they all include audio and ground truth transcripts and are similar in size, 272 and 57 dialogues respectively, compared to 61 for SIMSAMU [12,37]. SIMSAMU dialogues are shorter (3 min 2 s average duration) compared to Fareez et al. (11 min 57 average duration) and Primock57 (9 min 5 s average duration), consistent with the fact that SIMSAMU simulates emergency dialogues, whereas Fareez et al. simulate a medical consultation (mainly for respiratory diseases) and Primock57 a primary care consultation. Our estimate of WER without in-domain fine-tuning is consistent, slightly higher for Whisper (35.8 %) or lower for Wav2vec2 with CTC (24.8 %) than the evaluation of Google Cloud Speech-to-text (30.9 %) on Primock57, the best model evaluated at the time of the study (2022).

Further away from SIMSAMU, MTS-DIALOG is a transcript-only dataset of 1700 dialogues and dialogue summaries that simulate consultations in general practice and some medical specialties and are scripted based on real clinical notes taken by healthcare professionals [38].

From a design point of view, SIMSAMU differs in that it uses a custom
<figure><img src="image_6.png" /><figcaption>Fig. 4. Agglomerative clustering of [CLS] token embeddings details.</figcaption></figure>

card system to assign communication modes, patient details and reasons for calls, ensuring variability in the simulation content. But more fundamentally, it differs in that the primary objective of the simulation is not the constitution of the dataset, but the training of the doctors, which is driving the medical content of the simulated dialogues to be close to real dialogues.

Finally, SIMSAMU differs fundamentally from the MedDialog dataset [13], which is based on real online forum conversations in English and Chinese. The English part consists of 257,332 English "dialogues", but with only one turn, corresponding to question-answer pairs. The Chinese part of the dataset includes dialogues with more than one turn but are significantly shorter (3.3 utterances per dialogue on average) than ours (49.1 utterances per dialogue on average).

### 4.3. Limitations, strengths and perspective

Our study is not without limitations. Firstly, the dataset's random selection among seven communication modes introduces bias towards situations that are more challenging for doctors to manage. However, this minimally impacts conversation content, instead aiding in pinpointing areas that prolong discussions. In addition, the recordings made using phone recorders, particularly the caller parts, may exhibit reduced intelligibility due to phone speaker usage. Despite these limitations, the data generated from these simulations have proven representative of real emergency calls, with physicians and emergency responders often unable to distinguish between real and simulated calls during post-simulation debriefings. The dataset's exchange lengths closely mirror real-world scenarios, aligning with findings from a French study on MD-caller interactions [34]. Moreover, the dataset captures
<figure><img src="image_7.png" /><figcaption>Fig. 5. Emergency medical dispatch dialog scheme.</figcaption></figure>

<table><caption>**Table 4**

Performance scores on word error rate and character error rate metrics.</caption><thead><tr><td>Model</td><td>WER</td><td>CER</td></tr></thead><tbody><tr><td>Whisper</td><td>35.8 %</td><td>27.8 %</td></tr><tr><td>CTC</td><td>24.8 %</td><td>16.6 %</td></tr><tr><td>CTC fine-tuned</td><td>16.1 %</td><td>10.7 %</td></tr></tbody></table>

Note: Lower is better.

genuine emergency conditions, including hesitations and utterances. While our dataset size is smaller compared to other datasets, such as MedDialog, the one-in-a-million study, or a Canadian simulated patient-physician medical interviews dataset, it offers comprehensive transcription and coverage of diverse clinical scenarios. Furthermore, MedDialog is composed of short textual chats with a mean of 6–8 turns while our dataset is composed of dialog with a mean of 50 turns [11,13].

Regarding our medical dispatch scheme based on unsupervised clustering, the transfer learning method employed CamembERT, a generalist model dedicated to French. While more specialized models like BioBERT or, for French, Dr-BERT exist, they are not adapted for dialog structure [19,20]. Finally, we provide metrics (WER, DER) without uncertainty estimates, and thus caution their interpretation

<table><caption>**Table 5**

Transcription example of medical dispatch dialogs using the CTC and Whisper models.</caption><thead><tr><th></th><th>Example</th></tr></thead><tbody><tr><td><strong>Original</strong></td><td>pas de vomissements est ce qu'il y a eu des diarrhées</td></tr><tr><td>Whisper</td><td>pas de vomissements est ce qu'il y a eu des diarrhées</td></tr><tr><td>CTC</td><td>pas de vomissement est ce qu'il y a eu des dare</td></tr><tr><td>CTC fine-tuned</td><td>pas de vomissements est ce qu'il y a eu des diarrhées</td></tr><tr><td><strong>Original</strong></td><td>restez avec votre collègue prenez si vous avez un doliprane pour l'instant pour la douleur</td></tr><tr><td>Whisper</td><td>restez avec votre collègue prenez un doliprane pour la douleur</td></tr><tr><td>CTC</td><td>vous restez avec nos collègues vous prenez si vous avez un doliprane pour l'instant pour la boule</td></tr><tr><td>CTC fine-tuned</td><td>vous restez avec votre collègue prenez si vous avez un doliprane pour l'instant pour la douleur</td></tr><tr><td><strong>Original</strong></td><td>d accord ça c est c est ça saigne beaucoup ça saigne encore là</td></tr><tr><td>Whisper</td><td>d accord ça s aime beaucoup ça se saine encore là</td></tr><tr><td>CTC</td><td>d accord ça c est c est ça aime beaucoup ça se fait encore là</td></tr><tr><td>CTC fine-tuned</td><td>d accord ça c est c est ça saigne beaucoup ça se saigne encore là</td></tr></tbody></table>
when the difference is small. Furthermore, evaluating speech-to-text
models revealed differences in WER with Whisper with default
prompt, but prompt engineering is likely to improve these results.

Despite the quantitative balance in exchange frequencies between
callers and MDs, the chronology of dialogue differs, as evidenced by our
medical dispatch scheme. This scheme is a valuable reference for future
research aiming at developing task-specific medical dialog systems.
Indeed, employing the RIAS system for medical dialogue analysis un-
derscores the scheme’s applicability, given its wide use in various health
dialogue contexts, ensuring consideration of medical dialogue speci-
ficity and socioemotional aspects [35,36]. Moreover, leveraging auto-
matic clustering techniques on larger datasets could enable finer
dialogue act categorization, facilitating the identification of communi-
cation "slots" and informing the development of AI-assisted tools to
guide emergency medical decision-making. Robust and precise speech
recognition models in this domain could greatly benefit from the large
number of recordings of medical dispatch calls, which are to be stored
for 10 years by law in France, to learn domain-adapted unsupervised
representations of the audio signal.

5. Conclusion

We introduce SIMSAMU, an open French dataset of medical dispatch
dialogues, accompanied by an expert-validated dialogue scheme derived
from unsupervised analysis. The construction of SIMSAMU required a
substantial and rigorous effort: we curated, recorded, and manually
transcribed three hours of paired speech-text data across 61 simulated
dialogues, encompassing a broad range of realistic emergency scenarios.
High-quality transcription was ensured through dual independent
annotation, and we additionally provide a publicly available diarization
pipeline and speech models tailored to this complex domain.

Building upon this dataset, we conducted a comprehensive analysis of dispatch dialogues, identifying and formalizing their underlying structure into two distinct phases: medical interrogation and contractualization. This scheme, validated by medical experts, captures the specific dynamics of emergency medical communication and establishes a robust foundation for the development of task-specific dialogue systems.

Moreover, SIMSAMU enabled the first benchmarking of state-of-the-
art diarization and speech recognition systems on French emergency
medical data, revealing significant performance gaps between generalist
and domain-adapted models. These findings underscore the critical
importance of collecting and curating specialized datasets to advance
zero-shot and few-shot learning strategies in healthcare-specific speech
processing.

By systematically addressing the particularities of emergency call
interactions — including the brevity, high emotional load, overlapping
speech, and domain-specific vocabulary — SIMSAMU provides a unique
and valuable resource for the research community. We anticipate that
SIMSAMU will significantly contribute to the development of more
robust, adaptive, and context-aware medical dialogue systems, and will
serve as a reference point for future work in this field.

Ethics statement

The junior doctors involved in creating the dataset were fully
informed about the study and its aims. They were assured that refusing
to participate would not affect their training. They were then asked to
sign a consent form indicating their willingness to participate in the data
collection. The study was approved by the local ethics committee of the
Avicenne Hospital with the number CLEA-2024-n°384.

CRediT authorship contribution statement

Aimé Nun: Writing – review & editing, Writing – original draft,
Software, Methodology, Investigation, Formal analysis, Data curation,

Conceptualization. **Olivier Birot:** Writing – original draft, Software, Methodology, Investigation, Formal analysis, Data curation. **Gaël Gui-bon:** Writing – review & editing, Supervision, Resources, Methodology. **Frédéric Lapostolle:** Methodology, Investigation, Data curation. **Ivan Lerner:** Writing – review & editing, Writing – original draft, Validation, Supervision, Methodology, Formal analysis, Conceptualization.

Declaration of competing interest

AN, OB, GG, FL and IL have no conflicts of interest relating to this
research to declare.

Acknowledgments

The authors extend their gratitude to all the Junior Doctors who
agreed to participate in the formation of the datasets.

Funding

This work was supported by the LLM4All project, funded by The French National Research Agency (ANR) (Reference: ANR-23- IAS1-0008).

**Supplementary materials**

Supplementary material associated with this article can be found, in
the online version, at doi:10.1016/j.cmpb.2025.108857.

References

[1] K. Bohm, L. Kurland, The accuracy of medical dispatch - a systematic review, Scand. J. Trauma Resusc. Emerg. Med. 26 (1) (2018) 94, 9 nov.

[2] F. Adnet, F. Lapostolle, International EMS systems: France, Resuscitation 63 (1) (2004) 7–9, oct.

[3] M. Giroud, La régulation médicale en médecine d'urgence, Réanimation 18 (8) (2009) 737-741, déc.

[4] Challenging the “Scoop and run” model for management of out-of-hospital cardiac arrest | resuscitation | JAMA | JAMA Network [Internet]. [cité 29 nov 2023]. Disponible sur: https://jamanetwork.com/journals/jama/article-abstract/2770601?casa_token=8gbOi2DfT4UAAAAA:LPPa3KbX7Mj2nYzSTKgqijb6PHre3TRzTn7N2CvpDVp-u7lzQMPyd9RaeUhwatoBgFdomlFtGZ-C.

[5] Haute Autorité de Santé [Internet]. [cité 30 oct 2023]. Mettre en oeuvre la certification pour la qualité des soins. Disponible sur: https://www.has-sante.fr/jcms/r_1495044/fr/mettre-en-oeuvre-la-certification-pour-la-qualite-des-soins.

[6] M.L. Scholz, H. Collatz-Christensen, S.N. Blomberg, S. Boebel, J. Verhoeven, T. Krafft, Artificial intelligence in Emergency Medical Services dispatching: assessing the potential impact of an automatic speech recognition software on stroke detection taking the Capital Region of Denmark as case in point, Scand. J. Trauma Resusc. Emerg. Med. 30 (1) (2022) 36, 12 mai.

[7] S.N. Blomberg, H.C. Christensen, F. Lippert, A.K. Ersbøll, C. Torp-Petersen, M. R. Sayre, et al., Effect of machine learning on dispatcher recognition of out-of-hospital cardiac arrest during calls to emergency medical services, JAMA Netw. Open. 4 (1) (2021) e2032320, 6 janv.

[8] K.C. Chin, T.C. Hsieh, W.C. Chiang, Y.C. Chien, J.T. Sun, H.Y. Lin, et al., Early recognition of a caller's emotion in out-of-hospital cardiac arrest dispatching: an artificial intelligence approach, Resuscitation 167 (2021) 144-150, 1 oct.

[9] P. Ferri, C. Sáez, A. Félix-De Castro, J. Juan-Albarracín, V. Blanes-Selva, P. Sánchez-Cuesta, et al., Deep ensemble multitask classification of emergency medical call incidents combining multimodal data improves emergency medical dispatch, Artif. Intell. Med. 117 (2021) 102088 juill.

[10] M. Gellerstedt, N. Rawshani, J. Herlitz, A. Bâng, C. Gelang, J.O. Andersson, et al., Could prioritisation by emergency medicine dispatchers be improved by using computer-based decision support? A cohort of patients with chest pain, Int. J. Cardiol. 220 (2016) 734-738, 1 oct.

[11] M. Jepson, C. Salisbury, M.J. Ridd, C. Metcalfe, L. Garside, R.K. Barnes, The 'one in a million' study: creating a database of UK primary care consultations, Br. J. Gen. Pract. Mai. 67 (658) (2017) e345-e351.

[12] F. Fareez, T. Parikh, C. Wavell, S. Shahab, M. Chevalier, S. Good, et al., A dataset of simulated patient-physician medical interviews with a focus on respiratory cases, Sci. Data 9 (1) (2022) 313, 16 juin.

[13] G. Zeng, W. Yang, Z. Ju, Y. Yang, S. Wang, R. Zhang, et al., MedDialog: large-scale medical dialogue datasets, in: Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP) [Internet], 2020, pp. 9241–9250. Online: Association for Computational Linguistics[cité 6 févr 2023]Disponible sur, https://www.aclweb.org/anthology/2020.emnlp-main.743.

[14] T.M. Olasveengen, C. Hardeland, Man vs. machine? The future of emergency medical dispatching, Resuscitation 138 (2019) 304–305, 1 mai.
[15] J.A. Vieira, M. Bensoussan, S. Bard, A. Bompas, F. Nomede-Martyr, A. Martin, et al., Validation of a set of cards for the simulation of EMS calls. POKER study, Ann. Fr Médecine D'urgence. 14 (3) (2024) 149-156, juin.

[16] C. Brun, M. Ehrmann, Un système de détection d'entités nommées adapté pour la campagne d'évaluation ESTER 2. Actes De La 17e Conférence Sur Le Traitement Automatique Des Langues Naturelles Articles courts [Internet]. Montréal, ATALA, Canada, 2010 [cité 23 févr 2023]. p. 117-22. Disponible sur, https://aclanthology.org/2010.jeptalnrecital-court.20.

[17] Flamein H., Eshkol-Taravella I. Exploitation du corpus enquêtes sociolinguistiques à Orléans (ESLO) par les outils du traitement automatique des langues et de la géomatique. Humanit Numér [Internet]. 1 mai 2021 [cité 23 févr 2023];(3). Disponible sur: https://journals.openedition.org/revuehn/1911.

[18] L. Martin, B. Muller, P.J.O. Suárez, Y. Dupont, L. Romary, É.V. de la Clergerie, et al., CamemBERT: a tasty French language model, in: Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, 2020.

[19] J. Lee, W. Yoon, S. Kim, D. Kim, S. Kim, C.H. So, et al., BioBERT: a pre-trained biomedical language representation model for biomedical text mining, Bioinformatics 36 (4) (2020) 1234-1240, 15 févr.

[20] Y. Labrak, A. Bazoge, R. Dufour, M. Rouvier, E. Morin, B. Daille, et al., DrBERT: a robust pre-trained model in French for biomedical and clinical domains [Internet], Bioinformatics (2023) avr [cité 23 mai 2023]. Disponible sur, http://biorxiv.org/lookup/doi/10.1101/2023.04.03.535368.

[21] T. Akiba, S. Sano, T. Yanase, T. Ohta, M.O. Koyama, A next-generation hyperparameter Optimization Framework [Internet]. arXiv [cité 8 juin 2023]. Disponible sur, http://arxiv.org/abs/1907.10902, 2019.

[22] L. McInnes, J. Healy, J. Melville, UMAP: uniform Manifold Approximation and Projection for dimension reduction [Internet]. ArXiv [cité 9 oct 2023]. Disponible sur, http://arxiv.org/abs/1802.03426, 2020.

[23] H. Bredin, R. Yin, J.M. Coria, G. Gelly, P. Korshunov, M. Lavechin, et al., pyannote. Audio: neural building blocks for speaker diarization, in: ICASSP 2020, IEEE International Conference on Acoustics, Speech, and Signal Processing, 2020.

[24] H. Bredin, A. Laurent, End-to-end speaker segmentation for overlap-aware resegmentation, Proc. Interspeech (2021) 2021.

[25] Desplanques B., Thienpondt J., Demuynck K. ECAPA-TDNN: emphasized channel attention, propagation and aggregation in TDNN based speaker verification. In: Interspeech 2020 [Internet]. 2020 [cité 29 nov 2023]. p. 3830-4. Disponible sur: http://arxiv.org/abs/2005.07143.

[26] A. Graves, S. Fernández, F. Gomez, J. Schmidhuber, Connectionist temporal classification: labelling unsegmented sequence data with recurrent neural networks, in: Proceedings of the 23rd international conference on Machine learning [Internet], New York, NY, USA, Association for Computing Machinery,

2006, https://doi.org/10.1145/1143844.1143891 [cité 4 oct 2024]. p. 369-76. (ICML '06). Disponible sur.

[27] A. Baevski, Y. Zhou, A. Mohamed, M. Auli, wav2vec 2.0: a framework for self-supervised learning of speech representations. Advances in Neural Information Processing Systems, Curran Associates, Inc., 2020 [cité 29 nov 2023]. p. 12449-60. Disponible sur, https://proceedings.neurips.cc/paper/2020/hash/92d1e1eb1cd6f9fba3227870bb6d7f07-Abstract.html.

[28] R. Ardila, M. Branson, K. Davis, M. Henretty, M. Kohler, J. Meyer, et al., Common voice: a massively-multilingual speech corpus [Internet]. ArXiv [cité 29 nov 2023]. Disponible sur, http://arxiv.org/abs/1912.06670, 2020.

[29] A.C. Kocabiyikoglu, F. Portet, P. Gibert, H. Blanchon, J.M. Babouchkine, G. Gavazzi, A spoken drug prescription dataset in French for spoken language understanding [Internet]. ArXiv [cité 29 nov 2023]. Disponible sur, http://arxiv.org/abs/2207.08292, 2022.

[30] K. Heafield, KenLM: faster and smaller language model queries, in: C. Callison-Burch, P. Koehn, C. Monz, O.F. Zaidan (Eds.), Proceedings of the Sixth Workshop on Statistical Machine Translation [Internet], Edinburgh, Scotland, Association for Computational Linguistics, 2011 [cité 29 nov 2023]. p. 187-97. Disponible sur, https://aclanthology.org/W11-2123.

[31] A. Radford, J.W. Kim, T. Xu, G. Brockman, C. McLeavey, I. Sutskever, Robust speech recognition via large-scale weak supervision [Internet], arXiv, http://arxiv.org/abs/2212.04356, 2022.

[32] UMAP representation - Medecins [Internet]. [cité 10 avr 2024]. Disponible sur: https://nanyshoko.github.io/SIMSAMU-MED/.

[33] UMAP representation - Patients [Internet]. [cité 10 avr 2024]. Disponible sur: https://nanyshoko.github.io/SIMSAMU-PAT/.

[34] Bensoussan M., Lapostolle F., Reuter P.G. Facteurs impactant la durée de communication dans un EMCC. (under review). 2024.

[35] D. Roter, S. Larson, The Roter interaction analysis system (RIAS): utility and flexibility for analysis of medical interactions, Patient Educ. Couns. (2002).

[36] M. Sandvik, H. Eide, M. Lind, P.K. Graugaard, J. Torper, A. Finset, Analyzing medical dialogues: strength and weakness of Roter's interaction analysis system (RIAS), Patient Educ. Couns. 46 (4) (2002) 235-241, avr.

[37] A.P. Korfiatis, F. Moramarco, R. Sarac, A. Savkov, PriMock57: a dataset of primary care mock consultations [Internet], arXiv, http://arxiv.org/abs/2204.00333, 2022.

[38] A. Ben Abacha, W.W. Yim, Y. Fan, T. Lin, An empirical study of clinical note generation from doctor-patient encounters, in: Vlachos A, Augenstein I, éditeurs. Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics [Internet], Dubrovnik, Croatia, Association for Computational Linguistics, 2023 [cité 13 janv 2025]. p. 2291-302. Disponible sur, https://aclanthology.org/2023.eacl-main.168/.