# Vibravox: A Dataset of French Speech Captured with Body-conduction Audio Sensors

Julien Hauret, Malo Olivier, Thomas Joubaud, Christophe Langrenne, Sarah

Poirée, Véronique Zimpfer, Éric Bavu

► To cite this version:

Julien Hauret, Malo Olivier, Thomas Joubaud, Christophe Langrenne, Sarah Poirée, et al. Vibravox: A Dataset of French Speech Captured with Body-conduction Audio Sensors. Speech Communication, 2025, 172, pp.103238. 10.1016/j.specom.2025.103238. hal-05069756

HAL Id: hal-05069756

https://hal.science/hal-05069756v1

Submitted on 15 May 2025

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# Vibravox: A dataset of french speech captured with body-conduction audio sensors

Julien Hauret<sup>a, ID, *</sup>, Malo Olivier<sup>a, ID</sup>, Thomas Joubaud<sup>b, ID</sup>, Christophe Langrenne<sup>a</sup>, Sarah Poirée<sup>a</sup>, Véronique Zimpfer<sup>b, ID</sup>, Éric Bavu<sup>a, ID</sup>

## ARTICLE INFO

Dataset link: https://huggingface.co/datasets/Cnam-LMSSC/vibravox

**Keywords:**

Body-Conduction audio sensors

Robust Communication

Speech enhancement

Speech recognition

Speaker verification

## ABSTRACT

Vibravox is a dataset compliant with the General Data Protection Regulation (GDPR) containing audio recordings using five different body-conduction audio sensors: two in-ear microphones, two bone conduction vibration pickups, and a laryngophone. The dataset also includes audio data from an airborne microphone used as a reference. The Vibravox corpus contains 45 h per sensor of speech samples and physiological sounds recorded by 188 participants under different acoustic conditions imposed by a high order ambisonics 3D spatializer. Annotations about the recording conditions and linguistic transcriptions are also included in the corpus. We conducted a series of experiments on various speech-related tasks, including speech recognition, speech enhancement, and speaker verification. These experiments were carried out using state-of-the-art models to evaluate and compare their performances on signals captured by the different audio sensors offered by the Vibravox dataset, with the aim of gaining a better grasp of their individual characteristics.

## 1. Introduction

Unlike traditional microphones, which rely on airborne sound waves, body-conduction audio sensors—often referred to as body conduction microphones (BCMs) for simplicity—allow voice signals to be picked up directly from the body, offering potential advantages in noisy environments by greatly reducing the influence of ambient noise on recordings. Although BCMs have been available for decades (Graciarena et al., 2003; Zheng et al., 2003; Bos et al., 2005; Acker-Mills et al., 2005; Shahina and Yegnanarayana, 2007), the limited bandwidth of speech captured by such sensors has so far restricted their widespread use. However, thanks to two tracks of improvements, this technology could be made available to a wider audience for speech capture and communication in noisy environments.

Research into physics and electronics is improving with some skin-attachable sensors such as (Lee et al., 2019, 2022; Che et al., 2024). Similarly to earlier bone and throat microphones, these new wearable sensors detect skin vibration, which is highly and linearly correlated with the acoustic pressure produced by the wearer's voice (Björklund and Sundberg, 2016). Not only do they improve on the state of the art by having superior sensitivity over the vocal frequency range—thereby improving the signal-to-noise ratio—but they also have superior skin

compliance, which facilitates adhesion to curved skin surfaces. These new kinds of sensors, just like the previous ones, are, however, unable to capture the full bandwidth of the speech signal due to the inherent low-pass filtering of tissues. The manufacturing process also needs to be stabilized, which currently prevents them from being commercially available. In addition, some earbuds, such as the Xiaomi Buds 4 Pro, already include some voice pickup unit thanks to a retro-action microphone integrated into the loudspeaker at the entrance to the ear canal. Finally, the innovative Aria Gen 2 Meta's research platform now integrates a contact microphone. This exemplifies the significant role that BCMs will likely assume in the future.

Deep learning methods have shown excellent performance in a variety of speech and audio tasks. In this work, we show that the Vibravox dataset can be used to advance research in three key tasks that would enable overcoming the current limitations of body conduction sensors: bandwidth extension, speech recognition, and speaker verification. Since body conduction sensors exhibit reduced bandwidth transduction efficiency, works such as (Zhang et al., 2021; Han and Lee, 2022; Serrà et al., 2022; Mandel et al., 2023; Nisticò et al., 2023; Shuai et al., 2023; Hauret et al., 2023; Hauret et al., 2023; Karthikeyan et al., 2023; Andreev et al., 2023; Li et al., 2023a; He et al., 2023;

<footnote><sup>a</sup> Laboratoire de Mécanique des Structures et des Systèmes Couplés, Conservatoire national des arts et métiers, HESAM Université, 75003 Paris, France</footnote>

<footnote><sup>b</sup> Department of Acoustics and Soldier Protection, French-German Research Institute of Saint-Louis (ISL), France</footnote>

<footnote>* Corresponding author.</footnote>

<footnote>E-mail addresses: julien.hauret@lecnam.net (J. Hauret), malo.olivier@lecnam.net (M. Olivier), thomas.joubaud@isl.eu (T. Joubaud), christophe.langrenne@lecnam.net (C. Langrenne), sarah.poirée@lecnam.net (S. Poirée), veronique.zimpfer@isl.eu (V. Zimpfer), eric.bavu@lecnam.net (É. Bavu).</footnote>

<footnote>https://doi.org/10.1016/j.specom.2025.103238</footnote>

<footnote>Received 20 August 2024; Received in revised form 26 March 2025; Accepted 4 April 2025</footnote>

<footnote>Available online 19 April 2025</footnote>

<footnote>0167-6393/© 2025 The Authors. Published by Elsevier B.V. This is an open access article under the CC BY license (http://creativecommons.org/licenses/by/4.0/).</footnote>
<table><caption>**Table 1**
Open source BCM Dataset review.</caption><thead><tr><th>Dataset</th><th>Number of speakers</th><th>Number of BCM</th><th>Clean speech</th><th>Noisy speech</th><th>Speechless recordings</th><th>Language</th><th>Text transcript</th><th>Phonetic transcript</th><th>Download page</th></tr></thead><tbody><tr><td>ABCS (Wang et al., 2022a)</td><td>100</td><td>1</td><td>2 x 42 h</td><td>0 h</td><td>0 h</td><td>Chinese</td><td>✓</td><td>✗</td><td>GitHub</td></tr><tr><td>ESMB (ESMB-corpus, 2021)</td><td>287</td><td>1</td><td>2 x 128 h</td><td>0 h</td><td>0 h</td><td>Chinese</td><td>✗</td><td>✗</td><td>GitHub</td></tr><tr><td>EmoBone (Hosain et al., 2024)</td><td>28</td><td>1</td><td>2 x 19 h</td><td>0 h</td><td>0 h</td><td>English</td><td>✓</td><td>✗</td><td>Avail. upon request</td></tr><tr><td>TAPS (Kim et al., 2025)</td><td>60</td><td>1</td><td>2 x 13 h 18</td><td>0 h</td><td>0 h</td><td>Korean</td><td>✓</td><td>✗</td><td>HuggingFace</td></tr><tr><td>Vibravox (ours) (Hauret et al., 2024)</td><td>188</td><td>5</td><td>6 x 33 h 18</td><td>6 x 1 h 58</td><td>6 x 10 h 21</td><td>French</td><td>✓</td><td>✓</td><td>HuggingFace</td></tr></tbody></table>

Li et al., 2023b; Edraki et al., 2024; Ohlenbusch et al., 2024a; Sui et al., 2024; Yu et al., 2024; Scheibler et al., 2024; Liu et al., 2024) demonstrate the ability of deep learning approaches to regenerate mid and high frequencies from low-frequency audio content. They have adapted recent deep learning trends to the specific problem of bandwidth extension, also known as audio super-resolution. For robust speech recognition, models such as Whisper (Radford et al., 2023) or Canary1B (NVIDIA, 2024) have pushed the limits of usable signals. Finally, for speaker verification, approaches such as TitaNet (Koluguri et al., 2022), WavLM (Chen et al., 2022), Pyannote (Bredin, 2023), and ECAPA2 (Thienpondt and Demuynck, 2023) now allow a wide range of signals to be used thanks to their increased robustness.

The availability of large datasets is critical to advancing research and development of BCMs. These datasets, allowing the training and evaluation of deep learning models, have been a key missing ingredient in achieving high-quality, intelligible speech with such audio sensors. The assumption that airborne and body-transmitted speech share identical excitation sources and possess a simple transfer function between them is inadequate; producing realistic synthetic data is therefore still a challenging task. In a previous study, we performed out-of-distribution tests performed on real BCM signals with an EBEN model trained on simulated data (Hauret et al., 2023), showing the importance of using a sufficiently large set of real data for training. Despite their importance, existing datasets still exhibit several notable limitations. Data collection being labor-intensive, the existing BCM datasets often lack the necessary scale and diversity to comprehensively cover the full range of acoustic scenarios encountered in real-world applications. Another limitation is the absence of publicly available BCM datasets in French, restricting research primarily to the languages covered by existing resources. Problems also persist with signal quality, including noise and artifacts. On top of these issues, the variety of sensors used in existing datasets remains limited, making it difficult to generalize results across different recording environments and equipment configurations. The Vibravox dataset is being made available to fill these gaps and stimulate research in the field of speech capture using non-conventional audio sensors.

In this paper, we contribute along two major axes: the construction and utilization of the dataset. First, we introduce the Vibravox dataset, developed through a rigorous data collection process. We designed and implemented a recording protocol, assembled the necessary hardware, and built dedicated software to ensure a controlled and high-quality data acquisition setup. A total of 200 participants contributed to the dataset, with 188 retained after post-processing to ensure high-quality data. We applied post-filtering techniques to enhance data quality and facilitate interpretability, enabling meaningful comparisons between different microphones and insightful speech signal analyses. Second, we establish baselines using the Vibravox dataset for three key speech processing tasks: bandwidth extension, speech-to-phonemes recognition, and speaker verification. For each task, we conduct experiments in both quiet and noisy environments to comprehensively evaluate the strengths and limitations of body-conduction microphones under realistic conditions. For speech-to-phonemes recognition and speaker verification, we compare the performance of raw and enhanced body-conduction sensors in quiet conditions to evaluate the impact of speech enhancement on downstream applications. These contributions provide

new insights into the potential and challenges of BCMs for speech capture and processing, paving the way for future advancements in the field.

## 2. Related work

The need for comprehensive and publicly available datasets of own-speech recordings using BCMs, such as those listed in Table 1, is critical to the progress of research and development in the field of body-conducted speech capture. These datasets are essential for training and testing deep learning models.

### 2.1. Training datasets

Before delving into publicly available datasets, it is worth acknowledging that they have been made available for a relatively short period of time, are in some cases of insufficient size, and are only available in a restricted set of languages. Consequently, several studies have employed low-pass filtering of high-quality audio to artificially generate body-conducted speech data. A hybrid approach involves the collection of individual transfer functions between a reference microphone and a body-conducted microphone, which is the focus of the Hearpiece database, as referenced in Denk and Kollmeier (2021). In their article, Ohlenbusch et al. (2024b) proposed several speech-dependent models of one's own voice transfer function to simulate the degradation induced by body-conducted audio recordings. Their results indicate that the transfer function is, of course, speaker-dependent, but also phoneme-dependent due to the different position of the jaw. Their study brings a significant improvement in the ability to simulate data. Following on from this work, the authors also published (Ohlenbusch et al., 2024) to propose speech-dependent data augmentation to compensate for the lack of a large dataset of own speech signals. They were able to reduce the performance gap when testing the model on real signals. However, by analogy with FineWeb-Edu (Lozhkov et al., 2024), the performance improvement can also come from having a larger and highly curated dataset on which to perform classical supervised training.

A comprehensive literature review of existing, publicly available datasets of BCM-captured speech is essential to gain a full understanding of the subject. This encompasses all forms of capture, including bone, in-ear, and throat sensors. It should be noted that there exist several small private datasets (Erzin, 2009; Turan and Erzin, 2015; Shan et al., 2018; Davis et al., 2019; Tagliasacchi et al., 2020; Zheng et al., 2022; Schilk et al., 2023), albeit not open-sourced. The few remaining public datasets are listed in Table 1, including Vibravox. In terms of size, the ESMB dataset (ESMB-corpus, 2021) is the largest, with 128 h of recorded speech, followed by Vibravox with its 45h37 of recordings per sensor, ABCS (Wang et al., 2022a) (42 h), and EmoBone (Hosain et al., 2024) (19 h). The final dataset to be considered is TAPS (13h18) (Kim et al., 2025). This dataset includes post-recording processing to realign signals at the sample level, thereby easing the direct utilization of L1 loss in the waveform domain for the training of deep learning models to enhance body-conducted speech. Although Vibravox is not the largest dataset, it overcomes several limitations found in existing datasets, such as the limited diversity of
audio sensors and the lack of noise recordings. Vibravox addresses these issues by using five different BCMs and including recordings of speech in both noisy and quiet environments. Previous studies indicate that the transmission of external noise through the device is not only influenced by the device itself but also by individual variance (Denk and Kollmeier, 2021) and angle of arrival (Liebich et al., 2018). For this reason, the Vibravox dataset was recorded in a 3D sound spa- tializer to sample uniformly the sphere for noise emission. Its unique combination of simultaneous multi-microphone recordings and French- language data therefore provides complementary resources to existing datasets, offering new perspectives for non-conventional speech capture and comparative analyses of different sensor technologies.

## 2.2. Speech processing tasks

The body-conducted speech research community has made signifi- cant progress in recent years. The accessibility of public datasets such as Vibravox has become increasingly important for the development and improvement of deep learning models in this field. The continued availability of these resources is likely to facilitate further advances and innovations in body-conducted speech technology for tasks such as speech enhancement, speech transcription, and speaker verification.

### 2.2.1. Speech enhancement

BCM datasets, paired with airborne speech, are invaluable in the field of speech enhancement. This task refers to the process of improving the quality and intelligibility of speech, often in the presence of background noise. However, speech enhancement on BCM is mainly about bandwidth extension, as the sensors are robust to noise but cannot record high frequencies due to physical constraints. Several approaches have been published recently in this context. Among them, Zheng et al. (2022), Li et al. (2023a), Sui et al. (2024) model the long-term dependencies of speech by adapting the non-quadratic complexity variations of the attention mechanism (Vaswani et al., 2017; Liu et al., 2021; Gu and Dao, 2023). Some studies show superior reconstruction metrics compared to EBEN, but as shown in Table 1 of Hauret et al. (2023), one of the approaches reporting the best metrics is Kuleshov's U-net (Kuleshov et al., 2017), which is also the one that performs worst in the MUSHRA (Multiple Stimuli with Hidden Reference and Anchor) analysis. This evidence serves to illustrate the importance of conducting listening tests when comparing reconstructive and generative approaches. Furthermore, EBEN's computational and memory requirements are low, due not only to the limited number of parameters in the generator but also to the highly strided embeddings facilitated by the Pseudo-Quadrature Mirror Filters (PQMF) representation (Hauret et al., 2023). In Edraki et al. (2024), the authors proposed an innovative approach to deal with limited datasets. They developed a method to disentangle a speaker embedding vector from the hidden representations of the network during the enhancement process, which allowed a fair generalization performance with a limited training set of 5 h of 29 speakers and additional synthetic data. Conversely, the approach proposed in Sui et al. (2024) involves pre-training with simulated data and fine-tuning on a device with real data, thereby creating a speaker-dependent speech enhancement model.

### 2.2.2. Speech-to-text

Transcribed speech datasets are essential for training models to accurately recognize and transcribe spoken words. Eventually, performances can be boosted with a self-supervised pretraining (Oord et al., 2018) on unlabeled speech data to construct a semantic representation of speech prior to using text labels. To perform robust transcription, the inclusion of speech in both noisy and quiet environments in the Vibravox dataset is particularly beneficial, as it allows for the development of models that can perform well in a variety of acoustic conditions. To provide context, Ishimitsu (2010), Wang et al. (2022a), Cao et al. (2023) have previously addressed this task using this particular data modality. They have shown that body-conducted speech can improve the performance of a basic speech recognition system in adverse environments solely based on airborne speech.

<figure><img src="image_2.png" /><figcaption>Fig. 1. Fully equipped participant with the six audio sensors.</figcaption></figure>

### 2.2.3. Speaker verification

Finally, speaker verification is a biometric authentication method that uses a person's voice to confirm their identity. As with all the other tasks, if the microphone is less affected by reverberation and external noise, the problem of domain mismatch is reduced. This, in turn, enables higher recognition accuracy in challenging environments. Another benefit that is not immediately apparent is the amount of information available. Because everyone's skull and timbre are unique, the recorded signal can be more descriptive of a person's identity. This wealth of biometric information opens up several potential applications, including those in the military domain, as detailed in Everett (1985). One such application is access control, where it can strengthen security by adding an extra layer compared to conventional methods like keycards or codes. Another compelling application lies in intelligence gathering. By pinpointing known voices in intercepted transmissions, speaker identification can reveal the involvement of high-value targets. Beyond this paper's focus, this rich biometric data (heart rate, breathing) could be a game-changer in battlefield forensics, aiding in revealing emotional states and tracking life signals during critical events.

## 3. Building Vibravox

Creating the Vibravox BCM dataset involved a number of different tasks. These included designing pre-amplification and conditioning circuits for each transducer, coding the front-end and back-end software for the recording user interface, adapting the housings and fixings for some audio sensors to the human body through 3D modeling and printing, and selecting transducer technologies and recording parameters. For the sake of brevity, this section will only discuss the essential parts needed to gain a clear understanding of the collected data. All other details can be found at the following URL: https://vibravox.cnam.fr/, including pictures of the actual recording environment as well as some fully equipped participants during recording sessions.

### 3.1. Hardware

A comprehensive list of all BCMs employed in the Vibravox dataset is presented in Table 2. Their positioning on the participants' heads is shown in Fig. 1. Although not depicted in Fig. 1, a custom 3D-printed backpack and headset were also designed with the objective of facilitating the optimal placement of sensors, carrying the Zoom F8n field recorder, and routing audio cables to that device.<fnref n="1" /> The Zoom F8n records all six sensor signals simultaneously, operates at a sampling rate

<footnote n="1">¹ Several pictures and a video of participants wearing the sensors and the backpack during experiments are available at https://vibravox.cnam.fr/documentation/hardware/sensors.</footnote>
<table><caption>**Table 2**
Sensors specifications.</caption><thead><tr><th>Name</th><th>Location</th><th>Technology</th><th>Reference</th><th>Signal-to-self-noise</th><th>Signal-to-external-noise</th></tr></thead><tbody><tr><td>Headset microphone</td><td>Close to mouth</td><td>Cardioid electrodynamic microphone</td><td>Shure WH20XLR</td><td>36.0 dB</td><td>0.1 dB</td></tr><tr><td>Forehead accelerometer</td><td>Forehead</td><td>One-axis piezoceramic accelerometer</td><td>Knowles BU23173-000</td><td>23.7 dB</td><td>-3.5 dB</td></tr><tr><td>In-ear soft foam microphone</td><td>Left ear</td><td>Omni. electret condenser microphone</td><td>Knowles FG-23329-P07</td><td>22.8 dB</td><td>4.0 dB</td></tr><tr><td>In-ear rigid earpiece microphone</td><td>Right ear</td><td>Omni. MEMS microphone</td><td>Knowles SPH1642HT5H</td><td>29.0 dB</td><td>3.9 dB</td></tr><tr><td>Throat microphone</td><td>Larynx</td><td>Piezoelectric</td><td>iXradio XVTM822D-D35</td><td>41.1 dB</td><td>25.6 dB</td></tr><tr><td>Temple vibration pickup</td><td>Temple</td><td>Figure of-eight condenser transducer</td><td>AKG - C411</td><td>9.5 dB</td><td>8.9 dB</td></tr></tbody></table>

of 48 kHz and a resolution of 32 bits, thereby guaranteeing precise signal capture with high fidelity. To enhance audio data quality, a 20 Hz high-pass filter is uniformly applied to each acquisition channel, effectively attenuating low-frequency noise and artifacts.

In order to provide a balanced comparison between body conduction and airborne audio sensors in terms of immunity to ambient noise, we selected an airborne microphone mounted on a headset that presents a cardioid directivity pattern towards the speaker's mouth, hence producing a dry reference. The in-ear soft foam-embedded microphone is a prototype designed by the ISL and used in Blondé-Weinmann et al. (2023). The in-ear rigid earpiece-embedded microphone was initially presented in Denk et al. (2019) as a versatile research earpiece with multiple drivers and microphones. In this study, we use the occluded version of this device and only record the in-ear microphone signal. The laryngophone has been purchased from iXradio. This device consists of dual piezoelectric sensors on an adjustable neck rim, which captures skin vibrations from vocal cords and targets voice amplifiers, recorders, and Voice over Internet Protocol (VoIP) applications. The forehead accelerometer, affixed via a homemade headband, is a compact contact sensor that has been commercialized by Knowles. It is designed for use in high-noise environments and can be integrated into helmets for military and emergency service communications. Finally, the temple vibration sensor is an AKG product, a miniature vibration sensor originally designed for string instruments. In this study, this sensor has been diverted from its original purpose to capture bone vibrations by placing it in an adjustable 3D-printed enclosure adapted to the participants' zygomatic bones.

In addition to sensor specifications, Table 2 presents two signal-to-noise ratio (SNR) measures for each sensor. The first, signal-to-self-noise, is computed as the difference between the $L_{50}^{silence}$ and $L_{50}^{speech}$ values, which will be defined in Section 3.5, across the entire speechless-clean split of the dataset, as defined in Section 3.4. Notably, all sensors exhibit a high signal-to-self-noise ratio, except for the temple vibration sensor, which is primarily designed to capture much higher amplitude signals. The second measure is the signal-to-external-noise ratio, calculated under an ambient noise level of 90 dB. Since the reference signal is unavailable in this case, we derived this ratio by decomposing it into the signal-to-self-noise minus the external-noise-to-self-noise ratio. This was estimated using the speechless-noisy and speechless-clean subsets, as defined in Section 3.4. This decomposition assumes that speakers produce speech at the same loudness in both quiet and noisy environments. However, in practice, the Lombard effect causes speakers to increase their vocal effort in noisy conditions, meaning the reported signal-to-external-noise ratios represent a lower bound on the actual values. Despite this approximation, the signal-to-external-noise ratio remains a direct indicator of each sensor's noise resilience. Our results show that sensors directly capturing skin vibrations exhibit the highest noise resilience. However, these sensors also tend to have the smallest bandwidth, highlighting the trade-off between noise robustness and frequency range.

### 3.2. Textual data

The text utilized for participant readings originates from the French Wikipedia subset of Common Voice (Ardila et al., 2020). This textual data was further filtered to produce a simplified dataset with a

minimum number of textual and phonemic symbols and to minimize pronunciation uncertainty. To create this simplified dataset, we applied textual filters that excluded most proper names, retaining only common first names, French town or region names, and country names. Moreover, utterances containing numbers, Greek letters, mathematical symbols, or syntactic errors were carefully removed, resulting in a more straightforward dataset that is suitable for accurate pronunciation analysis. Once the unwanted sentences were removed, we applied common text normalization operations, such as lowercasing text and removing punctuation except for apostrophes. From this normalized text, we also generated the corresponding phonetic transcription using the International Phonetic Alphabet (IPA), which contains 33 phonemes in French. This operation was done with the text phonemizer of the CoML lab (Bernard and Titeux, 2021). For some words, the phonemizer produced some language switches that we wanted to avoid, to preserve the minimal French phonetic alphabet. A phase of manual phonetic transcription was then necessary for these unrecognized words, such as “algorithmie”.

### 3.3. External ambient noise data

The Vibravox contains 4 subsets: speech-clean, speech-noisy, speechless-clean and speechless-noisy. For the speech-noisy and speechless-noisy subsets, audio recordings were obtained from sensors worn by participants (either speaking or remaining silent) during the broadcast of 3D spatialized ambient noise. For all these ambient noise samples, the spatialization process was carried out using the ambitools library<fnref n="2" /> (Lecomte and Gauthier, 2015) and rendered by a spherical array with a radius of 1.07 m, made up of 56 loudspeakers placed around the participants (Lecomte et al., 2016). A significant proportion of the noise excerpts were sourced from the Audio Set (Gemmeke et al., 2017), with a particular focus on noise classes relevant to BCMs applications, such as those encountered in battlefield and industrial environments. The Audio Set noise samples were spatialized using a plane wave model and from random fixed positions uniformly distributed on the sphere. In addition, in-house recordings of concerts and street events in 3rd and 5th order ambisonics were also rendered around the participants.

### 3.4. Recording protocol

The recording process consists of four steps that are performed consecutively for each participant in the order listed below. Each of these steps corresponds to a subset of the Vibravox dataset.

* **speech-clean**: The participant reads sentences sourced from the French Wikipedia for a duration of 15 min. Each utterance generates a new recording, and the transcriptions are retained. The recordings from this step populate the Vibravox subset that contains the most data for training.

* **speechless-noisy**: For a period of 2 min and 24 s, the subject is required to remain silent yet free to move, swallow, and breathe naturally in a noisy environment created from the

<footnote n="2">² Available at https://github.com/sekisushai/ambitools.</footnote>
<figure><img src="image_3.png" /><figcaption>Fig. 2. Post-processing filtering process.</figcaption></figure>

AudioSet samples described in Section 3.3. Those samples are played in a spatialization sphere equipped with 56 loudspeakers surrounding the subject. The objective of this phase is to gather realistic background noises that will be combined with the speech-clean recordings to maintain a clean reference.

* **speechless-clean**: The procedure is repeated for 54 s in complete silence to record solely physiological and audio sensor noises. These samples can be valuable for tasks such as heart rate tracking or simply analyzing the noise properties of the various sensors. It could also be conveniently used to generate synthetic datasets with realistic physiological (and sensor-inherent) noise captured by body-conduction audio sensors.

* **speech-noisy**: The final phase (54 s) will serve to test the different systems (speech enhancement, automatic speech recognition, speaker verification, ...) that will be developed based on the recordings from the first three subsets. This real-world testing will provide valuable insights into the performance and effectiveness of these systems in practical scenarios. In this phase, the noise samples replayed in the 3D spatializer are sourced from in-house ambisonic recordings of concerts and street events.

### 3.5. Post processing

A total of $J=30,568$ sentences were initially collected from 200 participants in the speech-clean subset. However, a small number of

audio clips had various shortcomings: although we carefully monitored and validated each recording during the experimental process, the workflow was not entirely foolproof. Mispronounced sentences, sensors shifting from their initial positions, occasional sensor malfunctions, or instances where a microphone was inadvertently left unequipped led to some data inconsistencies. In order to ensure that our dataset would allow for meaningful error analysis between sensors across multiple speech processing tasks, we applied a series of three automatic filters to remove utterances with obvious deficiencies. We retained only sentences with correct pronunciation and a sufficiently high SNR, based on predefined criteria outlined in Fig. 2. However, in some cases, such as when a participant's ear canal was too small for the rigid in-ear microphone to achieve proper acoustic sealing, these filters were not triggered.

The filtering process comprises two main stages. In the first stage, indicators for evaluating audio capture quality are extracted (Fig. 2(a)). In the second stage, these indicators are used to derive a set of filters (Fig. 2(b)). The whisper-timestamped model<fnref n="3" /> constitutes a key component of our indicator extraction process. This model is a speech-to-text engine based on Whisper (Radford et al., 2023) that applies dynamic time warping (Giorgino, 2009) to cross-attention weights. By aligning the transcription with the raw waveform, accurate prediction of word

<footnote n="3">³ Available at https://github.com/linto-ai/whisper-timestamped.</footnote>
above 1500 Hz. The laryngophone follows, with coherence dropping below 0.25 at 2000 Hz. The soft and rigid in-ear sensors exhibit larger bandwidths yet display clear antiresonance frequencies. The soft in-ear bandwidth is slightly larger than its rigid in-ear counterpart, which may be due to a less effective acoustic seal. Finally, the forehead accelerometer offers the largest bandwidth but does not effectively filter out external noise pollution. As discussed in Section 4, this bandwidth is closely correlated with the performance of deep learning models on speech intelligibility, quality, speaker verification, and speech-to-phonemes.

Beyond bandwidth analysis, we can also observe that the shape of these coherences is not always flat, depending on the sensor placement. We cannot draw clear conclusions here because we also changed the technology at each placement, but some works like (McBride et al., 2011) have already explored the question of the best body-conducted microphone location. Their findings show that the forehead and temple are the best spots to capture speech with the highest intelligibility and sound quality, while the throat is among the worst positions.

# 4. Using vibravox

The Vibravox dataset offers considerable potential across various applications, particularly in improving communication in noisy environments. The speech-noisy subset is of particular interest for showcasing the noise-resilient capabilities of BCMs. However, this subset primarily serves testing purposes, as it lacks a clean reference under such conditions. As a consequence, the majority of the dataset is contained within the speech-clean subset, where the airborne microphone serves as the reference. In order to simulate real-world noisy scenarios while retaining a clean reference, data augmentation techniques can be applied by adding audio segments of the speechless-noisy subset with BCM utterances. Furthermore, the Vibravox dataset displays potential for applications such as developing voice commands and monitoring vital signs, including heart rate, particularly when utilizing the speechless-clean subset. In the following, we propose three classic tasks in automatic speech processing to investigate the distinctive characteristics of BCMs, establishing baselines for each task.

## 4.1. Speech enhancement

The high impedance of BCMs enables them to focus on the wearer's voice by rejecting ambient noise, but this also affects their ability to capture mid and high frequency components due to the intrinsic low-pass characteristics of the biological pathway of vibrations between the vocal cords and the body-conducted audio sensors. The absence of these frequencies in captured speech directly impacts the perceived quality, intelligibility, and distinctiveness. This highlights the necessity for a post-capture enhancement algorithm.

### 4.1.1. Model

We adopted the configurable EBEN (Hauret et al., 2023) approach, enriched with recent trends in the audio field. This GAN (Goodfellow et al., 2014) architecture offers several advantages. In particular, the lightweight design of the 1D U-net-like generator is compatible with integration in mobile and wearable systems, where memory is limited and real-time is a priority. Additionally, the PQMF bank (Nguyen, 1994) offers the advantage of manipulating frequency content without the need to consider phase or manage complex values, as it operates directly on waveform signals. It is important to note that the limitations of GANs, including the difficulty of reaching a Nash equilibrium, the lengthy training period, and the additional computational cost associated with the discriminators, do not affect the model once it has been deployed.

### 4.1.2. Architecture configuration

Separate EBEN models were trained for each sensor in the Vibravox dataset, using sensor-specific hyperparameters detailed in Table 4.

The original number of subbands was maintained at a value of $M = 4$ for each sensor due to the significant variability in intra-sensor bandwidth among participants, rendering precise spectrogram slicing impractical. Furthermore, a smaller number of subbands allows for a more permissive design of the passband filters with a softer slope for the same resulting aliasing, thus shortening the PQMF kernel length, which is useful for reducing the algorithmic latency. The number of subbands provided to the model, denoted as $P$, was tailored to match the bandwidth of each sensor. For example, the temple vibration pickup captures signals within the 0–2 kHz range, while the forehead accelerometer exhibits good coherence with the reference microphone up to 8 kHz. Consequently, after resampling the signals to 16 kHz for training, only the first band was retained for the temple vibration pickup, while all bands were retained for the accelerometer. In addition, the last configurable parameter, $Q$, which represents the number of bands fed to the PQMF discriminator, has been modified to match $M$ at 4 for all models. It was observed that even when the audio sensors capture information within a specific frequency range—including the corresponding subbands in the discriminator input—it is beneficial for denoising minor artifacts present in these initial bands. In the case of speech captured in noisy conditions, even though body conduction sensors are robust to external acoustic pollution, some noise is still captured, providing an additional reason to focus discriminators work on every subband. This minor modification necessitated an adjustment to the number of channels in the PQMF discriminators in order to align with the divisibility constraints of the grouped convolution.

### 4.1.3. Training procedure

In comparison to the original implementation proposed in Hauret et al. (2023), we have incorporated a spectral loss $L_G^{spec}$ into the generator objective. This modification proved to be effective in stabilizing the training process. This loss incorporates the same FFT, hop size, and window length parameters as described in Yamamoto et al. (2020). Implementation is taken from (Steinmetz and Reiss, 2020). In contrast to some speech enhancement papers (Défossez et al., 2020; Subakan et al., 2021), we did not include a reconstructive loss in the temporal domain due to the peculiarities of body-conduction speech. In fact, complex phase shifts occur: attempting to align the signals to the reference at the sample level degraded performance. We retained the original EBEN implementation's use of the other two adversarial and a feature matching losses, respectively noted $L_G^{adv}$ and $L_G^{eat}$. Also note that a slight adjustment to $L_G^{eat}$ has been done by normalizing it using the enhanced feature norm. The training strategy remained consistent across speech-clean and speech-noisy experiments, with a constant learning rate of $3e^{-4}$. To ensure reproducibility of experiments, the code has been made available on the GitHub repository https://github.com/jhauret/vibravox, which makes use of the following libraries (Paszke et al., 2019; Falcon and team, 2019; Yadan, 2019; Lhoest et al., 2021).

### 4.1.4. Experiments in quiet conditions

For this initial set of experiments, the goal is to evaluate the performance of bandwidth extension in quiet conditions by training EBEN models on the speech-clean subset. A total of 500 epochs were required for convergence on the Vibravox dataset. To address the relatively small amount of data, light data augmentation was applied, including speed perturbation, pitch shifting, and time masking, all implemented in the Torchaudio library (Hwang et al., 2023). The correlation analysis presented in the configurable EBEN article (Hauret et al., 2023) revealed that STOI (Taal et al., 2010; Jensen and Taal, 2016) and Noresqa-MOS (N-MOS) (Manocha and Kumar, 2022) were the metrics with the highest correlation with MUSHRA studies in the context of speech enhancement on BCMs simulated data. Consequently, these two metrics were retained for inclusion in Table 4, which presents the results of the bandwidth extension for each sensor in quiet conditions. The results reveal a clear relationship between the metrics
<figure><img src="image_4.png" /><figcaption>Fig. 4. Spectrograms of signals recorded by the different Vibravox audio sensors in the speech-clean subset and their corresponding EBEN-enhanced version.</figcaption></figure>

<table><caption>**Table 4**
Testing of EBEN models and raw signals on speech-clean.</caption><thead><tr><th>Sensor</th><th>Configuration</th><th>STOI</th><th>Noresqa-MOS</th></tr></thead><tbody><tr><td rowspan="2">Forehead</td><td>Raw signal</td><td>0.731</td><td>3.760</td></tr><tr><td>EBEN (M=4, P=4, Q=4)</td><td><strong>0.855</strong></td><td><strong>4.250</strong></td></tr><tr><td rowspan="2">Soft In-ear</td><td>Raw signal</td><td>0.752</td><td>3.315</td></tr><tr><td>EBEN (M=4, P=2, Q=4)</td><td><strong>0.868</strong></td><td><strong>4.331</strong></td></tr><tr><td rowspan="2">Rigid In-ear</td><td>Raw signal</td><td>0.782</td><td>3.392</td></tr><tr><td>EBEN (M=4, P=2, Q=4)</td><td><strong>0.877</strong></td><td><strong>4.285</strong></td></tr><tr><td rowspan="2">Throat</td><td>Raw signal</td><td>0.677</td><td>3.097</td></tr><tr><td>EBEN (M=4, P=2, Q=4)</td><td><strong>0.834</strong></td><td><strong>3.862</strong></td></tr><tr><td rowspan="2">Temple</td><td>Raw signal</td><td>0.602</td><td>2.905</td></tr><tr><td>EBEN (M=4, P=1, Q=4)</td><td><strong>0.763</strong></td><td><strong>3.632</strong></td></tr></tbody></table>

value and the sensors' available bandwidth. Yet, both STOI and N-MOS performance appear to hit a glass ceiling, plateauing at 0.88 and 4.3, respectively. This saturation could stem from the challenge of extracting fine details from body-conducted signals. While striving to produce authentic samples, the generator only picks one plausible clean signal out of all the possible ones, typical of one-to-many problems (Tan et al., 2024). The spectrograms in Fig. 4 provide a visual representation of the processing.

### 4.1.5. Experiments in noisy conditions

After analyzing speech enhancement in quiet conditions, the next focus is a joint denoising and bandwidth extension task for recordings in noisy environments. To this end, the model is tested on the speech-noisy subset; however, this subset cannot be used for training due to the absence of a clean reference. For training, the speechless-noisy subset is leveraged and dynamically mixed with the speech-clean subset. Several configurations were tested, including retraining the EBEN models from scratch for 500 epochs or initializing from the previously trained speech-clean models,

which converged after no more than 200 epochs for microphones with the largest domain gap. Additionally, speech-clean models were directly tested on speech-noisy. Hyperparameters of the training strategy were kept consistent with quiet condition experiments, with the same values for M, P, and Q across all microphones.

Prior to testing on speech-noisy, the model performance was first evaluated on the validation and test splits of the synthetically mixed speech-clean+speechless-noisy data. When testing on speech-noisy, no significant differences were observed in the Noresqa-MOS scores, reflecting a realistic modeling of the training data. When using speech-noisy for evaluating the models, the STOI—which is a reference-based metric—was replaced with the reference-free TorchSquim-STOI (Kumar et al., 2023), which provides an estimate of the same quantity. Although direct comparison between STOI and TorchSquim-STOI is not possible, both metrics showed improvement during training, and the TorchSquim-STOI results remained consistent across both synthetic mixed and real noisy data. The final test results on speech-noisy are presented in Table 5.

The results reveal that microphones with the lowest signal-to-external-noise ratios benefited the most from the retrained strategies. Some microphones, such as the temple vibration pickup, were so insensitive to noise that the non-retrained speech-clean models outperformed them. The Noresqa-MOS scores for the raw signals also increased compared to the speech-clean subset. However, these Noresqa-MOS values cannot be directly compared, as in the quiet conditions, the metric was computed using matched signals, whereas in the noisy conditions, a random headset signal from the speech-clean was used for the calculation.

As with the experiments conducted in quiet conditions, several spectrograms are presented in Fig. 5. Synthetically mixed data were selected to visually assess the effect of the models, as both the noise and the clean target signals remain available. This figure highlights several noteworthy observations. Notably, the excessive noise detected in the soft in-ear microphone suggests an insertion defect for this particular
<figure><img src="image_5.png" /><figcaption>Fig. 5. Synchronized spectrograms of multiple signals for the speech enhancement task in noisy conditions. (a) & (d): Synthetically mixed samples, speech comes from speech-clean and noise comes from speechless-noisy respective microphones - (b): Corresponding enhanced samples performed by EBEN models trained on the speech-clean subset - (c): Corresponding enhanced samples performed by EBEN models trained on the synthetically mixed speech-clean and speechless-noisy subsets - (e): Clean speech sample recorded by the headset microphone in the speech-clean subset before mixing - (f): Noise sample recorded by the headset microphone in the speechless-noisy subset before mixing.</figcaption></figure>

sample of speechless-noisy, whereas the rigid microphone appears to provide better acoustic sealing. However, such defects occur just as often in both microphone types and are absent in most cases. Additionally, EBEN models trained for bandwidth extension in quiet conditions exhibit limited noise suppression. In contrast, as shown in line (c), noise-robust EBEN models effectively suppress noise, with the difference particularly pronounced for forehead and in-ear microphones. For throat and temple microphones, which are inherently more resistant to noise, the domain gap between quiet and noisy conditions

remains small, enabling the speech-clean EBEN models to maintain strong performance. Achieving comparable denoising performance from the noisy headset using conventional denoising networks would be highly challenging.

## 4.2. Speech-to-phonemes

Automatic Speech Recognition (ASR) systems serve as crucial facilitators of seamless human-computer interaction, allowing users to
<table><caption>**Table 5**
Testing of EBEN models and raw signals on speech-noisy.</caption><thead><tr><th>Sensor</th><th>Initialization</th><th>Squim-STOI</th><th>Noresqa-MOS</th></tr></thead><tbody><tr><td rowspan="4">Forehead</td><td>Raw signal</td><td>0.901</td><td>3.85</td></tr><tr><td>Tested from pretrained*</td><td>0.949</td><td>4.08</td></tr><tr><td>Trained† from scratch</td><td>0.949</td><td>4.14</td></tr><tr><td>Trained† from pretrained*</td><td><strong>0.971</strong></td><td><strong>4.20</strong></td></tr><tr><td rowspan="4">Soft In-ear</td><td>Raw signal</td><td>0.813</td><td>3.38</td></tr><tr><td>Tested from pretrained*</td><td>0.865</td><td>3.71</td></tr><tr><td>Trained† from scratch</td><td>0.911</td><td>3.63</td></tr><tr><td>Trained† from pretrained*</td><td><strong>0.917</strong></td><td><strong>3.87</strong></td></tr><tr><td rowspan="4">Rigid In-ear</td><td>Raw signal</td><td>0.751</td><td>3.47</td></tr><tr><td>Tested from pretrained*</td><td>0.812</td><td>3.73</td></tr><tr><td>Trained† from scratch</td><td><strong>0.876</strong></td><td>3.51</td></tr><tr><td>Trained† from pretrained*</td><td>0.873</td><td><strong>3.80</strong></td></tr><tr><td rowspan="4">Throat</td><td>Raw signal</td><td>0.942</td><td>3.71</td></tr><tr><td>Tested from pretrained*</td><td>0.969</td><td><strong>4.05</strong></td></tr><tr><td>Trained† from scratch</td><td><strong>0.978</strong></td><td>3.87</td></tr><tr><td>Trained† from pretrained*</td><td>0.971</td><td>3.98</td></tr><tr><td rowspan="4">Temple</td><td>Raw signal</td><td>0.848</td><td>3.46</td></tr><tr><td>Tested from pretrained*</td><td>0.954</td><td><strong>3.77</strong></td></tr><tr><td>Trained† from scratch</td><td><strong>0.966</strong></td><td><strong>3.77</strong></td></tr><tr><td>Trained† from pretrained*</td><td>0.961</td><td>3.63</td></tr></tbody></table>

dictate text, control devices, and access information through spoken language. Proficiency in this realm with BCMs can confer significant advantages, particularly in military operations or industrial settings.

### 4.2.1. Model

ASR is often divided into two models as in Baevski et al. (2020): an acoustic model that infers the tokens from the raw waveform and a language model that is used to refine the probabilities given by the acoustic model with a beam search algorithm. However, our focus primarily revolves around investigating the acoustic characteristics of various sensors. Hence, we have opted to exclude the language model and instead predict phonemes directly with greedy decoding of the acoustic model. The tokens predicted by the linear head of our model correspond to the 33 phonemes of the French phonetic alphabet. We used a medium wav2vec2.0 (Baevski et al., 2020) as our acoustic model, pretrained on the multilingual VoxPopuli speech corpus (Wang et al., 2021).

### 4.2.2. Experiments in quiet conditions

For this first set of experiments, we finetuned the wav2vec2.0 models with a constant learning rate of $1e^{-5}$ on the speech-clean subset of the Vibravox dataset for 10 epochs, minimizing the CTC loss (Graves et al., 2006). We then tested each fine-tuned model not only on its corresponding test set but also on every other audio sensor, with the PER (Phoneme Error Rate) results shown in Fig. 6. As expected, the matrix shows dominance along the diagonal, which corresponds to in-distribution tests. From this diagonal, we infer that the temple vibration pickup and the throat microphone pose the greatest challenge for ASR in quiet conditions, given their limited bandwidth. A comparison between models trained on airborne and temple vibration sensors reveals a significant trend: the temple vibration pickup demonstrates enhanced robustness when tested on other sensor data. This trend can be attributed to the model's reliance primarily on low-frequency information, which is present on all sensors. Regarding the in-ear microphones, there is a possibility that a single model can be developed in future studies, since they are highly similar and exhibit minimal performance loss when cross-tested. At last, the performance of the forehead accelerometer is approaching that of the in-ear microphone, despite the larger bandwidth. This may be attributed to the considerable variance in the coherence function and the lower signal-to-self-noise ratio.

<figure><img src="image_6.png" /><figcaption>**Fig. 6.** PER of the finetuned audio phonemizers. Models are trained and tested on the speech-clean subset.</figcaption></figure>

To gain a detailed understanding of the acoustic properties of each sensor, we also identified the top five phonetic confusions of all phonemizers when tested on their raw signals training distribution. Table 6 offers valuable insights into the challenges inherent to each sensor placement and technology for the identification of phonemes due to the absence of high-frequency content. This table was obtained by counting the most frequently used editing operation to go from the model's decoded output to the labeled phonetic transcription when computing the Levenshtein distance (Levenshtein et al., 1966). It should be noted that only in-word phonetic confusions were considered to avoid including the suprasegmental phenomenon [~] (i.e. "liaison" in French), which poses difficulties for many native speakers during reading and which is not present in our labeled phonetic transcription. This phenomenon caused all models to either omit or add [z] (i.e. [θ] → [z] and [z] → [θ]) as their top errors. Once this issue has been disregarded, Table 6 reveals that the model trained on airborne microphone signals encounters difficulties in distinguishing the similar phoneme confusion we encounter in French, such as [e]/[ɛ] and [ɑ]/[ɔ]/[ɔ], which are all voiced sounds. Similar errors have been observed in other audio sensors with a sufficiently large bandwidth, albeit with varying extents. Of particular interest is the observation that the temple vibration pickup, which has a narrow bandwidth, most frequently fails to capture the voiceless alveolar and plosive fricatives [s]/[t] that rely on substantial high-frequency support. The wideband support [ɛ] is also left out to a significant extent for the temple vibration pickup and throat microphone. For the temple sensor, this could be due to the similarity of this phoneme with the sensor's self-noise. For the throat microphone, we do not have a clear explanation, especially since this sensor placement seems to be optimal for capturing this uvular fricative.

### 4.2.3. Effect of EBEN enhancement in quiet conditions

A supplementary analysis of the PER has been performed in order to quantify the effect of the bandwidth enhancement from EBEN models trained on speech-clean presented in Section 4.1 for ASR. This comparison was conducted employing solely the headset phonemizer. The results presented as a bar graph in Fig. 7 demonstrate that all sensors exhibited a strong improvement in PER thanks to the enhancement

<footnote>* On the speech-clean dataset.</footnote>

<footnote>† On the synthetically mixed speech-clean+speechless-noisy dataset.</footnote>
<table><caption>**Table 6**
Top five editops count for in-distribution testing of audio phonemizers on raw speech signals in speech-clean.</caption><thead><tr><th>Sensor</th><th>1st editop</th><th>2nd editop</th><th>3rd editop</th><th>4th editop</th><th>5th editop</th></tr></thead><tbody><tr><td>Headset Microphone</td><td>[0] → [>] (98)</td><td>[e] → [e] (75)</td><td>[a] → [>] (64)</td><td>[>] → [0] (64)</td><td>[θ] → [l] (52)</td></tr><tr><td>Forehead Accelerometer</td><td>[θ] → [l] (124)</td><td>[>] → [0] (106)</td><td>[e] → [e] (92)</td><td>[0] → [>] (80)</td><td>[θ] → [e] (74)</td></tr><tr><td>Soft In-ear Microphone</td><td>[e] → [e] (127)</td><td>[>] → [0] (115)</td><td>[θ] → [l] (93)</td><td>[a] → [>] (68)</td><td>[θ] → [e] (66)</td></tr><tr><td>Rigid In-ear Microphone</td><td>[e] → [e] (122)</td><td>[θ] → [l] (103)</td><td>[>] → [0] (97)</td><td>[0] → [>] (93)</td><td>[θ] → [j] (84)</td></tr><tr><td>Throat Microphone</td><td>[θ] → [l] (160)</td><td>[θ] → [e] (148)</td><td>[θ] → [i] (129)</td><td>[>] → [0] (128)</td><td>[θ] → [u] (107)</td></tr><tr><td>Temple Vibration Pickup</td><td>[θ] → [s] (294)</td><td>[θ] → [l] (290)</td><td>[θ] → [t] (281)</td><td>[θ] → [u] (281)</td><td>[θ] → [i] (229)</td></tr></tbody></table>

<figure><img src="image_7.png" /><figcaption>**Fig. 7.** PER on speech-clean of raw and EBEN-enhanced test recordings phonemized by the headset phonemizer.</figcaption></figure>

<figure><img src="image_8.png" /><figcaption>**Fig. 8.** PER of the test recordings phonemized by the models trained on speech-clean and speech-noisy in noisy conditions.</figcaption></figure>

obtained with EBEN models. This is a promising outcome, since con- temporary speech enhancement systems, when used as a pre-processing layer in ASR systems, continue to face challenges in enhancing perfor- mance due to the production of artifacts (Iwamoto et al., 2022). The aforementioned findings thus corroborate the hypothesis that speech intelligibility is enhanced by EBEN models. However, it is essential to note that the direct parallel between the headset phonemizer and the human ear cannot be taken for granted. Subjective testing could be considered to provide an additional layer of validation.

### 4.2.4. Experiments in noisy conditions

To complete the analysis of this task, a new set of audio phonem- izers was trained on the speech-noisy training set, using the raw, unprocessed samples. Unlike for the speech enhancement task, where a clean reference is unavailable in speech-noisy, labels (phonemized transcriptions) remain available as reference data in noisy conditions. Several training strategies were explored, including retraining from the VoxPopuli checkpoint and fine-tuning the previously trained speech- clean models with an even smaller learning rate of $1e^{-6}$. The latter approach demonstrated a clear advantage. Additionally, given that the speech-noisy training set is only 1 h and 34 min in length, an aggressive data augmentation strategy was necessary to prevent overfitting.

Fig. 8 presents the speech-noisy test results for phonemiz- ers trained exclusively on the speech-clean subset, along with those refined using speech-noisy training data. These results high- light the impact of signal-to-external-noise ratios on the final per- formance of the phonemizers, as well as the performance gap be- tween speech-clean and speech-noisy models. Specifically, as microphone-captured noise increases, the task becomes more challeng- ing, and the domain gap between quiet and noisy conditions broadens. However, fine-tuning helps reduce this gap. These findings offer a pseudo-ranking of speech intelligibility in a 90 dB noisy environment across different microphones. Notably, the headset microphone remains the top performer due to its strong directivity towards the speaker's mouth. Since no processing was applied to reconstruct the bandwidth of body-conducted signals, high-frequency phonemes were lost. A signifi- cant performance drop is observed for the in-ear microphone in noisy

conditions, likely due to inadequate acoustic sealing for certain ear canal anatomies that do not fit the sensor properly. While a “one-size- fits-all” design is ideal in theory, molded eartips would provide a better fit in practice, although this would require considerable additional manufacturing time for each participant. Throat and temple speech- clean phonemizers remain performant in noise, reflecting the high signal-to-external noise ratio in Table 2 of those sensors.

### 4.3. Speaker verification

This final section addresses speaker verification, which ascertains whether two audio files feature the same speaker, providing a boolean answer. Speaker verification constitutes a subset of speaker recognition tasks (Reynolds and Campbell, 2008), which also encompass speaker identification and diarization. This task is especially pertinent in radio communication scenarios with multiple speakers, where BCMs can ex- acerbate the probability of voice confusion. Such confusion can impede fluid conversations, particularly in noisy environments. Furthermore, robust speaker identification systems are essential in contexts where heavy machinery is operated via voice commands, ensuring rigorous safety protocols are upheld. As stated by Schwartz et al. (2018), “The presence and use of high-frequency information from the speech signal improves the recognition of individual speakers, particularly in noisy environments”. Therefore, as highlighted by previous research (Sahidullah et al., 2016, 2017; Liu et al., 2018; Shang and Wu, 2019; Gao et al., 2021), the lack of these frequencies presents a significant challenge to BCMs speaker verification, necessitating further investigation to produce a more robust technology for these sensors. We therefore assess the efficiency of speaker verification tasks using body-conducted sensors from the Vibravox dataset.

#### 4.3.1. Model

The task is based on a distance metric comparison of speaker em- bedding results generated by a neural network from two distinct speech signals. Recently, Brydinskyi et al. (Brydinskyi et al. (2024) investi- gated the effectiveness of various models trained on English datasets
for differentiating non-English speakers. Their findings indicated superior performance from TitaNet (Koluguri et al., 2022) and ECAPATDNN (Desplanques et al., 2020) compared to WavLM (Chen et al., 2022) or Pyannoote (Bredin, 2023). In light of the recent advances in speaker verification, particularly the ECAPAS2 model (Thienpondt and Demuynck, 2023), which achieves state-of-the-art results, we elected to utilize its pre-trained version<fnref n="4" /> on the Vibravox dataset for our analysis.

### 4.3.2. Testing procedure

It is crucial to highlight that the speaker verification task is only composed of a testing procedure, where we utilize a pretrained ECAPAS2 model without any further training. Consequently, we solely utilize the test splits. The speaker verification testing set is constructed by forming pairs of speech signals, designated as A and B, in alignment with the methodology outlined in Brydinskyi et al. (2024). Notably, A and B may correspond to different airborne or body-conduction microphones. For each speaker, we generate $N_p$ pairs with their own recordings and an additional randomly selected $N_p$ pairs with recordings from different speakers. To increase the difficulty of the task, we maintain the possibility of separating speakers by gender. Here, $N_p$ represents the number of possible pairs for the speaker with the fewest recordings, which is set at 2346. We thus have a total of 98532 test pairs, given the fact that the speech-clean test set is composed of 21 speakers. We generate pairs only for the speech-clean subset to evaluate the model under quiet conditions and reuse these same pairs for the noisy condition. The latter is handled through a static mixing of speech-clean with speechless-noisy, which was shown in Section 4.1 to be a suitable modeling approach. This allows for a direct comparison of results between the two recording conditions. When comparing pairs of signals, we compute the cosine similarity and the Euclidean distance between their normalized embeddings. After inference on the entire set of pairs, we compute the Equal Error Rate (EER) and the normalized minimum of the Detection Cost Function (DCF) (US National Institute of Standards and Technology, 2018) from the cosine similarity scores.

### 4.3.3. Experiments in quiet conditions

The results of the ECAPAS2 model test are presented in Fig. 9 for all sensor combinations. As with speech-to-phonemes, the matrix exhibits a diagonal with lower coefficients because it is easier to distinguish between two recordings taken from the same audio sensor. The asymmetry of the matrix is to be expected, given that the script for generating pairs is not commutative due to its randomness. Nevertheless, the results from the upper triangular and lower triangular parts are very close and show a low variance. It is immediately apparent that there is a considerable discrepancy in the diagonal EER between different sensors. To illustrate this point, the headset and temple vibration pickup exhibit an EER differing by a rate of 30. The cross-tested EER for the forehead/headset and soft/rigid in-ear microphones remains low compared to their initial performance, indicating the proximity of those sensors. Once more, we observe that the difficulty of the task is again mostly driven by the sensor's bandwidth.

### 4.3.4. Effect of EBEN enhancement in quiet conditions

To complete this study, the Equal Error Rate (EER) was compared between raw and EBEN-enhanced signals. Contrary to the observations in Section 4.2, a deterioration in performance is evidenced in Fig. 10. Note that speaker identity preservation was not included among the three losses—adversarial loss ($L_G^{adv}$), feature loss ($L_G^{feat}$), and spectral loss ($L_G^{spec}$) which were employed to adjust the EBEN parameters. One possible explanation for the observed performance drop is that the ECAPAS2 model may interpret the EBEN outputs as being out-of-distribution, which could account for the poor performance. This issue could be addressed in future work by incorporating speaker identity preservation losses into the overall objective, as explored in Wang et al. (2022b).

<figure><img src="image_9.png" /><figcaption>Fig. 9. EER on speech-clean with the speaker verification model for all pairs.</figcaption></figure>

<figure><img src="image_10.png" /><figcaption>Fig. 10. EER on speech-clean of the raw and EBEN-enhanced recordings with the speaker verification model on pairs of the same audio sensor.</figcaption></figure>

### 4.3.5. Experiments in noisy conditions

This section evaluates ECAPAS2's performance on both the speech-clean and the synthetic mix speech-clean + speechless-noisy datasets, the latter simulating speech-noisy while preserving identical pairs. Fig. 11 shows that the Equal Error Rate (EER) generally increases, although the extent of degradation varies. For the two microphones most resistant to noise—the throat and temple microphones—the performance decline is less significant. In contrast, the in-ear microphones, affected by acoustic sealing issues, exhibit a more substantial drop. It is noteworthy that raw temple and throat signals outperform those captured by the noise-resilient airborne microphone—even without bandwidth extension or denoising—demonstrating their potential for communication in such conditions. The raw throat signal outperforms the headset microphone signal exclusively in the speaker verification context, likely due to the low-pass filtering properties related to speaker anatomy, which may facilitate speaker identification, while the high-frequency components are more crucial for accurate phoneme recognition.

<footnote n="4">⁴ Available at https://huggingface.co/Jenthe/ECAPA2.</footnote>
<figure><img src="image_11.png" /><figcaption>Fig. 11. EER on speech-clean and synthetic speech-noisy with the speaker verification model on pairs of the same audio sensor.</figcaption></figure>

# 5. Discussion and future work

Building the Vibravox dataset has been a protracted undertaking that has necessitated the achievement of several milestones. These include the construction of equipment, the development of necessary software, the recording of participants, the filtering of data, and other related tasks. This highly curated dataset, comprising four subsets—speech-clean, speech-noisy, speechless-clean, and speechless-noisy — is now publicly available. The total duration of the 6-channel recordings is over 45 h per sensor, with 188 participants. This corpus is being made available online on Hugging-Face via Creative Commons Attribution 4.0 International license. Using Vibravox has proven to yield satisfactory results in Speech Enhancement, Speech-to-Phonemes and Speaker Verification, which are three fundamental speech processing tasks. The EBEN models used for speech enhancement demonstrated the capacity to enhance objective metrics such as Noresqa-MOS and STOI while simultaneously improving phoneme transcription on every sensor. However, it was found to have a detrimental impact on speaker verification performance. The aforementioned tasks further contributed to a deeper understanding of the distinctive acoustic subtleties of the various audio sensors and their respective placements.

We hope this release will drive progress in body-conducted speech analysis and support the development of robust communication systems for real-world applications.

# Ethical considerations

In accordance with the guidelines of the General Data Protection Regulation (GDPR), which governs the processing and protection of personal data of citizens and residents of the European Union (EU), all necessary measures were taken to protect the privacy and rights of participants in this study. This included obtaining informed consent, clearly communicating the purpose of data collection, and ensuring anonymization and encryption of data during storage and analysis. In addition, the use of Wikipedia text ensures compliance with copyright and licensing issues. Finally, to increase the representativeness and inclusivity of the dataset, a deliberate effort was made to recruit a diverse and gender-balanced group of participants.

# CRediT authorship contribution statement

**Julien Hauret:** Writing – review & editing, Writing – original draft, Visualization, Validation, Software, Resources, Project administration, Methodology, Investigation, Formal analysis, Data curation, Conceptualization.
**Malo Olivier:** Writing – review & editing, Investigation, Data curation.
**Thomas Joubaud:** Writing – review & editing, Software,

Methodology, Investigation, Funding acquisition, Conceptualization.
**Christophe Langrenne:** Writing – review & editing, Resources.
**Sarah Poirée:** Writing – review & editing, Resources.
**Véronique Zimpfer:** Writing – review & editing, Supervision, Funding acquisition.
**Éric Bavu:** Writing – review & editing, Visualization, Validation, Supervision, Software, Resources, Project administration, Methodology, Investigation, Funding acquisition, Formal analysis, Data curation, Conceptualization.

# Declaration of competing interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

# Declaration of Generative AI and AI-assisted technologies in the writing process

In the preparation of this work, the authors utilized QuillBot's Grammar Checker, Mistral's LeChat and Open AI'S ChatGPT to improve the quality of the English language in the manuscript. Following the use of these tools, the authors thoroughly reviewed and edited the content as necessary and assume full responsibility for the final version of the publication.

# Acknowledgments

The authors would like to express their gratitude to the 200 participants in the Vibravox measurement protocol, whose invaluable contributions made this research possible. We would also like to thank Jean-Baptiste Doc for his assistance in developing 3D-printed components that facilitated sensor positioning for participants with diverse morphologies and Philippe Chenevez for his expertise in electrical engineering for the design of pre-amplifier circuits. This work was granted access to the HPC/AI resources of [CINES/IDRIS/TGCC] under the grant 2022-AD011013469 awarded by GENCI and partially funded by the French National Research Agency under the ANR Grant No. ANR-20-THIA-0002.

# Appendix A. Dataset statistics

See Table 7

# Appendix B. Speech enhancement training curves on the speech-clean subset

See Fig. 12.

# Appendix C. Speech-to-phonemes training curves on the speech-clean subset

See Fig. 13.

# Appendix D. Coherence function statistics between body-conducted and airborne microphones on speech-clean

See Fig. 14.

# Appendix E. Power spectral density of microphones on speechless-clean

See Fig. 15.

# Appendix F. Speaker verification EER on the speech-clean subset

See Figs. 16 and 17
<table><caption>**Table 7**
Speakers age balance.</caption><thead><tr><th>Gender</th><th>Mean age (years)</th><th>Median age (years)</th><th>Min age (years)</th><th>Max age (years)</th></tr></thead><tbody><tr><td>Female</td><td>25.9</td><td>22</td><td>19</td><td>59</td></tr><tr><td>Male</td><td>31.4</td><td>27</td><td>18</td><td>82</td></tr><tr><td>All</td><td><strong>28.55</strong></td><td><strong>25</strong></td><td><strong>18</strong></td><td><strong>82</strong></td></tr></tbody></table>

<figure><img src="image_12.png" /><figcaption>**Fig. 12.** EBEN validation curves during training on the speech-clean subset.</figcaption></figure>

<figure><img src="image_13.png" /><figcaption>**Fig. 13.** Wav2vec2 validation curves during training on the speech-clean subset.</figcaption></figure>

## Appendix G. Speaker verification min-DCF

See Figs. 18 and 19

## Data availability

Dataset is linked here: https://huggingface.co/datasets/Cnam-LMS SC/vibravox.
<figure><img src="image_14.png" /><figcaption>Fig. 14. Median coherence function per sensor on speech-clean given with their 25%-75% IQR and a single participant.</figcaption></figure>
<figure><img src="image_15.png" /><figcaption>**Fig. 15.** Median power spectral density function per sensor on speechless-clean given with their 25%–75% IQR for all participants.</figcaption></figure>
<figure><img src="image_16.png" /><figcaption>Fig. 16. EER obtained with the speaker verification model for sensor pairs on speech-clean subset.</figcaption></figure>

<figure><img src="image_17.png" /><figcaption>Fig. 17. EER obtained with the speaker verification model for sensor pairs enhanced by EBEN on speech-clean subset.</figcaption></figure>
<figure><img src="image_18.png" /><figcaption>Fig. 18. min-DCF obtained with the speaker verification model for sensor pairs on speech-clean subset.</figcaption></figure>

<figure><img src="image_19.png" /><figcaption>Fig. 19. min-DCF obtained with the speaker verification model for sensor pairs enhanced by EBEN on speech-clean subset.</figcaption></figure>

## References

Acker-Mills, B., Houtsma, A., Ahroon, W., 2005. Speech Intelligibility with Acoustic and Contact Microphones. Tech. Rep., Army Aeromedical Research Lab Fort Rucker Al.

Andreev, P., Alanov, A., Ivanov, O., Vetrov, D., 2023. HIFI++: A unified framework for bandwidth extension and speech enhancement. In: International Conference on Acoustics, Speech and Signal Processing. ICASSP, IEEE, pp. 1–5.

Ardila, R., Branson, M., Davis, K., Henretty, M., Kohler, M., Meyer, J., Morais, R., Saunders, L., Tyers, F.M., Weber, G., 2020. Common voice: A massively-multilingual speech corpus. Proc. the Twelfth Lang. Resour. Eval. Conf.

Baevski, A., Zhou, Y., Mohamed, A., Auli, M., 2020. Wav2vec 2.0: A framework for self-supervised learning of speech representations. Adv. Neural Inf. Process. Syst. 33, 12449–12460.

Bernard, M., Titeux, H., 2021. Phonemizer: Text to phones transcription for multiple languages in python. J. Open Source Softw. 6 (68), 3958. http://dx.doi.org/10.21105/joss.03958.

Björklund, S., Sundberg, J., 2016. Relationship between subglottal pressure and sound pressure level in untrained voices. J. Voice 30 (1), 15–20.

Blondé-Weinmann, C., Joubaud, T., Zimpfer, V., Hamery, P., Roth, S., 2023. Numerical and experimental investigation of the sound transmission delay from a skin vibration to the occluded ear canal. J. Sound Vib. 542, 117345.

Bos, J.C., Tack, D.W., Bossi, L.L., 2005. Speech input hardware investigation for future dismounted soldier computer systems. DRCD Tor. CR 64, 2005.

Bredin, H., 2023. Pyannote audio 2.1 speaker diarization pipeline: principle, benchmark, and recipe. In: Proc. Interspeech. ISCA, pp. 1983–1987.

Brydinskyi, V., Khoma, Y., Sabashko, D., Podpora, M., Khoma, V., Konovalov, A., Kostiak, M., 2024. Comparison of modern deep learning models for speaker verification. Appl. Sci. 14 (4).

Cao, Y., Li, F., Chen, H., Liu, X., Duan, C., Wang, Y., 2023. I can hear you without a microphone: Live speech eavesdropping from earphone motion sensors. In: IEEE INFOCOM 2023-IEEE Conference on Computer Communications. IEEE, pp. 1–10.
Che, Z., Wan, X., Xu, J., Duan, C., Zheng, T., Chen, J., 2024. Speaking without vocal folds using a machine-learning-assisted wearable sensing-actuation system. Nat. Commun. 15 (1), 1–11.

Chen, S., Wang, C., Chen, Z., Wu, Y., Liu, S., Chen, Z., Li, J., Kanda, N., Yoshioka, T., Xiao, X., et al., 2022. Wavlm: Large-scale self-supervised pre-training for full stack speech processing. IEEE J. Sel. Top. Signal Process. 16 (6), 1505–1518.

Crocker, M.J., 2007. Rating measures, descriptors, criteria, and procedures for determining human response to noise. Handb. Noise Vib. Control. 394–413.

Davis, S.K., Calamia, P.T., Murphy, W.J., Smalt, C.J., 2019. In-ear and on-body measurements of impulse-noise exposure. Int. J. Audiol. 58 (sup1), S49–S57.

Défossez, A., Synnaeve, G., Adi, Y., 2020. Real time speech enhancement in the waveform domain. In: Proc. Interspeech. ISCA.

Denk, F., Kollmeier, B., 2021. The hearpiece database of individual transfer functions of an in-the-ear earpiece for hearing device research. Acta Acust. 5 (2), 1–16.

Denk, F., Lettau, M., Schepker, H., Doclo, S., Roden, R., Blau, M., Wellmann, J., Kollmeier, B., et al., 2019. A one-size-fits-all earpiece with multiple microphones and drivers for hearing device research. In: International Conference on Headphone Technology. Audio Engineering Society.

Desplanques, B., Thienpontd, J., Demuynck, K., 2020. ECAPA-TDNN: Emphasized Channel Attention, Propagation and Aggregation in TDNN Based Speaker Verification. In: Proc. Interspeech. ISCA, pp. 3830–3834.

Edraki, A., Chan, W.-Y., Jensen, J., Fogerty, D., 2024. Speaker adaptation for enhancement of bone-conducted speech. In: International Conference on Acoustics, Speech and Signal Processing. ICASSP, IEEE, pp. 10456–10460.

Erzin, E., 2009. Improving throat microphone speech recognition by joint analysis of throat and acoustic microphone recordings. IEEE Trans. Audio, Speech, Lang. Process. 17 (7), 1316–1324.

ESMB-corpus, 2021. Elevoc simultaneously-recorded microphone/bone-sensor. (Accessed 28 October 2023). [Online]. Available: https://github.com/elevoctech/ESMB-corpus.

Everett, S.S., 1985. Automatic Speaker Recognition for Military Applications: Applications Survey and Operational Requirements. Tech. Rep., Naval Research Lab. Washington DC.

Falcon, W., team, P.L., 2019, e1e8770, PyTorch Lightning. [Online]. Available: https://www.pytorchlightning.ai.

Gao, Y., Jin, Y., Chauhan, J., Choi, S., Li, J., Jin, Z., 2021. Voice in ear: Spoofing-resistant and passphrase-independent body sound authentication. Proc. the ACM Interact. Mob. Wearable Ubiquitous Technol. 5 (1), 1–25.

Gemmeme, J.F., Ellis, D.P., Freedman, D., Jansen, A., Lawrence, W., Moore, R.C., Plakal, M., Ritter, M., 2017. Audio set: An ontology and human-labeled dataset for audio events. In: 2017 IEEE International Conference on Acoustics, Speech and Signal Processing. ICASSP, IEEE, pp. 776–780.

Giorgino, T., 2009. Computing and visualizing dynamic time warping alignments in R: The dtw package. J. Stat. Softw. 31 (7).

Goodfellow, I., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., Courville, A., Bengio, Y., 2014. Generative adversarial nets. Adv. Neural Inf. Process. Syst. 27.

Graciarena, M., Franco, H., Sonmez, K., Bratt, H., 2003. Combining standard and throat microphones for robust speech recognition. IEEE Signal Process. Lett. 10 (3), 72–74.

Graves, A., Fernández, S., Gomez, F., Schmidhuber, J., 2006. Connectionist temporal classification: labelling unsegmented sequence data with recurrent neural networks. In: Proceedings of the 23rd International Conference on Machine Learning. pp. 369–376.

Gu, A., Dao, T., 2023. Mamba: Linear-time sequence modeling with selective state spaces. arXiv preprint arXiv:2312.00752.

Han, S., Lee, J., 2022. NU-Wave 2: A general neural audio upsampling model for various sampling rates. In: Proc. Interspeech. ISCA, pp. 4401–4405.

Hauret, J., Joubaud, T., Zimpfer, V., Bavu, É., 2023. Configurable EBEN: Extreme bandwidth extension network to enhance body-conducted speech capture. IEEE/ACM Trans. Audio, Speech, Lang. Process.

Hauret, J., Joubaud, T., Zimpfer, V., Bavu, É., 2023. EBEN: Extreme bandwidth extension network applied to speech signals captured with noise-resilient body-conduction microphones. In: International Conference on Acoustics, Speech and Signal Processing. ICASSP, IEEE, pp. 1–5.

Hauret, J., Olivier, M., Langrenne, C., Poirée, S., Bavu, É., 2024. Vibravox (revision 071945b). [Online]. Available: https://huggingface.co/datasets/Cnam-LMSSC/vibravox.

He, L., Hou, H., Shi, S., Shuai, X., Yan, Z., 2023. Towards bone-conducted vibration speech enhancement on head-mounted wearables. In: Proceedings of the 21st Annual International Conference on Mobile Systems, Applications and Services. pp. 14–27.

Hosain, M.S., Sugiura, Y., Rahman, M.S., Shimamura, T., 2024. EmoBone: A multi-national audio dataset of emotional bone conducted speech. IEEJ Trans. Electr. Electron. Eng. e24110.

Hwang, J., Hira, M., Chen, C., Zhang, X., Ni, Z., Sun, G., Ma, P., Huang, R., Pratap, V., Zhang, Y., Kumar, A., Yu, C.-Y., Zhu, C., Liu, C., Kahn, J., Ravanelli, M., Sun, P., Watanabe, S., Shi, Y., Tao, Y., 2023. TorchAudio 2.1: Advancing speech recognition, self-supervised learning, and audio processing components for pytorch. In: Automatic Speech Recognition and Understanding Workshop. ASRU, IEEE, pp. 1–9.

Ishimitsu, S., 2010. Body-conducted speech recognition and its application to speech support system. SCIYO. COM 51.

Iwamoto, K., Ochiai, T., Delcroix, M., Ikeshita, R., Sato, H., Araki, S., Katagiri, S., 2022. How bad are artifacts?: Analyzing the impact of speech enhancement errors on ASR. Proc. Interspeech.

Jensen, J., Taal, C.H., 2016. An algorithm for predicting the intelligibility of speech masked by modulated noise maskers. IEEE/ ACM Trans. Audio, Speech, Lang. Process. 24 (11), 2009–2022.

Karthikeyan, C., Kumar, T.R., Babu, D.V., Baskar, M., Jayaraman, R., Shahid, M., 2023. Speech enhancement approach for body-conducted unvoiced speech based on Taylor-Boltzschung machines trained DNN. Comput. Speech & Lang. 83, 101549.

Kim, Y., Song, Y., 2025. TAPS: Throat and acoustic paired speech dataset for deep learning-based speech enhancement. arXiv preprint arXiv:2502.11478.

Koluguri, N.R., Park, T., Ginsburg, B., 2022. Titanet: Neural model for speaker representation with 1d depth-wise separable convolutions and global context. In: International Conference on Acoustics, Speech and Signal Processing. ICASSP, IEEE, pp. 8102–8106.

Kuleshov, V., Enam, S.Z., Ermon, S., 2017. Audio super-resolution using neural nets. In: Proceedings of International Conference on Learning Representations. ICLR.

Kumar, A., Tan, K., Ni, Z., Manocha, P., Zhang, X., Henderson, E., Xu, B., 2023. Torchaudio-squim: Reference-less speech quality and intelligibility measures in torchaudio. In: International Conference on Acoustics, Speech and Signal Processing. ICASSP, IEEE, pp. 1–5.

Lecomte, P., Gauthier, P.-A., 2015. Real-time 3D ambisonics using faust, processing, pure data, and OSC. In: 18th International Conference on Digital Audio Effects, DAFx-15.

Lecomte, P., Gauthier, P.-A., Langrenne, C., Berry, A., Garcia, A., 2016. A fifty-node Lebedev grid and its applications to ambisonics. J. Audio Eng. Soc. 64 (11), 868–881.

Lee, S., Kim, J., Yun, I., Bae, G.Y., Kim, D., Park, S., Yi, I.-M., Moon, W., Chung, Y., Cho, K., 2019. An ultrathin conformable vibration-responsive electronic skin for quantitative vocal recognition. Nat. Commun. 10 (1), 2468.

Lee, S., Roh, H., Kim, J., Chung, S., Seo, D., Moon, W., Cho, K., 2022. An electret-powered skin-attachable auditory sensor that functions in harsh acoustic environments. Adv. Mater. 34 (40), 2205537.

Levenshtein, V.I., et al., 1966. Binary codes capable of correcting deletions, insertions, and reversals. In: Soviet Physics Doklady. vol. 10 (8), Soviet Union, pp. 707–710.

Lhoest, Q., Villanova del Moral, A., Jernite, Y., Thakur, A., von Platen, P., Patil, S., Chaumond, J., Drame, M., Plu, J., Tunstall, L., Davison, J., Šaško, M., Chhablani, G., Malik, B., Brandeis, S., Le Scao, T., Sanh, V., Xu, C., Patry, N., McMillan-Major, A., Schmid, P., Gugger, S., Delangue, C., Matusière, T., Debut, L., Bekman, S., Cistac, P., Goehringer, T., Mustar, V., Lagunas, F., Rush, A., Wolf, T., 2021. Datasets: A community library for natural language processing. In: Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing: System Demonstrations. Association for Computational Linguistics, Online and Punta Cana, Dominican Republic, pp. 175–184, [Online]. Available: https://aclanthology.org/2021.emnlp-demo.21.

Li, C., Yang, F., Yang, J., 2023a. Restoration of bone-conducted speech with U-net-like model and energy distance loss. IEEE Signal Process. Lett.

Li, C., Yang, F., Yang, J., 2023b. A two-stage approach to quality restoration of bone-conducted speech. IEEE/ ACM Trans. Audio, Speech, Lang. Process.

Liebich, S., Richter, J.-G., Fabry, J., Durand, C., Fels, J., Jax, P., 2018. Direction-of-arrival dependency of active noise cancellation headphones. In: Noise Control and Acoustics Division Conference. vol. 51425, American Society of Mechanical Engineers, V001T08A003.

Liu, H., Chen, K., Tian, Q., Wang, W., Plumbley, M.D., 2024. AudioSR: Versatile audio super-resolution at scale. In: International Conference on Acoustics, Speech and Signal Processing. ICASSP, IEEE, pp. 1076–1080.

Liu, R., Cornelius, C., Rawassizadeh, R., Peterson, R., Kotz, D., 2018. Vocal resonance: Using internal body voice for wearable authentication. Proc. the ACM Interact. Mob. Wearable Ubiquitous Technol. 2 (1), 1–23.

Liu, Z., Lin, Y., Cao, Y., Hu, H., Wei, Y., Zhang, Z., Lin, S., Guo, B., 2021. Swin transformer: Hierarchical vision transformer using shifted windows. In: Proceedings of the IEEE/CVF International Conference on Computer Vision. pp. 10012–10022.

Lozhkov, A., Ben Allal, L., von Werra, L., Wolf, T., 2024. FineWeb-edu. [Online]. Available: https://huggingface.co/datasets/HuggingFaceFW/fineweb-edu.

Mandel, M., Tal, O., Adi, Y., 2023. AERO: Audio super resolution in the spectral domain. In: International Conference on Acoustics, Speech and Signal Processing. ICASSP, IEEE, pp. 1–5.

Manocha, P., Kumar, A., 2022. Speech quality assessment through MOS using non-matching references. In: Proc. Interspeech. ISCA, [Online]. Available: https://arxiv.org/abs/2206.12285.

McBride, M., Tran, P., Letowski, T., Patrick, R., 2011. The effect of bone conduction microphone locations on speech intelligibility and sound quality. Appl. Ergon. 42 (3), 495–502.

Nguyen, T.Q., 1994. Near-perfect-reconstruction pseudo-QMF banks. IEEE Trans. Signal Process. 42 (1), 65–76.

Nisticò, S., Palopoli, L., Romano, A.P., 2023. Audio super-resolution via vision transformer. J. Intell. Inf. Syst. 1–15.
NVIDIA, 2024. Canary-1B. https://huggingface.co/nvidia/canary-1b. (Accessed 24 April 2024).

Ohlenbusch, M., Rollwage, C., Doclo, S., 2024. Speech-dependent data augmentation for own voice reconstruction with hearable microphones in noisy environments. arXiv preprint arXiv:2405.11592.

Ohlenbusch, M., Rollwage, C., Doclo, S., 2024a. Multi-microphone noise data augmentation for DNN-based own voice reconstruction for hearables in noisy environments. In: International Conference on Acoustics, Speech and Signal Processing. ICASSP, IEEE, pp. 416-420.

Ohlenbusch, M., Rollwage, C., Doclo, S., 2024b. Modeling of speech-dependent own voice transfer characteristics for hearables with in-ear microphones. Acta Acust. 8 (28), 1-13.

Oord, A.v.d., Li, Y., Vinyals, O., 2018. Representation learning with contrastive predictive coding. arXiv preprint arXiv:1807.03748.

Paszke, A., Gross, S., Massa, F., Lerer, A., Bradbury, J., Chanan, G., Killeen, T., Lin, Z., Gimelshein, N., Antiga, L., et al., 2019. Pytorch: An imperative style, high-performance deep learning library. Adv. Neural Inf. Process. Syst. 32.

Radford, A., Kim, J.W., Xu, T., Brockman, G., McLeavey, C., Sutskever, I., 2023. Robust speech recognition via large-scale weak supervision. In: International Conference on Machine Learning. PMLR, pp. 28492-28518.

Reynolds, D.A., Campbell, W.M., 2008. Text-independent speaker recognition. In: Springer Handbook of Speech Processing. pp. 763-782.

Sahidullah, M., Gonzalez Hautamäki, R., Lehmann, T.D.A., Kinnunen, T., Tan, Z.-H., Hautamäki, V., Parts, R., Pitkänen, M., 2016. Robust speaker recognition with combined use of acoustic and throat microphone speech. In: Proc. Interspeech. ISCA.

Sahidullah, M., Thomsen, D.A.L., Hautamäki, R.G., Kinnunen, T., Tan, Z.-H., Parts, R., Pitkänen, M., 2017. Robust voice liveness detection and speaker verification using throat microphones. IEEE/ ACM Trans. Audio, Speech, Lang. Process. 26 (1), 44-56.

Scheibler, R., Fujita, Y., Shirahata, Y., Komatsu, T., 2024. Universal score-based speech enhancement with high content preservation. In: Proc. Interspeech. ISCA, pp. 1165-1169.

Schilk, P., Polvani, N., Ronco, A., Cernak, M., Magno, M., 2023. In-ear-voice: Towards milli-watt audio enhancement with bone-conduction microphones for in-ear sensing platforms. In: Proceedings of the 8th ACM/IEEE Conference on Internet of Things Design and Implementation. pp. 1-12.

Schwartz, J.C., Whyte, A.T., Al-Nuaimi, M., Donai, J.J., 2018. Effects of signal bandwidth and noise on individual speaker identification. J. Acoust. Soc. Am. 144 (5), EL447-EL452.

Serrà, J., Pascual, S., Pons, J., Araz, R.O., Scaini, D., 2022. Universal speech enhancement with score-based diffusion. arXiv preprint arXiv:2206.03065.

Shahina, A., Yegnanarayana, B., 2007. Mapping speech spectra from throat microphone to close-speaking microphone: A neural network approach. EURASIP J. Adv. Signal Process. 2007, 1-10.

Shan, D., Zhang, X., Zhang, C., Li, L., 2018. A novel encoder-decoder model via NS-LSTM used for bone-conducted speech enhancement. IEEE Access 6, 62638-62644.

Shang, J., Wu, J., 2019. Enabling secure voice input on augmented reality headsets using internal body voice. In: 16th Annual International Conference on Sensing, Communication, and Networking. SECON, IEEE, pp. 1-9.

Shuai, C., Shi, C., Gan, L., Liu, H., 2023. mdctGAN: Taming transformer-based GAN for speech super-resolution with Modified DCT spectra. In: Proc. Interspeech. ISCA, pp. 5112-5116.

Steinmetz, C.J., Reiss, J.D., 2020. Auraloss: Audio focused loss functions in PyTorch. In: Digital Music Research Network One-Day Workshop (DMRN+15).

Subakan, C., Ravanelli, M., Cornell, S., Bronzi, M., Zhong, J., 2021. Attention is all you need in speech separation. In: International Conference on Acoustics, Speech and Signal Processing. ICASSP, IEEE, pp. 21-25.

Sui, Y., Zhao, M., Xia, J., Jiang, X., Xia, S., 2024. TRAMBA: A hybrid transformer and mamba architecture for practical audio and bone conduction speech super resolution and enhancement on mobile and wearable platforms. Proc. ACM Interact. Mob. Wearable Ubiquitous Technol. 8 (4), 1-29. http://dx.doi.org/10.1145/3699757.

Taal, C.H., Hendriks, R.C., Heusdens, R., Jensen, J., 2010. A short-time objective intelligibility measure for time-frequency weighted noisy speech. In: International Conference on Acoustics, Speech and Signal Processing. ICASSP, IEEE, pp. 4214-4217.

Tagliasacchi, M., Li, Y., Misiunas, K., Roblek, D., 2020. SEANet: A multi-modal speech enhancement network. In: Proc. Interspeech. ISCA, pp. 1126-1130.

Tan, X., Qin, T., Jian, J., Liu, T.-Y., Bengio, Y., 2024. Regeneration learning: A learning paradigm for data generation. In: Proceedings of the AAAI Conference on Artificial Intelligence. vol. 38, (20), pp. 22614-22622.

Thienpontd, J., Demuynck, K., 2023. ECAPA2: A hybrid neural network architecture and training strategy for robust speaker embeddings. In: Automatic Speech Recognition and Understanding Workshop. ASRU, IEEE, pp. 1-8.

Turan, M.T., Erzin, E., 2015. Source and filter estimation for throat-microphone speech enhancement. IEEE/ ACM Trans. Audio, Speech, Lang. Process. 24 (2), 265-275.

US National Institute of Standards and Technology, 2018. NIST 2018 Speaker Recognition Evaluation Plan. [Online]. Available: https://www.nist.gov/system/files/documents/2018/08/17/sre18_eval_plan_2018-05-31_v6.pdf.

Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, Ł., Polosukhin, I., 2017. Attention is all you need. Adv. Neural Inf. Process. Syst. 30.

Wang, M., Chen, J., Zhang, X.-L., Rahardja, S., 2022a. End-to-end multi-modal speech recognition on an air and bone conducted speech corpus. IEEE/ ACM Trans. Audio, Speech, Lang. Process. 1-12.

Wang, D., Liu, S., Wu, X., Lu, H., Sun, L., Liu, X., Meng, H., 2022b. Speaker identity preservation in dysarthric speech reconstruction by adversarial speaker adaptation. In: International Conference on Acoustics, Speech and Signal Processing. ICASSP, IEEE, pp. 6677-6681.

Wang, C., Riviere, M., Lee, A., Wu, A., Talnikar, C., Haziza, D., Williamson, M., Pino, J., Dupoux, E., 2021. VoxPopuli: A large-scale multilingual speech corpus for representation learning, semi-supervised learning and interpretation. In: Zong, C., Xia, F., Li, W., Navigli, R. (Eds.), Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers). Association for Computational Linguistics, Online, pp. 993-1003, [Online]. Available: https://aclanthology.org/2021.acl-long.80.

Yadan, O., 2019. Hydra - a framework for elegantly configuring complex applications. Github. [Online]. Available: https://github.com/facebookresearch/hydra.

Yamamoto, R., Song, E., Kim, J.-M., 2020. Parallel WaveGAN: A fast waveform generation model based on generative adversarial networks with multi-resolution spectrogram. In: International Conference on Acoustics, Speech and Signal Processing. ICASSP, IEEE, pp. 6199-6203.

Yu, G., Zheng, X., Li, N., Han, R., Zheng, C., Zhang, C., Zhou, C., Huang, Q., Yu, B., 2024. BAE-Net: A low complexity and high fidelity bandwidth-adaptive neural network for speech super-resolution. In: International Conference on Acoustics, Speech and Signal Processing. ICASSP, IEEE, pp. 571-575.

Zhang, K., Ren, Y., Xu, C., Zhao, Z., 2021. WSRGlow: A glow-based waveform generative model for audio super-resolution. In: Proc. Interspeech. ISCA, pp. 1649-1653.

Zheng, Y., Liu, Z., Zhang, Z., Sinclair, M., Droppo, J., Deng, L., Acero, A., Huang, X., 2003. Air-and bone-conductive integrated microphones for robust speech detection and enhancement. In: Workshop on Automatic Speech Recognition and Understanding (IEEE Cat. No. 03EX721). IEEE, pp. 249-254.

Zheng, C., Xu, L., Fan, X., Yang, J., Fan, J., Huang, X., 2022. Dual-path transformer-based network with equalization-generation components prediction for flexible vibrational sensor speech enhancement in the time domain. J. Acoust. Soc. Am. 151 (5), 2814-2825.

<figure><img src="image_20.png" /></figure>

Julien Hauret is a Ph.D. candidate at Cnam Paris, pursuing research in machine learning applied to speech processing. He holds two M.Sc. degrees from ENS Paris Saclay, one in Electrical Engineering (2020) and the other in Applied Mathematics (2021). His research training is evidenced by his experiences at Columbia University, the French Ministry of the Armed Forces and the Pulse Audit start-up. Additionally, he has lectured for two consecutive years on algorithms and data structures at the École des Ponts ParisTech. His research focuses on the use of deep learning for speech enhancement applied to body-conducted speech. With a passion for interdisciplinary collaboration, Julien aims to improve human communication through technology.

<figure><img src="image_21.png" /></figure>

Malo Olivier completed an internship in the LMSSC lab at Cnam Paris during his studies at INSA Lyon. He graduated from the Computer Science Department at INSA Lyon in June 2024 and he now works at Cnam as a research engineer on an experimental project. With skills in implementing solutions—from information systems challenges to deep neural network architectures, including web applications—he foresees pursuing a Ph.D. in Artificial Intelligence, specializing in deep neural networks applied to scientific fields, hoping his engineering profile highlights his implementation abilities in high-impact projects.

<figure><img src="image_22.png" /></figure>

Thomas Joubaud is a Research Associate at the Acoustics and Soldier Protection department within the French-German Research Institute of Saint-Louis (ISL), France, since 2019. In 2013, he received the graduate degree from Ecole Centrale Marseille, France, as well as the master's degree in Mechanics, Physics and Engineering, specialized in Acoustical Research, of the Aix-Marseille University, France. He earned the Ph.D. degree in Mechanics, specialized in Acoustics, of the Conservatoire National des Arts et Métiers (Cnam), Paris, France, in 2017. The thesis was carried out in collaboration with and within the ISL. From 2017 to 2019, he worked as a post-doctorate research engineer with
Orange SA company in Cesson-Sévigné, France. His research interests include audio signal processing, hearing protection, psychoacoustics, especially speech intelligibility and sound localization, and high-level continuous and impulse noise measurement.

<figure><img src="image_23.png" /></figure>

**Christophe Langrenne** is a Research Engineer at the Laboratoire de Mécanique des Structures et des Systèmes Couplés (LMSSC) at the Conservatoire National des Arts et Métiers (Cnam), Paris, France. After completing his Ph.D. on the regularization of inverse problems, he developed a fast multipole method (FMM) algorithm for solving large-scale scattering and propagation problems. Also interested in 3D audio, he co-supervised 3 Ph.D. students on this theme, in particular on Ambisonic (recording and decoding) and binaural restitution (front/back confusions).

<figure><img src="image_24.png" /></figure>

**Sarah Poiree** is a technician at the Laboratoire de Mécanique et des Systèmes Couplés (LMSSC) within the Conservatoire National des Arts et Métiers (Cnam), Paris, France. Her activities focus on the design and development of experimental setups. Notably, she contributed to the creation of the 3D sound spatialization system used during the recording of the Vibravox dataset.

<figure><img src="image_25.png" /></figure>

**Véronique Zimpfer** is a Scientific Researcher at the Acoustics and Soldier Protection department within the French-German Research Institute of Saint-Louis (ISL), Saint-Louis, France, since 1997. She holds a M.Sc in Signal Processing from the Grenoble INP, France and obtained a Ph.D. in Acoustics from INSA Lyon, France, in 2000. Her expertise lies at the intersection of communication in noisy environments and auditory protection. Her research focuses on improving adaptive auditory protectors, refining radio communication strategies through unconventional microphone methods, and enhancing auditory perception while utilizing protective gear.

<figure><img src="image_26.png" /></figure>

**Éric Bavu** is a Full Professor of Acoustics and Signal Processing at the Laboratoire de Mécanique des Structures et des Systèmes Couplés (LMSSC) within the Conservatoire National des Arts et Métiers (Cnam), Paris, France. He completed his undergraduate studies at École Normale Supérieure de Cachan, France, from 2001 to 2005. In 2005, he earned an M.Sc in Acoustics, Signal Processing, and Computer Science Applied to Music from Université Pierre et Marie Curie Sorbonne University (UPMC), followed by a Ph.D. in Acoustics jointly awarded by Université de Sherbrooke, Canada, and UPMC, France, in 2008. He also conducted post-doctoral research on biological soft tissues imaging at the Langevin Institute at École Supérieure de Physique et Chimie ParisTech (ESPCI), France. Since 2009, he has supervised 8 Ph.D. students at LMSSC, focusing on time domain audio signal processing for inverse problems, 3D audio, and deep learning for audio. His current research interests encompass deep learning methods applied to inverse problems in acoustics, moving sound source localization and tracking, speech enhancement, and speech recognition.