README.md

---
license: mit
language: fr
datasets:
- Cnam-LMSSC/vibravox
tags:
- audio
- audio-to-audio
- speech
---
# Master Model Card: Vibravox Audio Bandwidth extension Models

<p align="center">
  <img src="https://cdn-uploads.huggingface.co/production/uploads/65302a613ecbe51d6a6ddcec/zhB1fh-c0pjlj-Tr4Vpmr.png" style="object-fit:contain; width:280px; height:280px;" >
</p>

## Overview

This master model card serves as an entry point for exploring [multiple **audio bandwidth extension** (BWE) models](https://huggingface.co/Cnam-LMSSC/vibravox_EBEN_models#available-models) trained on different sensor data from the [Vibravox dataset](https://huggingface.co/datasets/Cnam-LMSSC/vibravox). 

These models are designed to to enhance the audio quality of body-conducted captured speech, by denoising and regenerating mid and high frequencies from low frequency content only.

The models are trained on specific sensors to address various audio capture scenarios using **body conducted** sound and vibration sensors.

## Disclaimer
Each of these models has been trained for **specific non-conventional speech sensors** and is intended to be used with **in-domain data**.

Please be advised that using these models outside their intended sensor data may result in suboptimal performance.

## Usage
All models are trained using [Configurable EBEN](https://github.com/jhauret/vibravox/blob/main/vibravox/torch_modules/dnn/eben_generator.py) (see [publication in IEEE TASLP](https://ieeexplore.ieee.org/document/10244161) - [arXiv link](https://arxiv.org/abs/2303.10008)) and adapted to different sensor inputs. They are intended to be used at a sample rate of 16kHz. 

## Training Procedure
Detailed instructions for reproducing the experiments are available on the [jhauret/vibravox](https://github.com/jhauret/vibravox) Github repository and in the [VibraVox paper on arXiV](https://arxiv.org/abs/2407.11828).

## Available Models

The following models are available, **each trained on a different sensor** on the `speech_clean` subset of (https://huggingface.co/datasets/Cnam-LMSSC/vibravox):

| **Transducer**                 | **Huggingface model link**  | **EBEN configuration** |
|:---------------------------|:---------------------|:---------------------|
| In-ear comply foam-embedded microphone |[EBEN_soft_in_ear_microphone](https://huggingface.co/Cnam-LMSSC/EBEN_soft_in_ear_microphone) | M=4,P=2,Q=4 |
| In-ear rigid earpiece-embedded microphone | [EBEN_rigid_in_ear_microphone](https://huggingface.co/Cnam-LMSSC/EBEN_rigid_in_ear_microphone) | M=4,P=2,Q=4 |
| Forehead miniature vibration sensor | [EBEN_forehead_accelerometer](https://huggingface.co/Cnam-LMSSC/EBEN_forehead_accelerometer) | M=4,P=4,Q=4 |
| Temple vibration pickup | [EBEN_temple_vibration_pickup](https://huggingface.co/Cnam-LMSSC/EBEN_temple_vibration_pickup) | M=4,P=1,Q=4 |
| Laryngophone | [EBEN_throat_microphone](https://huggingface.co/Cnam-LMSSC/EBEN_throat_microphone) | M=4,P=2,Q=4 |