You need to agree to share your Hugging Face information to access this model
This repository is publicly accessible, but you have to accept the conditions to access its files and content.
Prohibited use: Use of our model must comply with all applicable laws and regulations and must not result in, involve, or facilitate any illegal, harmful, deceptive, fraudulent, or unauthorized activity. Prohibited uses include, without limitation, voice impersonation or cloning without explicit and lawful consent; misinformation, disinformation, or deception (including fake news, fraudulent calls, or presenting generated content as genuine recordings of real people or events); and the generation of unlawful, harmful, libelous, abusive, harassing, discriminatory, hateful, or privacy-invasive content. We disclaim all liability for any non-compliant use.
Log in or Sign Up to review the conditions and access this model content.
Pocket TTS codec (Mimi)
The audio codec of Kyutai's Pocket TTS in 🤗 Transformers format (PocketTTSAudioTokenizerModel): Mimi's architecture with a continuous 32-dimensional bottleneck. It turns 24 kHz audio into latents at 12.5 Hz and back. It is the audio tokenizer of every Pocket TTS checkpoint's processor; its encoder is what clones a voice, so this repo is gated like kyutai/pocket-tts. Without the encoder: eustlb/pocket-tts-mimi-decoder-only.
Weights: the mimi.* tensors of kyutai/pocket-tts (identical across its checkpoints), converted with convert_pocket_tts_to_hf.py. Original code: kyutai-labs/pocket-tts.
from datasets import Audio, load_dataset
from transformers import AutoFeatureExtractor, AutoModel
model = AutoModel.from_pretrained("eustlb/pocket-tts-mimi", device_map="auto")
feature_extractor = AutoFeatureExtractor.from_pretrained("eustlb/pocket-tts-mimi")
dataset = load_dataset("hf-internal-testing/librispeech_asr_dummy", "clean", split="validation")
dataset = dataset.cast_column("audio", Audio(sampling_rate=24000))
inputs = feature_extractor(dataset[0]["audio"]["array"], sampling_rate=24000, return_tensors="pt").to(model.device)
latents = model.encode(**inputs).latents
audio_values = model.decode(latents).audio_values
See the Transformers docs.
Prohibited use
Use of our model must comply with all applicable laws and regulations and must not result in, involve, or facilitate any illegal, harmful, deceptive, fraudulent, or unauthorized activity. Prohibited uses include, without limitation, voice impersonation or cloning without explicit and lawful consent; misinformation, disinformation, or deception (including fake news, fraudulent calls, or presenting generated content as genuine recordings of real people or events); and the generation of unlawful, harmful, libelous, abusive, harassing, discriminatory, hateful, or privacy-invasive content. We disclaim all liability for any non-compliant use.
Authors
Manu Orsini*, Simon Rouard*, Gabriel De Marmiesse*, Václav Volhejn, Neil Zeghidour, Alexandre Défossez
*equal contribution
- Downloads last month
- -
Model tree for eustlb/pocket-tts-mimi
Base model
kyutai/pocket-tts