TC commited on
Commit Β·
9833bf0
1
Parent(s): 83110b3
Update README with demo link, fix file names and sizes
Browse files
README.md
CHANGED
|
@@ -10,83 +10,87 @@ tags:
|
|
| 10 |
- streaming
|
| 11 |
- q4_0
|
| 12 |
- gguf
|
|
|
|
| 13 |
base_model: kyutai/stt-1b-en_fr
|
| 14 |
---
|
| 15 |
|
| 16 |
-
|
| 17 |
|
| 18 |
-
|
| 19 |
|
| 20 |
-
|
| 21 |
|
| 22 |
-
|
| 23 |
|
| 24 |
-
|
| 25 |
-
|------|------|-------------|
|
| 26 |
-
| `stt-1b-en_fr-q4.gguf` | 531 MB | STT transformer weights (Q4_0 quantized) |
|
| 27 |
-
| `mimi.safetensors` | 367 MB | Mimi audio codec weights (f32) |
|
| 28 |
-
| `tokenizer.model` | 118 KB | SentencePiece tokenizer (32k vocab, EN+FR) |
|
| 29 |
|
| 30 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
| 31 |
|
| 32 |
-
|
| 33 |
|
| 34 |
-
|
| 35 |
-
import { SttClient } from './stt-client.js';
|
| 36 |
|
| 37 |
-
|
| 38 |
-
|
| 39 |
-
shardList: ['stt-1b-en_fr-q4.gguf'],
|
| 40 |
-
mimiUrl: 'https://huggingface.co/idle-intelligence/stt-1b-en_fr-q4-webgpu/resolve/main/mimi.safetensors',
|
| 41 |
-
tokenizerUrl: 'https://huggingface.co/idle-intelligence/stt-1b-en_fr-q4-webgpu/resolve/main/tokenizer.model',
|
| 42 |
-
onTranscript: (text) => console.log(text),
|
| 43 |
-
});
|
| 44 |
|
| 45 |
-
|
| 46 |
-
|
|
|
|
|
|
|
| 47 |
|
|
|
|
|
|
|
| 48 |
```
|
| 49 |
|
| 50 |
-
|
| 51 |
|
| 52 |
-
|
| 53 |
-
- HTTPS (required for WebGPU)
|
| 54 |
-
- ~1 GB download on first load (cached by browser afterward)
|
| 55 |
|
| 56 |
-
|
|
|
|
|
|
|
| 57 |
|
| 58 |
-
|
| 59 |
-
β Mimi codec [WASM, CPU] β 32 codebook tokens/frame at 12.5Hz
|
| 60 |
-
β STT transformer [WASM, WebGPU] β text tokens
|
| 61 |
-
β SentencePiece detokenizer β transcript
|
| 62 |
|
| 63 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 64 |
|
| 65 |
-
|
| 66 |
-
- Architecture: Decoder-only transformer with delayed-streams modeling
|
| 67 |
-
- Parameters: ~1B (STT) + ~25M (Mimi codec)
|
| 68 |
-
- Quantization: Q4_0 (4-bit) for STT transformer, f32 for Mimi codec
|
| 69 |
-
- Languages: English, French
|
| 70 |
-
- Streaming latency: ~500ms text delay (6 frames at 12.5Hz)
|
| 71 |
-
- License: CC-BY 4.0 (same as original)
|
| 72 |
|
| 73 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 74 |
|
| 75 |
-
|
| 76 |
-
CPU and is small enough not to benefit from quantization.
|
| 77 |
|
| 78 |
-
|
| 79 |
|
| 80 |
-
|
| 81 |
|
| 82 |
-
|
| 83 |
-
|
| 84 |
-
|
| 85 |
-
|
| 86 |
-
|
| 87 |
-
|
| 88 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
| 89 |
|
| 90 |
-
|
| 91 |
|
| 92 |
-
|
|
|
|
| 10 |
- streaming
|
| 11 |
- q4_0
|
| 12 |
- gguf
|
| 13 |
+
- burn
|
| 14 |
base_model: kyutai/stt-1b-en_fr
|
| 15 |
---
|
| 16 |
|
| 17 |
+
# STT 1B EN/FR β Q4 WebGPU
|
| 18 |
|
| 19 |
+
Q4-quantized weights for [kyutai/stt-1b-en_fr](https://huggingface.co/kyutai/stt-1b-en_fr), packaged for client-side browser inference via WASM + WebGPU.
|
| 20 |
|
| 21 |
+
Runs entirely in the browser β no server required. English + French, streaming, ~1B parameters.
|
| 22 |
|
| 23 |
+
[**Try the demo β**](https://idle-intelligence.github.io/stt-web/web/)
|
| 24 |
|
| 25 |
+
## Files
|
|
|
|
|
|
|
|
|
|
|
|
|
| 26 |
|
| 27 |
+
| File | Size | Description |
|
| 28 |
+
|------|------|-------------|
|
| 29 |
+
| `stt-1b-en_fr-q4_0.gguf` | 531 MB | STT transformer weights (Q4_0 quantized) |
|
| 30 |
+
| `mimi-encoder-f16.safetensors` | 107 MB | Mimi audio codec encoder (f16) |
|
| 31 |
+
| `tokenizer.model` | 118 KB | SentencePiece tokenizer (32k vocab, EN+FR) |
|
| 32 |
|
| 33 |
+
## Usage
|
| 34 |
|
| 35 |
+
These weights are consumed by [stt-web](https://github.com/idle-intelligence/stt-web), a Rust/WASM + WebGPU speech-to-text engine built with [Burn](https://burn.dev).
|
|
|
|
| 36 |
|
| 37 |
+
```js
|
| 38 |
+
import { SttClient } from './stt-client.js';
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 39 |
|
| 40 |
+
const stt = new SttClient({
|
| 41 |
+
onTranscript: (text, isFinal) => console.log(text),
|
| 42 |
+
onStatus: (text, ready) => console.log(text),
|
| 43 |
+
});
|
| 44 |
|
| 45 |
+
await stt.init();
|
| 46 |
+
await stt.startRecording();
|
| 47 |
```
|
| 48 |
|
| 49 |
+
Model weights are fetched from this repo automatically and cached by the browser.
|
| 50 |
|
| 51 |
+
## Requirements
|
|
|
|
|
|
|
| 52 |
|
| 53 |
+
- Chrome 113+ or Edge 113+ (WebGPU required)
|
| 54 |
+
- HTTPS (required for WebGPU)
|
| 55 |
+
- ~640 MB download on first load (cached afterward)
|
| 56 |
|
| 57 |
+
## Pipeline
|
|
|
|
|
|
|
|
|
|
| 58 |
|
| 59 |
+
```
|
| 60 |
+
Microphone β AudioWorklet (24kHz mono)
|
| 61 |
+
β Mimi codec [WASM, CPU] β 32 codebook tokens/frame at 12.5Hz
|
| 62 |
+
β STT transformer [WASM, WebGPU] β text tokens
|
| 63 |
+
β SentencePiece detokenizer β transcript
|
| 64 |
+
```
|
| 65 |
|
| 66 |
+
## Model Details
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 67 |
|
| 68 |
+
- **Base model:** [kyutai/stt-1b-en_fr](https://huggingface.co/kyutai/stt-1b-en_fr) by [Kyutai](https://kyutai.org)
|
| 69 |
+
- **Architecture:** Decoder-only transformer with delayed-streams modeling
|
| 70 |
+
- **Parameters:** ~1B (STT) + ~25M (Mimi codec encoder)
|
| 71 |
+
- **Quantization:** Q4_0 (4-bit) for STT transformer, f16 for Mimi codec
|
| 72 |
+
- **Languages:** English, French
|
| 73 |
+
- **Streaming latency:** ~500ms text delay (6 frames at 12.5Hz)
|
| 74 |
+
- **License:** CC-BY 4.0 (same as original)
|
| 75 |
|
| 76 |
+
## Quantization
|
|
|
|
| 77 |
|
| 78 |
+
The STT transformer weights were quantized from f32 to Q4_0 using a custom GGUF packer. Dequantization happens on-GPU via WGSL compute shaders at inference time. The Mimi codec encoder is stored at f16 as it runs on CPU via WASM.
|
| 79 |
|
| 80 |
+
## Citation
|
| 81 |
|
| 82 |
+
If you use this model, please cite the original authors:
|
| 83 |
+
|
| 84 |
+
```bibtex
|
| 85 |
+
@techreport{kyutai2024stt,
|
| 86 |
+
author = {Kyutai},
|
| 87 |
+
title = {Speech-To-Text models},
|
| 88 |
+
institution = {Kyutai},
|
| 89 |
+
year = {2024},
|
| 90 |
+
url = {https://huggingface.co/kyutai/stt-1b-en_fr},
|
| 91 |
+
}
|
| 92 |
+
```
|
| 93 |
|
| 94 |
+
## Disclaimer
|
| 95 |
|
| 96 |
+
This is an independent port by [idle intelligence](https://idleintelligence.org), not affiliated with or endorsed by Kyutai Labs. Transcription quality may differ from the original PyTorch implementation due to quantization.
|