README.md

# Streamalign (R16)

Complete R16 speech tokenizer stack. Paired SLM: `Streamalign-SLM-R16`
(reported metrics: SALMon 69.1, StoryCloze 72.1).

## Contents & roles

| Path | Role |
|------|------|
| `rvq_teacher/` | The acoustic tokenizer: the streaming encoder plus the R16 residual-VQ quantizer. Produces the R16 speech units. R16-specific. |
| `alignment_model/` | Char-level streaming Conformer-Transducer. Base scaffold the tokenizer is built on; supplies the RNN-T predictor/joiner for char-level alignment. R-independent. |
| `streaming_asr/` | Word-level streaming Conformer-Transducer. Generates the chunk TextGrids (word alignment) and drives the boundary classifier. R-independent. |
| `boundary_classifier/` | Word-boundary detector for streaming chunking; consumes `streaming_asr`. R-independent. |
| `alignment.yaml` | Extractor / `alignment_model` hparams. |

## Pipeline
audio -> `streaming_asr` (word chunks + TextGrids) + `boundary_classifier`
      -> `alignment_model` scaffold + `rvq_teacher` encoder -> R16 RVQ units.

Only `rvq_teacher` is R16-specific; the ASR / alignment / boundary
components are shared across R8/R16/R32.

## Tokenizer checkpoint

The checkpoint under `rvq_teacher/` is the final R16 tokenizer, trained on
LibriSpeech plus the full Emilia set.

Load it with `RVQ_R=16` and `RVQ_CODEBOOK_SIZE=512`, and leave
`RVQ_CODEBOOK_DIM` unset: `codebook_dim` defaults to `feat_dim=256`, which is
what these weights expect.

## Citation

Accepted to **Findings of EMNLP 2026**.

```bibtex
@inproceedings{kim2026streamalign,
  title     = {{StreamAlign: Streaming Text-Aligned Speech Tokenization}},
  author    = {Kim, Kang-wook and Park, Jinyoung and Kim, Jinsoo and
               Lee, Sehun and Woo, Tony and Kim, Gunhee},
  booktitle = {Findings of the Association for Computational Linguistics: EMNLP 2026},
  year      = {2026}
}
```