extraordinarylab/parakeet-tdt-1.1b

Parakeet TDT 1.1B (en) — Transformers weights

A conversion of nvidia/parakeet-tdt-1.1b from the NeMo .nemo format to 🤗 Transformers, produced with src/transformers/models/parakeet/convert_nemo_to_hf.py. The weights are unchanged — only the checkpoint format, the config and the tokenizer/feature-extractor wrappers differ.

FastConformer-XL encoder (42 layers, d_model=1024) with a TDT (Token-and-Duration Transducer) head, durations [0, 1, 2, 3, 4], 1024 BPE tokens plus a blank token. Transcribes lower-case English without punctuation.

Usage

import torch
from datasets import Audio, load_dataset
from transformers import AutoProcessor, ParakeetForTDT

model_id = "extraordinarylab/parakeet-tdt-1.1b"
processor = AutoProcessor.from_pretrained(model_id)
model = ParakeetForTDT.from_pretrained(model_id, dtype=torch.float32).eval()

ds = load_dataset("hf-internal-testing/librispeech_asr_dummy", "clean", split="validation")
ds = ds.cast_column("audio", Audio(sampling_rate=processor.feature_extractor.sampling_rate))

inputs = processor(ds[0]["audio"]["array"], sampling_rate=16000, return_tensors="pt")
outputs = model.generate(**inputs)
print(processor.batch_decode(outputs.sequences, skip_special_tokens=True))

Parity with NeMo

Checked against nemo_toolkit[asr] 3.0.0 on CPU in float32, on two LibriSpeech clips (13.7 s and 14.2 s), both unbatched and as a padded batch of two:

tensor	max abs diff
log-mel features	6.5e-05
encoder hidden states	1.2e-04
joint logits (vocab + durations, at the blank SOS step)	3.3e-04

Encoder output lengths agree (172 and 178 frames), and greedy decoding produces identical token ids and identical transcriptions in every configuration.

Notes on the conversion

Unlike parakeet-tdt-0.6b-v2/v3, this checkpoint's NeMo vocabulary carries no <pad> token, so — as for the RNN-T checkpoints — <blank> takes the first free id (1024, matching NeMo's blank logit) and doubles as the pad token. blank_token_id == pad_token_id == 1024, vocab_size == 1025, and the joint head emits 1025 + 5 logits.

License and attribution

Released under CC-BY-4.0, following the original model. The model was developed by the NVIDIA NeMo and Suno.ai teams; see the original model card for training data, evaluation results and limitations.