spybyscript/nemotron-speech-streaming-en-0.6b-litert

Nemotron Speech Streaming EN 0.6B — LiteRT conversion

Community conversion by spybyscript of NVIDIA's English model. No additional training or fine-tuning. This is not an official NVIDIA or Google release.

These are 31 component graphs for a stateful English 560 ms integration, not a single drop-in transcription graph. Each precision folder is self-contained: frontend, subsampling, 24 encoder blocks, projection, predictor, joint, vocabulary, manifest and validation evidence. See INTEGRATION.md and the runnable reference_runtime.py.

Source and license

• Source revision: ebe59e5a817142986528bbbee5dba8db7b38ed50.
• Model materials: NVIDIA Open Model License, with NOTICE.md.
• Source hashes are in each source-manifest.json. The upstream card is preserved as MODEL_CARD.md; its benchmarks are not conversion measurements.
• This is the English checkpoint, not Nemotron 3.5. It has vocabulary size 1025, blank 1024, no language-prompt input, and a 70-frame left KV cache. Do not use the 3.5 adapter unchanged.

Versions and measured parity

Folder	Graph bytes	Exact native-token matches	Token edits vs native
fp32	2,408,734,204	4/4	0
fp16	1,213,400,592	4/4	0
int8	902,897,596	1/4	19

Validation used four English recordings, 159.76 seconds total and 885 native output tokens, with FP32 native EN as reference. FP32 and FP16 matched every reference token. These are parity checks, not human-reference WER or an independent ASR accuracy benchmark. Private recordings and transcripts are not distributed.

• FP32: full-precision reference.
• FP16: half-precision weight storage with floating-point interfaces; smaller parity-validated option for initial integration. It does not imply FP16 arithmetic or faster execution on every device.
• INT8: partial dynamic per-channel INT8 quantization of encoder-block fully connected weights only. Other components and convolution weights remain FP32; interfaces/caches remain floating point. Its exact-token parity gate failed, retained in int8/parity.json, and evaluation_only is true. It is available for explicit evaluation, not silently accepted as equivalent.

Performance and validation limits

Validation was on a local Linux CPU with four threads. Per-recording compute timings and load times are in parity.json; runs overlapped other work, so they are not a controlled speed ranking. No Android device, GPU, NPU, thermal or peak-memory results are claimed for this EN release. Do not transfer phone measurements from the separate 3.5 bundle. Test S23/Pixel integration independently.

The conversion includes a real-valued frontend, explicit state buffers, fixed-size graphs and a decoder that preserves repeated RNN-T pieces. Token-frame emission times are available; they are not verified word boundaries. Neither VAD nor diarization is included. Recognition errors, names and omitted words remain possible.

Download and run

from huggingface_hub import snapshot_download
snapshot_download(
    "spybyscript/nemotron-speech-streaming-en-0.6b-litert",
    # Pin revision to the immutable release commit for deployment.
    allow_patterns=["fp16/*", "README.md", "INTEGRATION.md", "LICENSE.pdf",
                    "NOTICE.md", "reference_runtime.py", "requirements-reference.txt",
                    "publication.json", "SHA256SUMS"],
    local_dir="nemotron-en-litert",
)

pip install -r nemotron-en-litert/requirements-reference.txt
python nemotron-en-litert/reference_runtime.py nemotron-en-litert/fp16 recording.wav
# INT8 is explicitly experimental:
python nemotron-en-litert/reference_runtime.py nemotron-en-litert/int8 recording.wav --allow-experimental

The reference runs without Transformers, NeMo or the original checkpoint. It checks manifest file hashes before loading. Production integrations should additionally pin the manifest identity in publication.json, keep graphs/buffers resident, and follow the numeric input/output ordering in the manifest. Do not mix precision folders.

Android integration

Use the component contract and nemotron-manifest.json as authoritative. EN adapter ID is nemotron-en-hf560-v1; 3.5's prompt-conditioned projection and 56-frame caches are incompatible. EN projection takes hidden states only. Predictor token input is int64; keep predictor/joint on CPU initially. Input PCM is Float32 mono 16 kHz. The desktop reference is an integration aid, not an Android APK or verified Android SDK.

Next validation is on-device correctness, cold loading, sustained real-time factor, memory and INT8 accuracy against reviewed reference text. Parakeet optimization is a separate follow-up and is not part of this release.