spybyscript/nemotron-3.5-asr-streaming-0.6b-litert

Nemotron 3.5 ASR Streaming 0.6B — LiteRT conversion

Community conversion by spybyscript, derived from NVIDIA's official Nemotron 3.5 ASR model. This is not an official NVIDIA or Google release. No additional training or fine-tuning was performed.

These are component graphs for a stateful LiteRT integration, not a single drop-in transcription model. Each version contains 31 .tflite files plus a tensor/state manifest, vocabulary, source provenance and parity evidence. An application must implement audio chunking, cache management and greedy RNN-T decoding. The integration source repository is private; no APK or private implementation source is distributed here. The public integration contract describes how to use these files independently.

Source and license

• Base model: nvidia/nemotron-3.5-asr-streaming-0.6b.
• Exact source revision: 1c8deaecc64b91f034d73e08dd8b64625eb3395d.
• Original weights SHA-256: 9eebdd6590289cb3030f310858f3df93256600a800a3e8200c5993d5f967e174.
• Model materials: OpenMDW-1.1 license; official license page. Retain the license and applicable origin notices when redistributing.
• See NOTICE.md and each folder's source-manifest.json. Each MODEL_CARD.md is an unchanged copy of the upstream card, retained for attribution; its benchmarks and multilingual claims are not results of this conversion.

The upstream checkpoint is multilingual. This release's tested integration supports en-US, prompt ID 0, and the 560 ms chunk profile only. Other languages, automatic language detection and other chunk sizes have not been validated here. Do not infer their support from the upstream card or vocabulary.

Versions

Folder	Storage / quantization	Graph bytes	Intended use
fp32	FP32 baseline	2,486,920,752	Reference and accuracy comparison
fp16	FP16 weight storage, floating-point interfaces	1,269,685,476	Smaller reference; tested CPU and experimental mixed GPU
int8	Dynamic per-channel INT8 encoder fully-connected weights	981,082,800	Current CPU4 default in the development harness

INT8 is partial quantization, not a fully integer model: 192 fully-connected weight buffers across 24 encoder blocks are INT8; 72 convolution/depthwise weight buffers remain FP32. Non-encoder-block graphs remain FP32 and byte-identical to the FP32 baseline. Activations/interfaces and streaming caches are floating point. No representative-dataset static activation calibration was used. This is not a precompiled Qualcomm NPU model.

The FP16 variant includes a bounded attention-mask rewrite (-infinity to -10000) used in the GPU experiments. FP16 storage does not guarantee that every CPU operation executes with FP16 arithmetic. FP32 and INT8 retain their original mask graphs.

Measured accuracy and performance

Samsung S23 Ultra, Snapdragon 8 Gen 2, Android 16, LiteRT 2.2.0, four CPU threads. Small development measurements, not a standard full benchmark. Human audio was 23 clean English audiobook clips from 10 speakers, totaling 569 reference words. Three synthetic clips were evaluated separately. Corpus variant order rotated per clip, with a new app process for each run.

Measurement	FP32	FP16	INT8
Human word errors / 569 words	17	17	22
Human WER	2.99%	2.99%	3.87%
Synthetic word errors / 157 words	3	3	3
Corpus weighted compute RTF	0.607	0.553	0.416
Sustained compute RTF	0.697	0.599	0.447
Sustained later resident PSS, GiB	2.46	3.59	2.02

RTF is measured compute seconds divided by audio seconds; lower is faster and below 1 means faster than realtime in that test. It is not first-text latency. Model loading is excluded. INT8 added five word errors; equal FP32/FP16 aggregate error counts do not mean every transcript was identical. All successful runs had finite outputs and no RNN-T symbol-limit failures.

The sustained test used 574.935 seconds of concatenated/repeated audio without resetting model state. It is not additional independent accuracy data. It ran INT8 → FP16 → FP32 without cooling, with thermal statuses respectively 1–2, 2–3 and 3. These differing conditions limit direct speed comparisons. Resident PSS subtracts same-snapshot SwapPss; the later median excludes startup and teardown. It is neither model-file size nor an exact peak. The long replay also held an audio buffer in memory.

The corpus used intrusive full process memory dumps. An initial sustained FP32 attempt under that method was killed for LOW_MEMORY; all three variants completed after switching to local-only memory sampling approximately every 5–6 seconds. This does not establish that FP32 cannot run long sessions or isolate the cause of the earlier failure. These are accelerated replays, not hours-long microphone-paced battery tests. TTS/LLM coexistence, energy consumption and broad noisy/accent/domain accuracy remain untested.

INT8 acceptance and preserved failed parity

INT8 changes source outputs and fails the original exact-token/numerical smoke-parity gate. Its parity-*.json reports remain unchanged, including passed: false; its manifest retains evaluation_only: true. The development harness explicitly accepted only this exact manifest after the larger on-device comparison:

0b5b594dcd499de6ffc7b2511ade5d493bb3b93a99e311192c64b1fc1ae58405

This acceptance records a chosen accuracy/memory tradeoff, not parity equivalence. A generic loader that rejects evaluation bundles will still reject it. Do not edit the manifest or mark failed reports as passed to make it load. An independent integration should implement its own explicit acceptance policy while preserving file hashes and evidence.

FP32 manifest SHA-256: 334033700f259c780bab5620bd47e984702ecb68e36b7b968597619387cb4474.

FP16 manifest SHA-256: 15e537c192dab9eeb7cee15a34cfb8efa51c81db4ba17fb668efa68be65aa224.

Download and integrate

Download one complete variant, plus the root license, notices and documentation. With huggingface_hub installed:

from huggingface_hub import snapshot_download

snapshot_download(
    repo_id="spybyscript/nemotron-3.5-asr-streaming-0.6b-litert",
    # Pin revision to an immutable commit from the repository history for deployment.
    allow_patterns=["int8/*", "README.md", "INTEGRATION.md", "LICENSE",
                    "NOTICE.md", "publication.json", "SHA256SUMS"],
    local_dir="nemotron-litert",
)

Verify the manifest identity above and every inventoried file's SHA-256 and byte count before loading. SHA256SUMS covers all published payloads except itself; publication.json identifies the original conversion source revision and bundle manifests. Keep each variant in its own directory; do not mix graphs or vocabularies from different bundles/models.

Use the integration contract with the actual input/output names, shapes and dtypes in each manifest. LiteRT Android's standard Kotlin CompiledModel API was used for inference. CPU4 is the tested INT8 configuration. GPU experiments required explicit FP32 arithmetic and mixed CPU/GPU execution on FP16 graphs, used more memory, and established no general sustained advantage. NPU acceleration has not been implemented or verified. These files do not load directly with transformers.pipeline or nemo.from_pretrained, and this repository does not provide hosted inference.

Limitations and intended use

For local speech recognition experiments and application integration. Errors can include names, word substitutions, omissions and punctuation. The small clean-speech evaluation does not establish performance in meetings, noise, varied accents or specialist terminology. Upstream training, bias, intended-use and ethical information is available in the pinned NVIDIA model card. No claim of government-contract suitability or certification is made. The model download is public; the private development repository is not required to download or inspect the artifacts.