spybyscript/nemotron-3.5-asr-streaming-0.6b-coreai

Nemotron 3.5 ASR Streaming 0.6B — Core AI

Community conversion by spybyscript of NVIDIA Nemotron 3.5 ASR to Apple's Core AI .aimodel format for macOS, iOS and iPadOS 27. Exported directly from the original PyTorch/safetensors checkpoint, without training or fine-tuning. This is not an official NVIDIA or Apple release.

The approximately 1.343 GB mixed/ package contains 31 component assets for a stateful 560 ms RNN-T pipeline, plus tokenizer, source configuration, tensor contracts, hashes and validation. The included Swift streaming runner accepts incremental mono 16 kHz PCM, carries encoder/decoder state, supplies the language prompt and decodes tokens to text.

The source is multilingual and its language-prompt interface is preserved. This conversion has been validated only with English en-US (prompt 0). Other languages and automatic language detection need separate parity and recognition tests. The language metadata above reflects tested coverage, not the full upstream capability list.

Related formats: Nemotron 3.5 LiteRT, and the separate Nemotron English Core AI. The English-specific checkpoint has a different vocabulary and projection contract; its components cannot be substituted here.

Source and license

• Source revision: 1c8deaecc64b91f034d73e08dd8b64625eb3395d.
• Original safetensors SHA256: 9eebdd6590289cb3030f310858f3df93256600a800a3e8200c5993d5f967e174.
• Model materials retain OpenMDW 1.1: LICENSE, NOTICE.md.
• The original card is preserved in SOURCE_MODEL_CARD.md. Its multilingual benchmarks are upstream results, not measurements of this conversion.
• Export commands and pinned dependencies: conversion/README.md.

Precision and validation

Components	Precision
Waveform frontend and initial frontend	FP32
Initial/steady subsampling, 24 encoder blocks, language-conditioned projection	FP16
RNN-T predictor and joint	FP32

The initial all-FP16 network changed source tokens on one public recording. Restoring FP32 for the predictor and joint restored exact source-token parity across the entire public set. The failed candidate's evidence is preserved; its assets are not shipped. The frontend retains FP32 waveform/DFT/mel arithmetic. No W8 variant is included.

Evaluation uses 20 selected LibriSpeech test-clean recordings, 10 speakers, 171.555 seconds and 488 reference words. This is a small conversion test, not the complete LibriSpeech benchmark. Original FP32 streaming references were independently cross-checked against the source model's generate() with the same language and chunk profile.

Measurement	Result
Exact original FP32 token matches	20/20 recordings
Human-reference WER, original and converted	3.07% (15/488)
Latest warm real-time factor	0.0727
Throughput	13.75× real time
Compute per minute of audio	4.36 seconds
Median per-recording chunk p50	38.5 ms per 560 ms audio chunk

Quality: mixed/validation.json. Latest speed: raw report, summary. The original quality run reported warm RTF 0.0772; the table uses the later speed run. Warm RTF excludes the first recording and loading. The measured chain includes frontend, encoder, autoregressive decoding, host copies and reference-array comparisons. Reference-file reads, microphone capture and UI are excluded. Compute time is not live-caption latency.

Latest benchmark host: Apple M5 Max, 128 GiB, macOS 27.0 (26A428), Xcode 27.0 (27A266a), Swift 6.4. Default Core AI specialization was used; actual CPU/GPU/Neural Engine placement was not profiled. Normal desktop activity was present. These are not phone/tablet performance estimates.

Streaming and platform checks

The standalone adapter was tested over 80 utterances (20 recordings × four cycles; 686.22 seconds of repeated audio), varying callback sizes, resetting all caches, comparing tokens and decoded text, checking retained PCM bounds, and finishing with an empty stream. See streaming evidence. Repetitions check stability; they do not form a new accuracy set or an uninterrupted long conversation.

All 31 final assets compiled for iOS 27, architecture h19p, and the Swift adapter typechecks against the iPhoneOS 27 SDK: asset compilation, Swift check. No physical iPhone/iPad execution, battery, thermal or microphone test is claimed. Portable .aimodel assets are distributed; local compilation caches are not included.

Download and transcribe

Requirements: Apple silicon, macOS 27, Xcode 27 and Apple's Metal Toolchain. The Swift executable uses the system Core AI framework; Python, NeMo, ONNX and LiteRT are not runtime dependencies.

hf download spybyscript/nemotron-3.5-asr-streaming-0.6b-coreai \
  --include 'mixed/*' '*.swift' '*.md' 'LICENSE' 'publication.json' 'SHA256SUMS' \
  --local-dir nemotron35-coreai
cd nemotron35-coreai
xcodebuild -downloadComponent MetalToolchain
xcrun swiftc -parse-as-library -O CoreAI35SpeechRunner.swift Swift35Transcribe.swift -o transcribe
./transcribe mixed recording-mono-16khz.wav --language en-US

Pin an immutable repository revision for deployment and recorded experiments. The executable verifies bundle hashes, streams the WAV through 997-sample callbacks and prints JSON containing text, tokens, prompt ID, timing and memory. Displayed text skips special tokens; raw IDs preserve language tags. Inputs must already be mono 16 kHz. VAD, resampling, diarization and speaker segmentation belong to the host.

--language defaults to en-US. The source dictionary is in mixed/processor_config.json; unknown keys are rejected. Other source language keys and auto are available for testing and marked unvalidated in CLI output. A dictionary entry alone does not establish upstream language quality; consult NVIDIA's card and evaluate the desired language.

For app use, see INTEGRATION.md. Keep the runner resident and serialize operations. The reference uses the main actor and host copies between graphs; production apps should use a dedicated serialized inference context and measure their complete pipeline. Core AI caches specialization across launches; routine resource loading remains necessary. OS updates, source/options changes and cache removal can require preparation again.

Limits

Conversion does not improve the source model's handling of names, emotional speech, noise or overlapping speakers. English parity cannot establish multilingual parity. The export fixes the 560 ms profile; editing the manifest cannot change graph shapes. Evaluate memory, preparation time, sustained latency and energy on the target phone with concurrent emotion analysis before choosing shipping defaults.

No private recordings or transcripts are distributed. Public corpus provenance is in evidence/README.md. Apple references: Core AI, PyTorch conversion, model examples.