cdismus4/NVIDIA-NemotronLabs-RE-USE-WebGPU

NVIDIA NemotronLabs RE-USE — WebGPU (FP16 Binary + WGSL Compute Engine)

100% On-Device Browser & Mobile WebGPU Port of nvidia/NVIDIA-NemotronLabs-RE-USE (30-Block Bidirectional Time-Frequency SEMamba for Universal Speech Enhancement: De-noising, De-reverberation, De-clipping, and Bandwidth Restoration).

• Live Interactive Demo (PC & Mobile WebGPU): https://daw.voivent.com/lab/reuse
• Production Integration: Built for Voivent DAW Satellite (One-tap on-device vocal cleaning in the mobile DAW workflow).

---

Why This Exists

Standard browser AI runtimes (onnxruntime-web, transformers.js) struggle with Mamba / Selective State Space Models (SSM) because:

1. PyTorch's official mamba-ssm relies on custom CUDA kernels (selective_scan_cuda) that cannot be exported to a clean static ONNX graph without exploding into thousands of slow sequential loop nodes.
2. Mobile GPUs (Android Adreno / Mali, iOS Apple GPU) enforce a strict WebGPU limit of maxComputeWorkgroupsPerDimension = 65,535, which causes standard 1D convolution dispatches on dense spectrograms (130,190 workgroups) to silently fail.

Instead of ONNX, this project converts the 9.6M-parameter SEMamba checkpoint directly into a zero-dependency 20.1 MiB (21.1 MB) hybrid FP16/FP32 binary (nvidia-reuse-fp16.bin) and executes all 30 Bidirectional Time-Frequency Mamba blocks directly in hand-written WebGPU WGSL Compute Shaders.

---

Key Technical Highlights

Feature	Specification
Base Architecture	30-Block Bidirectional Time-Frequency SEMamba (hid_feature=64, d_state=16, d_conv=4, expand=4, d_inner=256)
Original Checkpoint	nvidia/NVIDIA-NemotronLabs-RE-USE (clean_data_ckpt, 9,607,747 parameters across 1,416 tensors, 38.4 MB FP32 PyTorch)
WebGPU Weight Binary	nvidia-reuse-fp16.bin (20.1 MiB / 21.1 MB, 437 large projection/conv tensors in IEEE 754 FP16 + 979 precision-critical tensors in FP32)
Runtime Dependencies	Zero (Pure TypeScript/JavaScript + inline WGSL Compute Shaders; no ONNX Runtime, no PyTorch, no Python)
Mobile GPU Compliance	2D Workgroup Dispatch (ceil(spatial / 64), out_channels, 1) strictly <= 65,535 per dimension
Zero-Recompile Looping	4 fused WGSL compute pipelines shared across all 60 Mamba blocks (120 directional passes) via 256-byte aligned GPUBuffer binding offsets (555,776 bytes per block)

---

Binary Format (nvidia-reuse-fp16.bin)

1. 16-Byte Magic Header: "REUS" (0x52, 0x45, 0x55, 0x53) + uint32 format version (1) + uint32 JSON header byte length (173,752) + uint32 tensor count (1,416, little-endian).
2. JSON Tensor Directory: Maps all 1,416 tensor entries to { name, dtype, shape, offset, byteLength }.
3. Raw Contiguous Tensor Payload:
   • 437 large projection & 2D convolution weight tensors (in_proj, out_proj, x_proj, output_proj, and DenseBlock weights) are stored in IEEE 754 float16 (F16).
   • 979 precision-critical tensors (A_log, D, dt_proj, depthwise conv1d, biases, InstanceNorm2d, LayerNorm, and PReLU) are stored in IEEE 754 float32 (F32) to preserve numerical stability in the SSM state recurrence.
   • Upon initialization, A_log is pre-exponentiated into A = -exp(A_log) and all 60 Mamba blocks are packed into a single contiguous GPUBuffer at 256-byte aligned boundaries (555,776 bytes per block).

---

Supported Devices & Browsers

• Android: Chrome / Edge (Pixel 8–10+, Galaxy S23–S26+, Snapdragon 8 Elite / Adreno & Google Tensor G3–G5 GPUs)
• iOS / iPadOS: Safari 26+ (iOS / iPadOS 26+ WebGPU enabled by default)
• macOS / Windows / Linux: Chrome, Edge, Safari 26+, Firefox (Apple Silicon M1–M5, NVIDIA RTX, AMD Radeon, Intel Arc)
• Includes automatic Web Worker CPU fallback when WebGPU is unavailable.

---

License & Attribution

• Model Weights & Architecture: Derived from nvidia/NVIDIA-NemotronLabs-RE-USE by NVIDIA Corporation, licensed under the Open Model License Agreement (OpenMDW-1.1).
  • Copyright © 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
• WebGPU WGSL Compute Engine & Studio UI: Developed by C-Dismus4 / Voivent Project.