razaanstha/swar-onnx-webgpu

Swar ONNX for browser experiments

Experimental ONNX exports of Naamche Labs' Swar, a Nepali text-to-speech and voice-cloning model. This repository is an independent conversion, not an official release by the Swar authors. It is not a Transformers.js pipeline()-compatible model. The reference browser prototype uses ONNX Runtime Web directly.

Status: model components, not a complete runnable application

The release contains the Swar-derived encoder and two transformer graphs, but does not include the Vocos-based audio decoder or the Maya voice reference. Those are required for complete text-to-speech. The decoder was omitted because the original Vocos model card explicitly identifies an MIT license for its code, but we could not establish a clear primary-source license statement covering the exact pretrained checkpoint used in this export. The reference voice was omitted pending verification of its separate distribution terms. No generated audio from a user is included.

The browser/ files are reference implementation source, not a turnkey demo: they require a separately obtained compatible decoder, reference audio and transcript, ONNX Runtime Web assets, and a directory layout matching their paths. The scripts in export/ document the conversion, including decoder creation from Vocos assets obtained independently. Verify the terms of any separate assets before using or redistributing them.

Model files

• onnx/encoder.onnx: FP32 Swar encoder and preprocessing graph.
• onnx/transformer-32.onnx: FP32 transformer with a 32-step schedule.
• onnx/transformer-16.onnx: FP32 transformer with a nonuniform 16-step schedule, included as an experimental faster option.

Both transformers use a WebGPU-oriented rewrite of a multi-output Split into equivalent Slice operations. The schedule and time embeddings are fixed in each exported transformer: do not run only the first 16 iterations of the 32-step graph as a substitute for the 16-step graph. Both variants are approximately 1.32 GB each; this is not the unverified FP16 export.

Base source: Aarjan/swar at revision 8ca6f1bbc9da567d92158822e24673c2084e7e86; source checkpoint podvoices_ema.pt. The export scripts are based on DakeQQ/F5-TTS-ONNX, with modifications for Swar, dynamic decoder slicing, separate 16/32-step exports, and the WebGPU Split rewrite. See licenses/ and export/vendor/LICENSE.

Testing and limitations

The 32-step text embedding, one denoising step, and decoder were compared with PyTorch in Colab; max errors are in validation/validation.json. The 16-step model passed a comparison at denoising step 8 against the corresponding PyTorch calculation; see validation/validation-16.json. These are stage checks, not an evaluation of speech quality.

The full prototype, using a separately held decoder and voice reference, produced Nepali audio on one Apple GPU using a Chromium test browser with WebGPU flags. On one sentence, 32-step generation took 77.3 and 74.0 seconds and 16-step generation took 52.2 and 39.4 seconds, excluding loading. These are limited tests with independent random noise; browser compatibility, performance, accuracy, and voice quality are not established across devices. Multi-paragraph input is split into segments. The prototype supports plain whole numbers, but its original Swar text normalization is incomplete, including English and formatted dates, times, and decimals.

Attribution and licenses

• Swar weights: Naamche Labs / Aarjan/swar, CC BY-SA 4.0. The ONNX files are conversions of those weights. Attribution, change notices and applicable share-alike terms must be retained when redistributing.
• Swar source code and adapted text normalizer: Naamche Labs, MIT; see licenses/NAAMCHE-MIT.txt and the Swar repository.
• Exporter: DakeQQ and contributors, F5-TTS-ONNX, Apache-2.0; see licenses/DAKEQQ-APACHE-2.0.txt. The included exporter is modified.
• Vocos: The omitted decoder was generated using charactr/vocos-mel-24khz. Its original model card explicitly identifies MIT for its code but is unclear about the weights; no claim is made here about permission to redistribute that checkpoint or decoder.
• ONNX Runtime Web: used in the prototype; runtime binaries are not bundled in this repository. Consult the ONNX Runtime repository for its license and notices if distributing the runtime.

Swar's authors ask users to clone a person's voice only with that person's permission and not use the model to impersonate or deceive. See the original model card for details and training-data attribution.