adversarial-tts checkpoints

English | 简体中文 | Italiano

Reference-conditioned speech synthesis with an AdaSpeech acoustic model and three BigVGAN vocoders, accompanying Detector-Guided Vocoder Training for Adversarial Text-to-Speech Synthesis.

Code and installation · Audio samples

Files and variants

File	Role
acoustic_inference.pt	Acoustic model state dictionary, without optimizer state; use this for inference.
g_V	Baseline BigVGAN vocoder.
g_VR	BigVGAN variant targeting RawNet2.
g_VRT	BigVGAN variant targeting RawNet2 and TSSD.
96000.pth.tar	Full original acoustic training checkpoint, retained unchanged with its optimizer state.
tts.yaml	Relative-path bundle selecting the acoustic assets and each vocoder.
model.yaml, preprocess.yaml, config_*.json	Matching architecture and frontend configurations.
inference_stats.json	Constructor-only pitch/energy bin endpoints extracted from the acoustic checkpoint.
*.dict	Pronunciation dictionaries used by the acoustic frontend.
SHA256SUMS, checksums.json	File integrity and acoustic export records.

The interface is 22,050 Hz, 80 mel bands, a 1,024-sample FFT/window, a 256-sample hop, and 0–8,000 Hz mel frequencies. AASIST is held out of vocoder fine-tuning for transfer evaluation. These labels describe the research variants; evaluation metrics are not inferred from file names or loading tests.

The inference acoustic export preserves all 296 model-state tensors exactly, including their dtypes and stored pitch/energy bins. inference_stats.json initializes the constructor; strict checkpoint loading restores every original bin. It is not the original training normalization statistics and is not a substitute for those statistics when resuming preprocessing or training. Original checkpoints are retained byte for byte.

Run synthesis

From a checkout of the code repository:

python3.12 -m venv .venv
.venv/bin/python -m pip install -e ".[acoustic,vocoder]" huggingface_hub
.venv/bin/hf download Cocii/adversarial-tts --local-dir checkpoints
.venv/bin/python inference.py --config checkpoints/tts.yaml \
  --text "Hello." --reference-embedding /path/to/speaker.npy \
  --reference-audio /path/to/reference.wav --vocoder VRT \
  --output-dir outputs/demo --acoustic-device cpu --vocoder-device cpu

Use your own matching [512] or [1,512] speaker embedding and reference audio. A speaker embedding alone does not supply utterance conditioning. A reference mel may replace reference audio; optional speaker-encoder generation, text-file input, controls, and separate environments are documented in the code repository. On Apple Silicon, select --vocoder-device mps; acoustic prediction uses CPU or CUDA.

Verified scope

On an Apple M2, the exported acoustic checkpoint loaded strictly with all tensors matching the original. The unified CLI then synthesized Hello. through each original V/VR/VRT checkpoint using a synthetic unit speaker embedding and a public reference clip. All three produced a finite [80,50] predicted mel and mono 22,050 Hz PCM16 WAV with 12,800 samples. Vocoding used explicit MPS with fallback disabled.

This verifies checkpoint compatibility and computation. It does not establish original speaker identity, trained speech quality, detector scores, training/resume correctness, or reproduction of the manuscript's metrics. The demonstration audio page contains the original research samples, not these smoke-test outputs.

Credits and licensing

We thank the authors of AdaSpeech and NVIDIA BigVGAN, and the contributors of the supporting acoustic, vocoder, and detector implementations. See the code repository's third-party notices. No blanket license for model weights is granted by this model card. Reference voices, personal speaker embeddings, private datasets, and the manuscript PDF are not included in this package.