bezzam/wavlm_large_finetune_seed_tts_eval

WavLM-Large + ECAPA-TDNN speaker verification (seed-tts-eval SIM model)

A verbatim mirror of Microsoft's speaker-verification checkpoint used as the SIM (speaker similarity) metric by seed-tts-eval, and therefore by the Seed-TTS / F5-TTS / CosyVoice line of papers. Re-hosted so TTS evaluation jobs can fetch it from the Hub; the weights are unmodified.

Provenance

Origin	microsoft/UniSpeech → downstreams/speaker_verification
Row in upstream's table	WavLM large, Fix pre-train: No — their best SV model
Reported EER	Vox1-O 0.431, Vox1-E 0.538, Vox1-H 1.154
Upstream download	Google Drive
File	wavlm_large_finetune.pth (1,301,926,579 bytes)
sha256	51f07e3b94d9e0262a6a675ef5a087be3dd09e8c62e9d886827f44f82fe7f94b
License	CC BY-SA 3.0, per the UniSpeech repository

Fix pre-train: No means the WavLM-Large backbone was fine-tuned during speaker-verification training, so these backbone weights do not equal microsoft/wavlm-large.

This mirror exists because neither upstream URL is reachable from a job container: the Google Drive link requires a confirm-token dance, and the Azure blob URL in seed-tts-eval's vendored copy of the README has an expired SAS token.

Contents

wavlm_large_finetune.pth is a torch.save dict with two top-level keys:

• model — 711 tensors: feature_extract.model.* (the WavLM-Large backbone, in original-WavLM naming), the ECAPA-TDNN head (feature_weight, layer1-layer4, conv, pooling, bn, linear), and loss_calculator.projection.weight (the training-time AM-softmax classifier, unused at inference).
• best_valid_eer — an unset sentinel (100.0); ignore it.

Usage

Upstream runs this via s3prl + torch.hub. The open TTS leaderboard instead rebuilds the backbone as an HF WavLMModel and remaps the checkpoint's keys onto it — verified bit-exact (0.0 max difference across all 25 hidden states and the final 256-d embedding) against UniSpeech's original WavLM.py. See transformers/score_similarity.py:

python transformers/score_similarity.py \
    --manifest_path=<manifest>.jsonl \
    --sim_backend=wavlm_seed_tts \
    --sim_ckpt_repo=bezzam/wavlm_large_finetune_seed_tts_eval

Scale

Cosine similarities from this model are not interchangeable with those from microsoft/wavlm-base-plus-sv. Measured on the VoxCeleb1 clips shipped with UniSpeech:

	this model	wavlm-base-plus-sv
same speaker	0.60 - 0.69	0.89 - 0.96
different speaker	-0.17 - 0.18	0.60 - 0.84

Only the values from this model are comparable with published seed-tts-eval SIM numbers.

Citation

@article{chen2022wavlm,
  title={WavLM: Large-Scale Self-Supervised Pre-Training for Full Stack Speech Processing},
  author={Chen, Sanyuan and Wang, Chengyi and Chen, Zhengyang and others},
  journal={IEEE Journal of Selected Topics in Signal Processing},
  year={2022}
}