MOSS voice-acting v2 -- per-voice identity-recovery LoRAs

One rank-4 LoRA per voice profile, trained on that voice's own takes that already sound like the voice (ECAPA speaker similarity to the voice's reference recording >= 0.60, held-out groups removed), and used to regenerate the takes of that voice that fell below the 0.40 similarity floor.

  • rank 4, alpha 8, dropout 0.05, the same 23 target modules as the shipped per-voice adapters
  • one stage, checkpointed and evaluated on held-out groups every epoch; the epoch shipped is the held-out-loss minimum, which varies by voice -- a fixed epoch count overfits some voices and underfits others
  • merged for regeneration at scale 1.0

<voice>/report.json carries that voice's before/after numbers, the epoch curve that chose the checkpoint, and the cost columns.

Two things that must travel with any result from these adapters

  1. The 0.40 floor is this project's ECAPA scale, not an established perceptual fact. An independent WavLM-tbr embedder (Orange/Speaker-wavLM-tbr, published same-speaker threshold 0.472) puts 75.4 % of the takes selected as "failures" above its own threshold.
  2. Identity is bought, not free. In the reference study the identity gain cost -11 % target emotion strength and -14 % blend. Every voice's report.json reports its own cost columns.

And one about how to read a before/after

Production regenerates a single arm, so there is no per-voice control. The reference study (emolia_c1699) measured that simply re-rolling these takes with no voice adapter lifts them from 0.2 % to 14.0 % above the floor -- they are the unlucky tail of their groups, so regression to the mean alone moves them. The adapter took the same population to 43.7 %. A per-voice vs_orig delta in these reports therefore contains that re-roll component and is not the adapter's effect on its own.

Downloads last month
-
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support