flaukowski/kannaka-rvc-voice

Kannaka RVC voice

An RVC v2 voice-conversion model of Kannaka, the AI operator of Kannaka Labs. Given a vocal sung or spoken by anyone, it re-voices it in Kannaka's timbre while keeping the melody. It was made so Kannaka can sing on songs generated with flaukowski/kannaka-ace-step-lora, since her Piper voice can only speak.

This is a first version, published to be improved. Its limits are measured and listed below.

Where the voice comes from, honestly

The chain is synthetic end to end; no human voice was recorded or used.

1. Kannaka's podcast voice is an ElevenLabs voice.
2. kannaka-piper-voice (e7009) was fine-tuned on 17.8 minutes of that voice.
3. This model was trained on 30.5 minutes of Piper speech (251 clips of 3 to 15 s), read from plain project prose (repository READMEs and study write-ups), with the speaking rate varied between 0.90 and 1.15 so the model hears a range of pace and pitch.

It has heard Kannaka speak, never sing.

Files

• kannaka.pth: the model (RVC v2, 40 kHz, pitch-guided, HiFi-GAN vocoder, contentvec embedder), epoch 80, step 8000. sha256 begins bc35b1d0.
• kannaka.index: the feature index (sha256 begins aa6a8a1b).
• config.json, model_info.json: training configuration and dataset length.
• samples/, the first song it sang, "Signal in the Dark" (2:30, made with the Kannaka ACE-Step LoRA):
  • signal-in-the-dark-original.flac: the song as generated, with ACE-Step's own singer, for comparison.
  • signal-in-the-dark-kannaka-air.flac: the one to start with. Kannaka's converted vocal, with the original vocal's band above 9 kHz blended back in to restore air (see the limits below).
  • signal-in-the-dark-kannaka.flac: the plain conversion.
  • signal-in-the-dark-kannaka-minus12.flac: converted an octave down. It stays in key but has a very different character.

How it was made

• Trainer: Applio 3.6.6, on one RTX 3050 (8 GB). Batch 8, fp16, about one minute per epoch, stopped at a 90-minute cap after 80 epochs (96 minutes in total). The lowest generator loss was at epoch 79.
• Song pipeline: BS-RoFormer vocal separation (model_bs_roformer_ep_317_sdr_12.9755, via audio-separator 0.47), then conversion with rmvpe pitch, transpose 0, index rate 0.6 and protect 0.33. The vocal is mixed back at the original vocal's level with a light reverb, peaking at -1 dBFS.

Use it in Applio or any RVC v2 front end: load kannaka.pth with kannaka.index, choose rmvpe, and start from index rate 0.6 and protect 0.33.

Limits, measured

Neither the trainer nor this card's author listened to the output; these are measurements.

• No air. Piper renders at 22.05 kHz, so nothing in the training data was above 11 kHz, and the converted voice has none (0.000% of its power, against 0.63% in the original vocal). On its own it sounds muffled against a full-range track. The -air sample works around this; a better fix is training data with real high frequencies.
• Range. Kannaka's training speech spans about 107 to 250 Hz. The sung melody sits around 210 to 500 Hz, roughly an octave above anything the model heard. High and sustained notes are where it is weakest, and it has never heard singing or vibrato.
• Pitch. In loud sung passages the converted pitch follows the melody exactly. About 14% of frames show octave jumps, mostly in quiet passages.
• Gaps. Where the vocal is silent, the conversion leaves a faint hum around 90 to 100 Hz, about 70 dB below the vocal peak, which is inaudible under the music.
• Separation bleed (the original vocal's reverb converted along with it) was not measured.

What would improve it

Training data at a higher sample rate with real high frequencies; material in a singing range (sustained vowels, wider pitch, some vibrato); and more than 80 epochs.

License

Released under the Space Child License (see LICENSE): free for peaceful use, with accountability when a work is directed toward harm. Applio and RVC are separate projects under their own licenses.

Kannaka Labs (Nick Flach) with Kannaka; model and card by Flaukowski.