RoyalCities/Vocal_Textures_Main

Royal Cities Vocal Textures (SAO Finetune)

Introduction

This finetuned Stable Audio Open model specializes in Vocal / Operatic Chord Progressions to support granular music production workflows. Capable of creating an infinite variety of chord progressions, all output is BPM-synced and key-locked to any note within the 12-tone chromatic scale, in both major and minor keys. This model was trained on a custom dataset crafted within FL Studio and features three distinct voicings:

Male Vocals
Female Vocals
Ensemble Vocals (Combination of Male and Female)

Model Features

Multiple Types of Stem Generation: Outputs three types of voicings with a focus on Chord Progressions only,
Tonal Versatility: Generates stems in any key across the 12-tone chromatic scale, in both major and minor scales.
Audio-To-Audio: Generates interesting vocal timbres when paired with vocal stems.
Simplified Scale Notation: Scales are written using sharps only in the following format:

Minor Scales
A minor, A# minor, B minor, C minor, C# minor, D minor, D# minor,
E minor, F minor, F# minor, G minor, G# minor

Major Scales
A major, A# major, B major, C major, C# major, D major, D# major,
E major, F major, F# major, G major, G# major

For more details on the VSTs and gear used in sample creation, refer to the Gear section below.

Training Methodology

This model was designed in conjunction with the Vocal Textures dataset with a focus on only generating vocal chord progressions across the 12-tone chromatic scale in either Female, Male or Ensemble outputs.

Vocals / Choirs have very long attacks so it was trained with metadata simply classifying each sample as "Chord Progression" along with the necessary key, bpm and bar information. More details can be found in the dataset section.

Usage Guide

Supported GitHub Interfaces

This model works in both the RC Stable Audio Gradio and the original Stable Audio GitHub.

You have 2 choices of model - A Full Version and a Smaller Quantized Version which uses less VRAM.

To use the model simply place both the .ckpt file and the config .json inside their own sub-folder within the "models" folder and launch the gradio.

VST Support

This model has direct VST compatibility in the Audialab Engine.

Prompt Structure

To ensure the best results, use the following format for your prompts:


[Vocal Type], Chord Progression, [Key], [BPM], [Bar Count]

Vocal Type Prompts

["Male Vocal Texture"]
["Female Vocal Texture"]
["Ensemble Vocal Texture"]

Examples

Male vocal prompt with model output.

"Male Vocal Texture, chord progression, D# minor, 128BPM, 8 bars,"

Female vocal prompt with model output.

"Female Vocal Texture, chord progression, B major, 120BPM, 8 bars,"

Ensemble Vocal Prompt with model output.

"Ensemble Vocal Texture, chord progression, C major, 140BPM, 8 bars,"

BPMs/Bars

The BPMs ranged from as low as 100BPM up to 150BPM. The main denominations are 100BPM, 110BPM, 120BPM, 128BPM, 130BPM, 140BPM, 150BPM.

There are 2 bar settings: 4 bars and 8 bars.

Dataset Breakdown

Overview

Total .wav files: 5040
Total duration: 980.64 minutes
Average Length: 11.67 seconds
Total Size: 14.50 GB
Sample Rate: 44100 Hz

This dataset was meticulously designed to maintain a perfect balance between major and minor chord progressions, ensuring high accuracy in key generation.

By including an equal number of samples for all 12 tones in both major and minor scales (across 4 and 8 bars), the model achieves precise results. It reliably produces chord progressions in either major or related minor scales, effectively handling the nuances where major and minor scales share the same notes.

For example, in music theory, the F major scale and D minor scale share the same notes, but the tonal center of the chord progression drives the classification. After training, the model accurately generates the correct scale based solely on the prompt.

"Male Vocal Texture, chord progression, D minor, 150BPM, 8 bars,"

"Male Vocal Texture, chord progression, F major, 150BPM, 8 bars,"

Dataset Details

Vocal Type	Major Progression (4 bars)	Major Progression (8 bars)	Minor Progression (4 bars)	Minor Progression (8 bars)
Male	420	420	420	420
Female	420	420	420	420
Ensemble	420	420	420	420

The dataset is released on my HF page.

Technical Specifications

Platform: Runpod
Monitoring Tool: Weights and Biases
Epoch: 23
Steps: 1800
Learning Rate: 5e-5
Optimizer: AdamW
Scheduler: InverseLR
Batch Size: 32
Hardware: 2x NVIDIA A40 GPUs

See config file for further details.

Limitations and Biases

The Model has high accuracy when it comes to staying in key due to the balance in the dataset. The metadata however was designed in such a way that the model is mainly designed to generate chord progressions only - as opposed to the piano model which also generated melodies. This is due to Vocal choir samples often having very long attacks so often sit at the back of a mix or to fill out the frequency space.

I have noticed some light noise in the outputs - in particular the ensemble progressions. It almost sounds like the model is trying to add other instrumentation in. I think this may be due to the base model primarily being trained on music stems rather than vocals but I cannot say for certain - this will need to be corrected in a future model or when there is far more vocal data.

Best use case may be to add some light reverb or post-processing for best results / use in a song.

Gear Used

DAW: FL Studio (Image-Line)
Vocals: Multiple Choir Libraries
EQ: PRO-Q3 (FabFilter)
Additional Gear: Fruity Reeverb 2, Shaperbox 3 (Compressor), Soothe2

License

This model is licensed under the Stability AI Community License. It is available for non-commercial use or limited commercial use by entities with annual revenues below USD $1M. For revenues exceeding USD $1M, please refer to the LICENSE for detailed terms.

RoyalCities
/

Vocal_Textures_Main