GM.pth

**GM.pth** is an AI voice model that mimics Gabriel Mann.

Its intended use is for The Lorax/The Biggering-related projects, though the choice to recreate Mann was mainly out of convenience and make this first attempt at voice cloning go faster. Audio of him had already been collected for prior and future works.

GM's creation started in early July of 2023, but progress was aborted around late August when Google Colab reinforced its rules on "disallowed code"; and made it no longer possible to freely train AI models through means of the RVC-Project web UI or similar Colab templates.

There's no certainty on whether GM can or will be revisited.

## Technical details

GM and its variants were assembled through a Colab-hosted Python script, known as the RVC-Project, on standard 40k settings.

About 1 hour and 20 minutes of audio files were collected, of which half are song performances and the rest the isolated dialogue from Youtube interviews. Audio with intrusive, unerasable background noise, notably low recording quality, strong echo, or other effects that altered the voice from sounding otherwise natural were not considered for use.

Because AI models with large audio libraries require hours of processing, the files were exported in MP3 format -opposed to the often recommended, but heavier WAV- and roughly two months of uninterrupted work was devoted to the overall experiment.

About 4 additional Gmail accounts needed to be created in order to circumvent Colab's daily GPU use limitations and reduce the time spent on the project.

### Used data

* Door to Door; Gabriel Mann interview (speech extraction)
* "Lighted Up" by Gabriel Mann (vocal extraction)
* "Remember" by Gabriel Mann (vocal extraction)
* "For Good" by Gabriel Mann (vocal extraction)
* "Gravity" by Gabriel Mann (vocal extraction)
* "Artichoke" by Gabriel Mann (vocal extraction)
* "Spend Another Minute" by Gabriel Mann (vocal extraction)
* "Breathing Underwater" by Gabriel Mann (vocal extraction)
* "Eight Feet Tall" by Gabriel Mann (vocal extraction)
* "Slow Motion" by Gabriel Mann (vocal extraction)
* "This Feeling" by Gabriel Mann (vocal extraction)
* "In Deep" by Gabriel Mann (vocal extraction)
* "We Are We" by Gabriel Mann (vocal extraction)
* "Over the World" by Gabriel Mann (vocal extraction)
* "The Man With Too Many Ideas" by Gabriel Mann (vocal extraction)
* "She Makes" by Gabriel Mann (vocal extraction)
* "Tightrope Walker" by Gabriel Mann (vocal extraction)
* "Daybreaks" by Gabriel Mann (vocal extraction)
* "Go Up to Go Down" by Gabriel Mann (vocal extraction)
* "Careen" by Gabriel Mann (vocal extraction)
* "Forever Young" cover by Gabriel Mann (vocal extraction)
* "Biggering" by The 88 (vocal extraction)
* "Can't Stand the Rain" by The Rescues (vocal extraction)
* "You're not Listening" by The Rescues (vocal extraction)
* Test performances by an earlier version of the model, fed back to it:
	+ "Rise Above" by Reeve Carney
	+ "Wandering" by Ben Folds
	+ "They Don't Know" by Tracey Ullman
	+ "iPad" by Parry Gripp
	+ "Broken" by Gorillaz
	+ "Breng Jij Mij Naar Huis" by Marco Borsato
	+ "A Monster in Paris" by Sean Lennon
	+ "Another Day" by WHIGFIELD
	+ "Omarm" by Bløf
	+ "This Plan" by Moke
	+ "The First Time Ever I Saw Your Face" cover by Leona Lewis
	+ "A Whiter Shade of Pale" cover by Annie Lennox
	+ "Scared of Heights" cover by Morten Harket
	+ "Your Song" cover by Ewan McGregor
	+ "Baby Can I Hold You" cover by Boyzone
	+ "Fur is Mine" (first recorded demo)

### Voice description

GM sounds human and comes with little artifacting -usually none when the recording it copies speaks properly and has no oddities itself. It's able to reach fairly high and low octaves, though may struggle to translate boomy, raspy singers, or instances of vocal fry. In certain instances, this can be circumvented by letting GM sing at a different octave, then changing that recording to its preferred pitch in editing software.

The model's sound quality is clear and decent for song mixing, though of lower quality than that of, for example, the common animation studio. Mann's Sounds of Hollywood interview appears to be GM's biggest influence in terms of overall quality.

Like AI models in general at this stage (2023), it can't properly copy tracks with strong echo in them, vocals with background singers performing at the same time, or group performances/overtone singing. Occasionally, it may circumvent it, but tends to break again when the pitch strays from the average or there's a long note.

If the original singer has a particular accent, it will try to copy it, though may exaggerate certain dialects. Since Mann is an American-English speaker, the model sings foreign songs with an accent, but the severity depends on what model variant is used, with the regular GM's accent being slightly stronger than GM50100p. On the flip side, foreigners sound more American when they speak English through GM, unless their accent is especially strong.

## Model variants

GM
"GM" is a 1000 epoch model and the only fully trained variant.
It contains the complete audio library and was revised/restarted multiple times. Even so, there are minor pronunciation flaws that other, lesser models did not have, leading to further experimentation and the creation of GM50p and GM50100p.

**download link to model**

GMz
"GMz", or "GMzang" (GMsinging), is a 900 epoch model.
It was trained to near-completion, but halted after noticing little difference between it and the already existing 1000 epoch GM, though its pronunciation appears weaker at some places.
Since it hasn't been properly tested against other existing models, since the voice cloning experimentation was still going, it hasn't yet been discarded of.

**download link to model**

GMp
"GMp", or "GMpraat" (GMtalk), is an abandoned model.
It was only fed speech files, which caused the model to sing in an exaggerated American accent, with less emotion, less volume, and worse pronunciation, especially whenever the language was foreign. The model had not been tested on normal speech before deletion.

GM50p
"GM50p", or "GM50%", is an experimental 200 epoch model that attempted to retrace some previously made steps and was fed the original first half the audio library. Training could not be continued after Colab's rule change, and as such, GM50p's usefulness compared to the current 1000 epoch GM is unsure.

GM50100p
"GM50100p", or "GM50%praat100%zang" (GM50%talk100%singing), is an experimental 700 epoch model that uses all vocal clips, but half of the interview dialogue as an attempt to strengthen the model's liveliness, while keeping its pronunciation strong.
Training could not be finished after Colab's rule change.

**download link to model**

## Trivia

* With there being little to no (recent) online fuss about the code restriction that prevents further development on GM, the possibility exists that the real problem concerns something Google Colab refuses to specify and is specific to VampireMeerkat.

## See also

* The Biggering
* The Once-ler
* Fur is Mine (Song)
* Kindadus My Blood (Song)
* Embrace Me (Song)
* Deathbed Excuses (Song)
* The Lorax AI