Duck and Cover - Genre Autoencoder

This model is part of the duck_and_cover repository. Scope of this repository is to generate album covers based on several conditions like release year, artist & album name, and genre(s) using different types of GANs. The possible list of genres that this encoder covers can be found here.

For training prajjwal1/bert-mini has been finetuned on a list of 466.045 albums with different genre combinations taken from the aforementioned list to embed genre information, while a simple Linear Layer was trained to decode and predict the given genre from the embeddings. The albums are real-world albums retrieved using the Spotify API. The intention behind this model is that Hard Rock is somehow related to Rock, while Pop Rock is related to Rock as well and a BERT Tokenizer can capture this information as a lot of music genres are described by using pre- and suffixes. The model was validated on 133.155 during training and tested on 66.578. It yields a 98.29% Exact Match ratio on the testset and a 98.24% Exact Match Ratio on the validation set, which is extremely high given that the model can embed up to 3452 labels and most of the albums only had up to 5 labels.

Usage

The model can be used to embed genres to a 256 dimensional space using the following input.

from transformers import AutoModel, AutoTokenizer

model = AutoModel.from_pretrained("mnne/duck-and-cover-genre-encoder")
tokenizer = AutoTokenizer.from_pretrained("mnne/duck-and-cover-genre-encoder")

genres = " , ".join(["classic soul", "memphis soul", "soul", "soul blues", "southern soul"])

x = tokenizer([genres], return_tensors="pt")

output = model(**x)