DOI: 10.5281/zenodo.10938615
title: LFM2b Lyrics Descriptor Analyses

description: LFM2b Lyrics Descriptor Analyses


This dataset provides lyrics descriptors for 580,000 songs, including lexical, structural, diversity-related, readability, rhyme, structural, and emotional descriptors. This dataset was the basis of an analysis of the evolution of song lyrics over the course of five decades and five genres (pop, rock, rap, country, and R&B).


Dataset Generation
As a basis for the dataset, we relied on the LFM-2b dataset (http://www.cp.jku.at/datasets/LFM-2b) of listening events by Last.fm. It contains more than two billion listening records, and more than fifty million songs by more than five million artists. We enrich the dataset with information about songs' release year, genre, lyrics, and popularity information. For quantifying the popularity of tracks and lyrics, we distinguish between the listening count, i.e., the number of listening events in the LFM-2b dataset, and lyrics view count, i.e., the number of views of lyrics on the Genius platform (https://genius.com). Release years, genre information, and lyrics are obtained from the Genius platform.  Genres are expressed by one primary genre. We used https://polyglot.readthedocs.io/ to automatically infer the language of the lyrics and considered only English lyrics. Adopting this procedure, we ultimately obtain complete information for 582,759 songs.


Data and Features
We provide the full dataset, containing features for 582,759 songs (`full_dataset.json.gz`). For each song, the dataset contains track title and artist information, genre, popularity, and release date information, and a wide variety of lexical, structural, diversity-related, readability, rhyme, structural, and emotional descriptors.


For further information on the semantics of the features, we provided a short overview in the following. Please check the implementation of the feature extractor at https://github.com/MaximilianMayerl/CorrelatesOfSongLyrics/ for further details.


- Track and artist
- Genre
- Popularity descriptors:
    - Lyrics view count
    - Last.fm playcount
- Lexical descriptors: 
    - Line counts: Total number of lines, blank lines, unique lines, ratio of blank and repeated lines
    - Token counts: Number of tokens, characters, repeated token ratio, unique tokens per line, and avg. tokens per line
    - Character counts: Number of \texttt{[!?.,:;"-()]} (total amount of these characters and individual counts per character) and digits, ratio of punctuation and digits
    - Token length: Average length of tokens
    - n-gram ratios: Ratio of unique bigrams and trigrams
    - Legomenon ratios: Ratio of hapax legomena, dis legomena and tris legomena
    - Parts of speech: Frequency of adjectives, adverbs, nouns, pronouns, verbs
    - Past tense: Ratio of verbs in past tense to other verbs
    - Stop words: Number and ratio of stop words, stop words per line
    - Uncommon words: Number of uncommon words (i.e., words not contained WordNet)
- Diversity descriptors
    - Compression ratio: Ratio of the size of zlib compressed lyrics vs the original lyrics
    - Diversity measures: Measure of Textual Lexical Diversity (MTLD), Herdan's C, Summer's S, Dugast's U^2, and Maas' a^2 
- Readability Descriptors
    - Readability formulas: Flesch Reading Ease, Flesch Kincaid Grade, SMOG (Simple Measure of Gobbledygook), Automated Readability Index, Coleman Liau Index, Dale Chall Readability Score, Linsear Write Formula, Gunning Fog, Fernandez Huerta, Szigriszt Pazos and Gutierrez Polini
    - Difficult words: Number of difficult words (three or more syllables)
- Rhyme Descriptors
    - Rhyme structures: Numbers of couplets, clerihews, alternating rhymes and nested rhymes
    - Rhyme words: Number of unique rhyming words, percentage of rhyming lines in the lyrics
    - Alliterations: Number of alliterations of length two, three, and four or more
- Structural Descriptors
    - Element counts: Number of sections and verses
    - Distribution: Relation between the number of verses vs. sections and number of choruses vs sections
    - Title occurrences: Number of times the song's title appears
    - Pattern: Verse and chorus alternating, two verses and at least one chorus, two choruses and at least one verse
    - Start: Starts with chorus (binary attribute)
    - Ending: Ends with two chorus repetitions (binary attribute)
- Emotional/Affective Descriptors
    - Sentiment scores: Positivity and negativity scores via AFINN, the sentiment lexicon by Bing Liu et al., the MPQA opinion corpus, the sentiment140 dataset, and the SentiWordNetlexicon
    - NRC: Emotion scores according to the NRC affect intensity lexicon
    - LIWC: Descriptors provided by LIWC
    - Happiness: Happiness score according to labMT