Lyrics for Success: Embedding Features for Song Popularity Prediction (2024)

Abstract: Accurate song success prediction is vital for the music industry, guiding promotion and label decisions. Early, accurate predictions are thus crucial for informed business actions. We investigated the predictive power of lyrics embedding features, alone and in combination with other stylometric features and various Spotify metadata (audio, platform, playlists, reactions). We compiled a dataset of 12,428 Spotify tracks and targeted popularity 15 days post-release. For the embeddings, we used a Large Language Model and compared different configurations. We found that integrating embeddings with other lyrics and audio features improved early-phase predictions, underscoring the importance of a comprehensive approach to success prediction.

## introduction predicting music release success is crucial for the music industry and influences artists' signings and careers. strategies are planned before release and adjusted based on success expectations @xcite . post-release efforts could target demographics that may not have initially responded. as depicted in figure 1 , a song reaches peak audience within two weeks of release, going from novelty to stabilized exposure. a song's lifecycle include production, release planning and implementation, and post-release reactions. the further in the lifecycle, the more information can gather to improve a song's success prediction. audio and lyric data are the primary information available from the early stages. while much research in music information retrieval (mir) has focused on utilizing audio data to extract predictive features @xcite , less attention has been given to leveraging lyrics @xcite . we investigate the predictive power of lyrics features in the context of song success prediction. we first compile a dataset of 12,428 spotify tracks and set the popularity at day 15 as the prediction target, which is based on recency and relative quantity of plays @xcite thus a good proxy of actual streams. we split our data into training, validation, and test sets in chronological order. to the best of our knowledge, no previous work has framed music success in this way. second, we combine large language models (llms) lyrics embeddings with stylometric features, and integrate them with other features to train regression models. third, we perform an evaluation to compare the efficacy of using lyrics features alone versus integrating them with other features across the phases of the song's lifecycle. llm embeddings improve the contribution of lyrics features already after the song production phase. we make our code openly available foot_0 . ## related work hit song science aims to predict whether a song can attain a "hit" status based on song features extracted by mir techniques @xcite . this has sparked debates about its efficacy @xcite , yet subsequent work have tackled the challenge of music popularity prediction by framing it as a classification task of "hit" or "miss" based on: rankings in music charts like billboard @xcite ; listing in playlists (araujo et al., 2020); or categorisation of levels of popularity @xcite . music success prediction has also been tackled as a regression problem, based on the spotify popularity scores @xcite ) from 0 to 100. xg-boost @xcite has been praised for its efficacy @xcite . other methods included lyrics features @xcite , genrebased and cluster-based approaches @xcite , and vector autoregressive model @xcite . we refer to @xcite for a comparative analysis. lastly, we highlight the work of berjamin et al. @xcite in which features are grouped in song intrinsic (e.g., from audio), song extrinsic (e.g., about the release context) and crowdsourced opinions. in turn, we group features by their availability over time. our research emphasizes the utility of llm embeddings from lyrics, by evaluating them alongside stylometric features @xcite . to the best of our knowledge, we are the first ones to use lyrics embeddings to predict music popularity within a chronological regression framework. ## data collection and preprocessing we monitored two sets of spotify playlists: the first, relevant for the french market, of 299 playlists from 22-07-2019 to 17-09-2022; the second, relevant to the uk market, of 235 playlists from 24-11-2020 to 03-02-2023. we kept track of the songs that were released and listed during this time, totalling 24,266 tracks. we stored the metadata made available through spotify's api. we kept evolving data (like song popularity, artist's followers, positions in playlists, etc.) recording their dynamics daily. for the lyrics, we queried the title and artist's name through the genius api @xcite . we focused on success prediction for new releases, which is more relevant to the music industry. by tracking song's popularity daily, we set the regression target for success as the popularity on the 15th day after release, uniformly for every song. figure 1 shows that, by that day, novelty has faded and popularity stops growing. tracks with no value for the popularity target (due to faulty or late collection) were removed. after manually inspecting randomly sampled data, lyrics were automatically preprocessed by: cleaning recurrent artifacts in the text; removing tracks with no english lyrics. for language extraction, a transformer model for 51 language classification was used @xcite . 12,428 tracks remained after preprocessing. type # features lexical 14 token count, unique token ratios, avg. token length, repeated token ratio, hapax dis-/tris-/legomenon, unique tokens/line, avg. tokens/line, line counts, punctuation and digit ratios, stop words ratio, stop words/line linguistic 1 lemma ratio semantic 4 vader scores (4) syntactic 3 pronoun frequency, past tense ratio we explored (i) fine-tuning the embeddings for popularity prediction by adding a regression layer on top of the original language model to directly optimize for predicting popularity scores; (ii) finetuning the llm through unsupervised masked language modeling (mlm), thus continuing to train the model to refine its contextual understanding. we identified six strategies for lyrics embeddings: 1) b, embeddings from the pre-trained all-mpnet-base-v2 model; 2) b-reg, b + finetuning for regression; 3) b-red-reg b + dimensionality reduction + fine-tuning for regression; 4) ft, embeddings from the fine-tuned all-mpnet-base-v2 model; 5) ft-reg, ft + finetuning for regression); 6) ft-red-reg, ft + dimensionality reduction + fine-tuning for regression. regression model. we used lightgbm (lgbm), a tree-based model built on gradient boosting @xcite . the model was trained with 5-fold cross-validation @xcite , with parameters: @xmath0.001, @xmath1, 000, and hyperparameter grids: @xmath2, -1, @xmath3, 60, @xmath4.5, 0.7, 1. feature importance is measured by the frequency of a feature's use in lgbm's decision splits. ## experimental set-up we first select the embedding strategy that best predicts popularity, then measure the contributions of each group of features to the popularity prediction. we divided the data into train, validation and test sets, in a 80/10/10 split based on time of release. songs in the training set were released before those in validation, which were released before those in the test, to reflect a realistic scenario. after preprocessing, the train, validation and test sets contained 9, 812, 1, 391 and 1, 225 songs respectively. we used both the training and validation sets to fine-tune the llm with mlm, as this method is unsupervised. for the regression task, we fine-tuned the embeddings using the training set and evaluated them on the validation set. we reduced the embeddings to dimensions 5, 10 or 20, using either euclidean or cosine distance for umap. we assessed ten different embeddings for each of the three sizes: b, b-reg, ft, ft-reg with umap l2 or umap cosine (4 • 2 models); b-red-reg and ft-red-reg (2 models). 30 embeddings were compared on the training and validation sets joint together. for comparability with sizes from other feature groups (see ## conclusion we incorporated lyrics features into regression models to predict the popularity of a song at day 15. we experimented with various models with stylometric and embedding-based features, selected the best ones on the training and validation sets, and evaluated how the prediction improved if we included lyrics features at different stages of the song life. we find that lyrics embeddings are useful for song popularity prediction at early stages, complementing with other features. future work may benefit by the rapid advances of llms. multilingual models could be used to process lyrics from languages other than english. we also plan to extend the features to include text aesthetics @xcite and social media communications. there is a lack of data about marketing campaigns, despite their centrality in the business, and it would be valuable to quantify the predictive power derived from those interventions. spotify popularity was set as a proxy for music success, yet this metric does not offer the same resolution as actual streams, which have a richer dynamic. other aspects could also be targeted beyond popularity, such as relative success or potential audience, providing new insights on the science of success @xcite .