Improving the return on investment (ROI) of music industry using machine learning
By: Siong Jong Hang
Tags: Automl, Machine Learning, Music Business

Background

According to IFPI, The global recorded music revenues for 2018 were US$19.1 billion. The recording industry is highly competitive and is dominated by three big production companies. Record company-driven investment, innovation and partnerships are supporting artists to connect with fans around the world whilst also yielding dynamic growth in high-potential markets.

At the core of this industry lies the artists where record labels provided them with the necessary resources to sell and promote their music. The cost of producing music include studio recording, marketing, distribution, and tours in order to earn a percentage of the profits from album sales, singles, streaming, digital downloads and concert tickets.

In order to find the best talents and vocals to increase the recording labels’ chances on topping charts, singing competitions such as American Idol, The Voice, and X Factor and talent shows such as America’s and Britain’s Got Talent were organized. By topping charts, record labels are able to increase album sales, streaming and other events that generate revenues, thereby, increase the return on investment (ROI).

However, according to USA Today the best vocals did not always translate into commercial success. An article published in Medium.com addresses some of the shortcomings in those talent shows.

In this exercise, I embarked on using data science thought process coupled with machine learning to address these challenges. I first used machine learning (ML) technique to train a ML model to predict if a song reaches Billboard Top 10. Then, I attempted to make use artifacts produced by the ML model to turn lowly ranked songs into chart topping songs. In turn, increase ROI.

Data

Data used in this experiment was obtained from course 15.071 Analytics Edge conducted by Dimitris Bertsimas at Massachusetts Institute of Technology via edX. Kaggle has a similar dataset from Spotify but did not have a group of columns attributed to musical timbre.

The dataset has the following columns and definitions:

Timbre

According to Wikipedia, timbre in music is also known as tone color or tone quality. It is the perceived sound quality of a musical note, sound or tone. Timbre distinguishes different types of sound production, such as choir voices and musical instruments, such as string instruments, wind instruments, and percussion instruments. It also enables listeners to distinguish different instruments in the same category (e.g., an oboe and a clarinet, both woodwind instruments).

Timbre also allows the ear to distinguish sounds which have the same pitch and loudness. Timbre is mainly determined by the harmonic content of a sound and the dynamic characteristics of the sound.

Machine Learning

In recently years, we have seen tremendous amount of AI development culminated in sophisticated algorithms and products. One of the products is automated machine learning (AutoML). In this exercise, I have used a commercially available AutoML — H2O Driverless AI (DAI) from Mountainview, California based H2O.ai.

DAI provides Basic and Expert settings for data scientist to configure machine learning experiments. There are 4 options in the Basic Setting — Accuracy, Time, Interpretability and Scorer. By adjusting Accuracy, DAI determine which algorithms, cross validations and ensemble to use. At high configuration, more complex algorithms such as XGBoostGBM will be used. At lower configurations, less complex algorithms like Decision Tree and GLM might be deployed so that algorithms could compete with each other. Time option specifies the relative time in term of the number of iterations allowed but DAI might perform early stopping if there is convergence. Higher number might increase the duration to complete the training. Interpretability affects the number and complexity of data transformation which be used during the model building. At lower interpretability, DAI uses a greater number of and more complex data transformer and vice versa. The Scorer indicates which metric to be optimized for this experiment. AUC was the default and remained so. It can be easily replaced by others such as Accuracy, F1, F2, etc. More details can be found at the documentation site.

The Results

The songs data was uploaded to DAI to build machine learning models. The results of the best model are summarized as follow:

The results showed a reasonable 86% accuracy on the Test set while AUC achieved 81% using XGBoostGBM as the final model.

Portable Machine Learning Model

The machine learning model built in DAI was downloaded as a zip file, mojo.zip which can be used outside of DAI as a portable scoring model. Thus, the MOJO model could be used at edge devices. The MOJO model packaged into a java archive file (JAR) which was executed using java command. The model was used to score 2 new datasets in order to improve the probability of the selected songs to reach Billboard Top 100. Details are described in the following sections.

Interpretability Artifact of ML

After the machine learning model was built, DAI generated Interpretability artifacts at global as well as row level. Reason Codes, K-LIME chart, Decision Tree, Shapley Values are some of the available interpretability components. I have used the resulting Global Reason Code (global) and K-LIME (row level) to make changes to the existing data in an attempt to improve the probability songs to reach Billboard Top 10.

A Case for Whitney Houston

Whitney Houston was one of the most commercially successful artists in history. She has achieved a remarkable 11 №1 hits, 23 Top 10 and a total of 40 songs on Billboard chart throughout her career. One of her Top 100 songs, Million Dollar Bill peaked only at #100. In order to help Whitney to improve the probability to reach Top 10, I had resorted to Global Reason Code to make changes to some of the features.

Image of Whitney Houston is obtained from https://www.pngitem.com/middle/iThTboJ_whitney-houston-pre-grammy-2009-png-download-whitney/

The following diagram describes quantitatively how each feature has changed when songs predicted as 1 were compared to those predicted as 0. The table shows the existing values for the features and the column on the right shows how much increment or reduction of those values that I had applied to produce new values.

The new data obtained were scored using MOJO which had previously been downloaded from DAI to my local computer:

The resulted predictions before and after modifications based on Global Reason Code is summarized as follow:

With this, the probability of the song Million Dollar Bill to reach Top 10 increased from 36% to 85%.

Using Partial Dependence Plot to Improve Predictions

In the previous analysis, I have used global values from artifacts generated by DAI to help Whitney Houston’s Million Dollar Bill to enter Billboard Top 10.

Another interpretability components generated by ML model in DAI was Partial Dependence Plots. The plots were examined to find the point that made sudden uptrend adjustment in the prediction probability. Upon identification of such points, their values will be copied to replace the existing values for the corresponding features.

In order to demonstrate this, I have selected an Eric Clapton’s song, Circus. This song did not at all make it to Billboard chart. Let’s see if we can adjust the features based on DAI’s Partial Dependence suggestions to improve its standing.

Eric Clapton

In the next analysis, let’s examine Circus in the K-LIME chart. The chart displayed the Reason Codes for this song not making it to the chart.

Image of Eric Clapton was captured from YouTube video at https://www.youtube.com/watch?v=veGF9Vjo_IE

Subsequently, I visually inspected Partial Dependence plots for each feature, details for each feature from the plot are described in the subsequent section.

Pitch

Timbre 0 Minimum

Timbre 0 Maximum

Timbre 1 Minimum

Timbre 4 Maximum

Timbre 6 Minimum

Timbre 10 Maximum

Timbre 11 Minimum

The following table summarizes the adjustment made to the existing values. MOJO model was then used to predict the data with the new values. Predicted results were then compared to determine if the adjustments improved the likelihood of Eric Clapton’s Circus to become a chart topper.

The results of predictions using MOJO is summarized as follow:

Literature Survey

In a research carried out by scientists at University College London published in The Quarterly Journal of Experimental Psychology which had been made freely available at the National Library of Medicine, timbre was discovered to affect perceptions of emotion in musics. Their findings showed that timbre (instrument identity) independently affects the perception of emotions in music after controlling for other acoustic, cognitive, and performance factors.

Concluding Remarks

This exercise has demonstrated machine learning as a potential vehicle in transforming the music industry. It has also illustrated the values of ML interpretability in assisting businesses in their decision making by pinpointing the most important factors to attend to in addressing business challenges. By making use of such provisions, record labels are able to execute their operations more accurately, more effectively and plan their investment better to yield higher ROI on talents, events, instruments, etc.