Mobile Marketing Campaigns: ROAS (Return On Ad Spend) | by Gonçalo Guimarães Gomes | Towards Data Science

As a marketing data analyst, the job is to combine efforts to properly conduct a digital marketing data analysis to accurately monitor and evaluate the performance of all of the company’s ongoing digital actions and campaigns.
What I am specifically referring to is the creation of an effective analytical system capable of monitoring the metrics and key performance indicators, from which it is possible to measure and optimize the results of the digital operational efforts our marketing department team is conducting.
The dataset contains information collected during the 1 month since each activation, of some marketing campaigns (Vungle and Facebook partners), implemented to get users to install and try out a mobile application, specifically an online game for Android and iOS smartphones.
The goal is to determine the overall performance and monitor the return on ads spend (ROAS) along the period of analysis delivering updates insights to the marketing department.
Media source (the name of the advertising partner)
Campaign name (the name of the advertising campaign)
Cost (the spend on marketing)
Installs (the number of installs generated)
D01 Total Revenue
D03 Total Revenue
D07 Total Revenue
D30 Total Revenue
The Dx Total Revenue is the total revenue generated within the first x days after the install from users who installed as a direct response to a specific marketing campaign.
CPI (Cost Per Install) CPI = Cost of Campaign / App Installs
Dx ROAS (Return On Ads Spend) in wich x = [1, 3, 7, 30]ROAS is a measurement that determines the efficacy of a digital advertising campaign and it can be understood as the following:ROAS = Total Campaign Revenue / Ad Campaign Costs
The following are some of the business questions we will be answering:
Q1. Are there more Vungle or Facebook campaigns?
Q2. Which partner’s campaign has generated more user installs?
Q3. Of how much is the overall CPI of Facebook?
Q4. How many Facebook campaigns have provided installs above the average?
Q5. Of how much is the overall CPI of Vungle?
Q6. How many Vungle campaigns have provided installs above the average?
Q7. Do both platforms have similar performance (cost vs. install)?
Q8. Of how much is the overall D30 ROAS of Facebook?
Q9. Of how much is the overall D30 ROAS of Vungle?
Q10. Overall Cost vs. D30_ROAS comparison of both platforms
Q11. Is there any correlation between the stages during the campaigns?
Q12. The campaigns with the highest ROAS have the highest Total Revenue?
Q13. Do the campaigns with a ROAS on DAY_1 under 10% break-even after the first month?
Q14. Is it safe to assume the campaigns with a ROAS on DAY_7 higher than 70% will break even after 30 days?
Q15. Monitoring the overall ROAS evolution of both platforms
We will be conducting this Exploratory Data Analysis in Python 3.You can find the entire code here.The ‘mkt_ua_campaigns.csv’ dataset can be downloaded here.
We will start by importing the Python libraries that will be needed (pandas, numpy, matplotlib.pyplot, seaborn and regex), loading the dataset from the CSV file, inspecting the number of rows (89 observations) and features (8 columns), no missing values, and displaying a few rows.
Notice there is an anomalous pattern with the values of the ‘Installs’ variable: when is a Facebook campaign, there is a comma separating the thousands, and there no comma in the Vungle campaigns.
Secondly, the features ‘Cost’, and D(01 to 30) Total Revenue expresses dollars as a categoric type, which includes the ‘$’ symbol.
To transform the variables’ data types into numeric, those issues must be solved and the dataset cleaned as follows.
# Lowering columns' capital letters for easy typingdf.columns = map(str.lower, df.columns)# Selecting columns cols= ["cost","installs","d01 total revenue","d03 total revenue","d07 total. revenue","d30 total revenue"]# Cleaning unvalid charactersdf[cols] = (df[cols].replace({‘,’:’’}, regex=True))df["cost"] = (df["cost"].str.replace('$','')).str.strip()df["d01 total revenue"] = (df["d01 total revenue"].str.replace('$','')).str.strip()df["d03 total revenue"] = (df["d03 total revenue"].str.replace('$','')).str.strip()df["d07 total revenue"] = (df["d07 total revenue"].str.replace('$','')).str.strip()df["d30 total revenue"] = (df["d30 total revenue"].str.replace('$','')).str.strip()# Transforming columns as float (numeric type)df[cols] = df[cols].astype(float).copy()
Now that we have all set up and ready to move forward, it is time to calculate our KPIs from which we’ll be able to analyze the performance of both the overall evolution and by zooming in a campaign by campaign.
First, I’ll start by determining the CPI (cost per install) of each campaign. To perform the calculation, we simply divide the cost of each campaign by the number of installations that each one has generated.
# Determine "Cost Per Install" in a new columndf["cpi"] = df.apply(lambda x: x["cost"] / x["installs"],axis=1)
Let’s also calculate the ROAS (return on ads spend) on the first, the third, the seventh, and the thirtieth days counting from the first day the user has installed the app. To do so, we divide the total revenue generated by the total cost of the campaign for each desired day.
# Calculations to determine "dx_roas" in new columnsdf["d01_roas"] = df.apply(lambda x: x["d01 total revenue"] / x["cost"], axis=1)df["d03_roas"] = df.apply(lambda x: x["d03 total revenue"] / x["cost"], axis=1)df["d07_roas"] = df.apply(lambda x: x["d07 total revenue"] / x["cost"], axis=1)df["d30_roas"] = df.apply(lambda x: x["d30 total revenue"] / x["cost"], axis=1)
Let’s display our final dataset and take a close look at our metrics and KPIs. From this point, we are ready to extract some important and relevant insights from the data.
From a total of 89 campaigns, 77 are running on Facebook and only 12 through the Vungle platform, corresponding to 87% and 13% respectively.
# Side tableabsolut = df["media source"].value_counts().to_frame()percent = (df["media source"].value_counts(normalize=True)*100).to_frame().rename(columns={"media source":"percent"})out_bal = pd.concat([absolut,percent],axis=1).round(decimals=2)display(out_bal)# Pie graphabsolut.plot(kind='pie', subplots=True, autopct='%1.2f%%', explode= (0.05, 0.05), startangle=80, legend=False, fontsize=12, figsize=(16,7))# Paramsplt.xticks(rotation=0, horizontalalignment="center")plt.title("Total marketing campaigns", fontsize=10, loc="right");
Vungle has generated a total of 242112 installs (55%) whereas Facebook is responsible for 197357 installs (45%).
# Bar plotax = df.groupby(["media source"])["installs"].sum().plot(kind="bar", figsize=(9,6), fontsize=12, color=sns.color_palette("rocket"), table=False)for p in ax.patches:    ax.annotate("%.2f" % p.get_height(), (p.get_x() + p.get_width() / 2., p.get_height()), ha='center', va='center', xytext=(0, 7), textcoords='offset points')# Paramsplt.xlabel("media source", fontsize=10)plt.xticks(rotation=0, horizontalalignment="center")plt.ylabel("Absolute frequency", fontsize=10)plt.title("Overall installs proportion", fontsize=10, loc="right")# side tableinstalls = df.groupby(["media source"])["installs"].sum().sort_values(ascending=False).to_frame()installs["percent"] = (installs.apply(lambda x: x/x.sum())*100).round(2)display(installs)
To find the overall CPI, we sum the costs of all Facebook campaigns and divide it by the total Facebook installs. The result is 1.97 dollars. For an easier calculation, we can select only the Facebook campaigns.
# Filtering records from Facebook onlyfb = df[["media source","cost","installs"]].copy()fb = fb[fb["media source"] == "Facebook"].copy()
# Calculating Costs Per Installing (Facebook)fb.cost.sum() / fb.installs.sum()>> 1.9671306312925307# Visualizing Costs vs. Installs sorted by highest costfb = fb.reset_index().copy()fb.drop(["index"], axis=1, inplace=True)fb.plot.line()plt.xlabel('Number of campaigns')plt.ylabel('Absolute Frequency')plt.title('Costs vs. Installs [Facebook]', fontsize=10, loc="right")plt.show()
2563 is the average number of installs per campaign. 14 Facebook campaigns have generated installs above the Facebook average, and only 6 Facebook campaigns have generated installs above the total average (4937).
# Filtering records from Facebook onlyfb_mean = df[df["media source"] == "Facebook"].copy()# Visualizing comparisons between Facebook campaings and between Facebook average campaings and total average campaignsax = fb_mean.groupby("campaign name")["installs"].mean().sort_values(ascending=False).head(15).plot(kind="bar", figsize=(16,9), fontsize=12, color=sns.color_palette("rocket"), table=False)# Paramsplt.xticks(rotation=30, horizontalalignment="right")plt.ylabel("Absolute values", fontsize=10)plt.xlabel("Facebook campaigns", fontsize=10)plt.title("Facebook campaigns vs. Facebook average installs", fontsize=10, loc="right")plt.axhline(fb_mean.groupby("campaign name")["installs"].mean().mean(), linewidth=2, color ="r")plt.axhline(df.groupby("campaign name")["installs"].mean().mean(), linewidth=2, color ="b")plt.legend(('Facebook average installs', 'Total average installs'))plt.tight_layout()
To determine the overall CPI of Vungle, the procedure is similar to the previous one in which we sum the costs of all Vungle campaigns and divide it by the total number of the Vungle installs. The result is 0.74 dollars. Again, we can select only the Vungle campaigns.
# Filtering records from Vungle onlyvun = df[["media source","cost","installs"]].copy()vun = vun[vun["media source"] == "Vungle"].copy()
# Calculating Costs Per Installing (Vungle)vun.cost.sum() / vun.installs.sum()>> 0.7351143272535026# Visualizing Costs vs. Installs sorted by highest costvun.reset_index(drop=True, inplace=True)vun.plot.line()plt.xlabel('Number of campaigns')plt.ylabel('Absolute Frequency')plt.title('Costs vs. Installs [Vungle]', fontsize=10, loc="right")plt.show()
20176 is the average number of installs per campaign. Only 3 Vungle campaigns have generated installs above the Vungle average, and 11 Vungle campaigns have generated installs above the total average (4937).
# Filtering records from Facebook onlyvn_mean = df[df["media source"] == "Vungle"].copy()# Visualizing comparisons between Vungle campaings and between Vungle average campaings and total average campaignsax = vn_mean.groupby("campaign name")["installs"].mean().sort_values(ascending=False).plot(kind="bar", figsize=(16,9), fontsize=12, color=sns.color_palette("rocket"), table=False)# Paramsplt.xticks(rotation=30, horizontalalignment="right")plt.ylabel("Absolute values", fontsize=10)plt.xlabel("Vungle campaigns", fontsize=10)plt.title("Vungle campaigns vs. average installs", fontsize=10, loc="right")plt.axhline(vn_mean.groupby("campaign name")["installs"].mean().mean(), linewidth=2, color ="r")plt.axhline(df.groupby("campaign name")["installs"].mean().mean(), linewidth=2, color ="b")plt.legend(('Vungle average installs', 'Total average installs'))plt.tight_layout()
Actually, there is a colossal difference between both platforms when the comparison is made in terms of cost vs. installations they generate.
With $177980 of total investment in the advertisement, Vungle is capable of generating 242112 installations, whereas Facebook generates 197357 installs but with an amount of $388227 invested.
# Concatenating both Facebook + Vungle datasetscpi2brand = pd.concat([fb, vun])# Bar plotax = cpi2brand.groupby(["media source"])[["cost","installs"]].sum().plot(kind="bar", figsize=(9,6), fontsize=12, color=sns.color_palette("rocket"), table=False)for p in ax.patches:    ax.annotate("%.2f" % p.get_height(), (p.get_x() + p.get_width() / 2., p.get_height()), ha='center', va='center', xytext=(0, 7), textcoords='offset points')# Paramsplt.xlabel("media source", fontsize=10)plt.xticks(rotation=0, horizontalalignment="center")plt.ylabel("Absolute frequency", fontsize=10)plt.title("Overall Cost vs Installs comparison [Facebook + Vungle]", fontsize=10, loc="right")# side tableinstalls = cpi2brand.groupby(["media source"])[["cost","installs"]].sum().sort_values(by="cost", ascending=False)display(installs)
To determine the Return On Ads Spend after 30 days, we need to sum the total revenue on the 30th day and divide it by the total invested on Facebook.
The result is that on the 30th day, 72,65% of ROAS was captured for Facebook.
# Selecting Facebook segment datasetfb_d30 = df[["media source","cost","d30 total revenue"]].copy()fb_d30 = fb_d30[fb_d30["media source"] == "Facebook"].copy()
# Calculating Return On Advert Spend (Facebook) over 30 daysfb_d30["d30 total revenue"].sum() / fb_d30.cost.sum()>> 0.7265414306578368# Visualizing Costs vs. installs over 30 days sorted by highest cost (Facebook)fb_d30.plot.line()plt.xlabel('Number of campaigns')plt.ylabel('Absolute Frequency')plt.title('Costs vs. d30 total revenue [Facebook]', fontsize=10, loc="right")plt.show()
To determine the D30 ROAS of Vungle, sum the total revenue on the 30th day, and divide it by the total invested on the platform.
The result is that on the 30th day, 92,88% of ROAS was captured for Vungle.
# Selecting Vungle segment datasetvun_d30 = df[["media source","cost","d30 total revenue"]].copy()vun_d30 = vun_d30[vun_d30["media source"] == "Vungle"].copy()
# Calculating Return On Ads Spend (Vungle) after 30 daysfb_d30["d30 total revenue"].sum() / fb_d30.cost.sum()>> 0.9287728958309922# Visualizing Costs vs. installs over 30 days sorted by highest cost (Vungle)vun_d30.plot.line()plt.xlabel('Number of campaigns')plt.ylabel('Absolute Frequency')plt.title('Costs vs. d30 total revenue [Vungle]', fontsize=10, loc="right")plt.show()
To determine this result, we must vertically concatenate (axis=0) the above sub-datasets of both D30_ROAS Facebook and Vungle, and then group them by their cost and the total revenue on the 30th day.
The result, displayed in the bar chart below, tells us that 30 days after, Vungle campaigns generated $165303 which is almost the revenue needed to reach that platform break-even point, translated by a total cost of $177980. On the opposite direction is Facebook with a total investment of $388277 and revenue of $282063.
# Concatenating both Facebook + Vungle datasetsroas2brand = pd.concat([fb_d30, vun_d30])# Bar plotax = roas2brand.groupby(["media source"])[["cost","d30 total revenue"]].sum().plot(kind="bar", figsize=(9,6), fontsize=12, color=sns.color_palette("rocket"), table=False)for p in ax.patches:    ax.annotate("%.2f" % p.get_height(), (p.get_x() + p.get_width() / 2., p.get_height()), ha='center', va='center', xytext=(0, 7), textcoords='offset points')# Paramsplt.xlabel("media source", fontsize=10)plt.xticks(rotation=0, horizontalalignment="center")plt.ylabel("Absolute frequency", fontsize=10)plt.title("Overall Cost vs Roas _d30 comparison [Facebook + Vungle]", fontsize=10, loc="right")# side tableroas = roas2brand.groupby(["media source"])[["cost","d30 total revenue"]].sum().sort_values(by="cost", ascending=False)display(roas)
Let’s display a correlation heatmap (Pearson method).
mask = np.triu(df.corr(), 1)plt.figure(figsize=(19, 9))sns.heatmap(df.corr(), annot=True, vmax=1, vmin=-1, square=False, cmap="BrBG", mask=mask);
From the analysis of the correlations map we can easily spot many strong correlations as one might be expecting. But what I am really interested in is to verify the correlation between the ROAS on day 1, 3 and 7 and ROAS on the 30th day.
The d30_ROAS has a positive and strong correlated with d01_ROAS of 71%, an even more strong and positive correlation with d03_ROAS, of 86%, but the strongest correlation is between d07_ROAS and d30_ROAS of 90.37%.
# Calculating corr. coef. between ROAS 7 days and 30 daysdf['d07_roas'].corr(df['d30_roas'])>> 0.9037410337635488# Display scatter plot: d07_ROAS and d30_ROASdf.plot(x="d07_roas", y="d30_roas", kind="scatter", figsize=(9,6), fontsize=12)plt.title("Correlation d07_ROAS and d30_ROAS", fontsize=10, loc="right")plt.xticks(rotation=0, horizontalalignment="center");
We need to check the campaigns with the maximum revenue for d01, d03, d07 and d30, and also the ones with the maximum ROAS for the exact same periods and compare both pairs to verify if they match.
Assigning maximum ROAS on 1st day, and maximum total revenue on 1st day.
# Selecting columns to work withhigh_d01 = df[["campaign name","cost","d01 total revenue","d01_roas"]].copy()
# Highest d01_roasd01_roas_max = high_d01['d01_roas'].max()high_d01.loc[high_d01['d01_roas'] == d01_roas_max, 'campaign name']>> 59    IMT-FB-Android-RoW2-AE-Playables-Feed# Highest d01 total revenued01_total_max = high_d01['d01 total revenue'].max()high_d01.loc[high_d01['d01 total revenue'] == d01_total_max, 'campaign name']>> 0    IMT-FB-iOS-EN-US-AE-Playables
The campaigns with the highest d01_roas and the highest d01 total revenue don’t match.
Assigning maximum ROAS on 3rd, and maximum total revenue on 3rd day.
# Selecting columns to work withhigh_d03 = df[["campaign name","cost","d03 total revenue","d03_roas"]].copy()
# Highest d03_roasd03_roas_max = high_d03['d03_roas'].max()high_d03.loc[high_d03['d03_roas'] == d03_roas_max, 'campaign name']>> 86    IMT-FB-Android-RoW2-AE-Value-Video-FANIS# Highest d03 total revenued03_total_max = high_d03['d03 total revenue'].max()high_d03.loc[high_d03['d03 total revenue'] == d03_total_max, 'campaign name']>> 0    IMT-FB-iOS-EN-US-AE-Playables
The campaigns with the highest d03_roas and the highest d03 total revenue don’t match.
Assigning maximum ROAS on 7th, and maximum total revenue on 7thday.
# Selecting columns to work withhigh_d07 = df[["campaign name","cost","d07 total revenue","d07_roas"]].copy()
# Highest d07_roasd07_roas_max = high_d07['d07_roas'].max()high_d07.loc[high_d07['d07_roas'] == d07_roas_max, 'campaign name']>> 86    IMT-FB-Android-RoW2-AE-Value-Video-FANIS# Highest d07 total revenud07_total_max = high_d07['d07 total revenue'].max()high_d07.loc[high_d07['d07 total revenue'] == d07_total_max, 'campaign name']>> 0    IMT-FB-iOS-EN-US-AE-Playables
The campaigns with the highest d07_roas and the highest d07 total revenue don’t match.
Assigning maximum ROAS on 30th, and maximum total revenue on 30th day.
# Selecting columns to work withhigh_d30 = df[["campaign name","cost","d30 total revenue","d30_roas"]].copy()
# Highest d30_roasd30_roas_max = high_d30['d30_roas'].max()high_d30.loc[high_d30['d30_roas'] == d30_roas_max, 'campaign name']>> 2    IMT-Vungle-iOS-CN# Highest d30 total revenued30_total_max = high_d30['d30 total revenue'].max()high_d30.loc[high_d30['d30 total revenue'] == d30_total_max, 'campaign name']>> 0    IMT-FB-iOS-EN-US-AE-Playables
The campaigns with the highest d30_roas and the highest d30 total revenue don’t match.
The threshold where the revenue reaches and gets equal to its costs determines the break-even point of a campaign.
To check if all campaings that start below 10% on the first day did not break-even after the 30 days, we need to select the campaigns who’s d01_ROAS would be lower than 10%, and select the campaigns from the segment in which the d30 total revenue is higher than or at least equal to its costs.
# Selecting campaigns segment of ROAS d01 under 10%d01_roas_under10 = d01_roas_under10[d01_roas_under10["d01_roas"] < 0.10].copy()# Filtering campaigns segment in wihch "d30 total revenue" is higher than its costd01_roas_under10[d01_roas_under10["d30 total revenue"] > d01_roas_under10["cost"]]
>> The 3 above campaings did break-even after 30 days.
Let’s select all the campaigns with a d07_ROAS higher than 70%, and select all the campaigns from the segment in which the d30 total revenue is lower than its costs.
# Selecting campaigns segment of ROAS 7 days above 70%d07_roas_up70 = df[df["d07_roas"] > 0.70].copy()
# Filtering campaigns segment in which "d30 total revenue" is lower than its cost(d07_roas_up70[d07_roas_up70["d30 total revenue"] < d07_roas_up70["cost"]]).values.any()>> False
All Campaigns with a D07 ROAS higher than 70% did break-even after 30 days.
In the next histogram, we can observe that the majority of campaigns with a ROAS on the 1st day is highly concentrated around 10% with a few campaigns reaching almost 20%.
On the other hand, after 3 days, the majority of campaigns is concentrated between 15% and 35% with some reaching a ROAS between 40% and 55% and even almost 80% in a few cases.
# Display histogram: d01_roas vs d03_roas evolution comparisond01_roas = df.d01_roasd03_roas = df.d03_roasd01 = np.array(d01_roas)d03 = np.array(d03_roas)plt.figure(figsize=(14,8))plt.hist(d01, bins=89, density=True, color="g", alpha = 0.7, label="d01_roas")plt.hist(d03, bins=89, density=True, color="r", alpha = 0.5, label="d03_roas")plt.legend(loc='upper right')plt.xlabel("Relative observations", fontsize=10)plt.xticks(rotation=0, horizontalalignment="center")plt.ylabel("Density scale", fontsize=10)plt.title("d01_roas vs. d03_roas", fontsize=10, loc="right")plt.show()
Although there is a big cluster of campaigns below 20% on the 3rd day, more than half are concentrated between 30% and 45% of ROAS. We can see that there are a few campaigns beyond 60% and even 100% surpassing the break-even.
As expected, after 30 days we can see a fair distribution of the campaigns from the d30_ROAS perspective. There is a cluster that didn’t even go beyond 20% and with this valuable information, the marketing team can investigate what went wrong with the campaigns that don’t have a great return on ad spend and redefine those operations.
To determine the overall performance and estimate the campaigns’ return on ad spend (ROAS), we have analyzed some important metrics and have made some KPI calculations concerning the amount invested in marketing operations and advertising, the number of well-succeeded app installations carried out by the users, and monitored the evolution of the partial return in each timeframe delivering important insights to the marketing team.
In a marketing environment, the metrics are indicators by which the results of tactic operations can be measured, such as purchases made on an e-commerce website, installations of a mobile application, the user’s behavior concerning exposure to an ad, and are not associated with any conversion goal, that is, to a specific objective.
On the other hand, the Key Performance Indicators (KPIs) are management tools that allow monitoring, evaluating and controlling the performance levels of a company, product, or service, regarding specific or general operational objectives, contributing to an overall understanding of the evolution, performance, impact, intensity, and direction of the results concerning the defined marketing strategy.
The marketing report is concluded but so much more could be deeper investigated and analysed from this point on.
You can find the entire code here.
The ‘mkt_ua_campaigns.csv’ dataset can be downloaded here.
Check out other articles you might also like:
towardsdatascience.com
towardsdatascience.com
towardsdatascience.com
Linkedin
Twitter
Medium
GitHub
Kaggle
Email
Good readings, great codings!