Predicting the impact of social media advertising on sales with linear regression | by Sonali Verghese | Towards Data Science

This article will detail the steps involved in performing a simple linear regression using the package R. It will also cover relevant and essential introductory statistics.
A linear regression measures the relationship between a response variable Y and a predictor variable X. Does Y increase with a unit increase in X? Or does it decrease with a unit increase in X? We want to predict the Y variable from X using a linear relationship. We can understand how two variables might be related to or influenced by each other.
For example, suppose Y is the number of stabbings per 1000 people in different neighbourhoods across London — the larger the number of stabbings, the greater the rate of crime. If X is the median house value of different neighbourhoods, we can use linear regression to test if neighbourhoods with lower median house values have a larger crime rates or vice versa.
Mathematically, a linear regression can be expressed as follows:
Y=β1+β2X+ε
The response (dependent) variable Y is what we are trying to predict.
The predictor (independent) variable X is used to predict the response.
β1 is the intercept and is a constant value. If X = 0, then Y will depend entirely on β1.
β2 is the slope of the regression line. We refer to these terms as coefficients.
ε is the residual error term. This will show how much of Y can be explained by X. The error term has mean value of 0.
Let’s visualise this with an example.
We will use the marketing data set included with the datarium package, which contains the advertising budgets (in thousands of US dollars) for three media (Facebook, YouTube and newspapers) of a fictional company and sales data for that company.
Formulating a research question can be a useful method to guide the exploratory data analysis process. It can help to clean the data set and prepare it for rigorous analysis by eliminating unwanted variables, dealing with missing values etc. Our general question will be:
What is the effect of Facebook advertising on the company’s sales, given the effects of YouTube and newspaper advertising?
We want to develop a linear model to explain the relationship between the predictor variablefacebook and the numeric response variable sales. In this article, we will use the package R to perform only a simple linear regression. Here’s a simple linear regression equation:
Sales=β0+β1*Facebook+ε
A multiple regression, which involves more than one predictor variable, will be performed and explained in another article (coming soon!). For example, if we want to test the effects of Facebook advertising on company sales given the advertising budgets for YouTube and newspaper advertising, our multiple regression equation will look like this:
Sales=β0+β1*Facebook+ β2*YouTube +β3*Newspaper+ε
We will first begin with a simple linear regression to find the best line to predict the response sales as a function of the predictor facebook.
First, install the tidyverse, ggpubrand datarium packages in R Studio. Import the marketingdata set from the datarium package.
library(tidyverse)library(ggpubr)data(“marketing”, package = “datarium”)
To start with, we will use the str() function to get a snapshot view of the structure of our data. We can see there are 200 rows of data and 4 variables — youtube, facebook, newspaper and sales. We can also see the first few individual values for each variable. Additionally, we can see the classes of each column of variables — each of our variables is classed as ‘numeric’.
> str(marketing)'data.frame':	200 obs. of  4 variables: $ youtube  : num  276.1 53.4 20.6 181.8 217 ... $ facebook : num  45.4 47.2 55.1 49.6 13 ... $ newspaper: num  83 54.1 83.2 70.2 70.1 ... $ sales    : num  26.5 12.5 11.2 22.2 15.5 ...
We can also use dim() to understand the dimensions of the data frame i.e. the number of rows and columns.
> dim(marketing)[1] 200 4
Next, we will use the head() function to get a glimpse of our raw data. By default, this function will return the first six rows but we can view any number of rows, depending on our preferences. We want to see the first 10 rows:
> head(marketing, 10)   youtube facebook newspaper sales1   276.12    45.36     83.04 26.522    53.40    47.16     54.12 12.483    20.64    55.08     83.16 11.164   181.80    49.56     70.20 22.205   216.96    12.96     70.08 15.486    10.44    58.68     90.00  8.647    69.00    39.36     28.20 14.168   144.24    23.52     13.92 15.849    10.32     2.52      1.20  5.7610  239.76     3.12     25.44 12.72
Let’s look at the end of our data set as well with tail. This is very useful simply because, in some data sets, the last few columns may contain totals or summaries of the data, which can be irrelevant.
> tail(marketing)    youtube facebook newspaper sales195  179.64    42.72      7.20 20.76196   45.84     4.44     16.56  9.12197  113.04     5.88      9.72 11.64198  212.40    11.16      7.68 15.36199  340.32    50.40     79.44 30.60200  278.52    10.32     10.44 16.08
Now, I want to pull up some summary statistics using summary.
> summary(marketing)youtube          facebook       newspaper      Min.   :  0.84   Min.   : 0.00   Min.   :  0.36   1st Qu.: 89.25   1st Qu.:11.97   1st Qu.: 15.30   Median :179.70   Median :27.48   Median : 30.90   Mean   :176.45   Mean   :27.92   Mean   : 36.66   3rd Qu.:262.59   3rd Qu.:43.83   3rd Qu.: 54.12   Max.   :355.68   Max.   :59.52   Max.   :136.80       sales       Min.   : 1.92   1st Qu.:12.45   Median :15.48   Mean   :16.83   3rd Qu.:20.88   Max.   :32.40
We must understand what these important terms and numbers represent.
Mean: The sum of all the values divided by the total number of values. In our example data set, mean budget for YouTube advertising is 176.45 or $176,450. The average Facebook advertising budget, on the other hand, is much lower at 27.92 or $27,920.
Median: The middle number in a sorted list with odd numbers. In a list with even numbers, the median is average of the two numbers that divide the sorted data into upper and lower halves.
Min: This is the lowest number value for each variable.
Max: This is the highest number value for each variable.
So we’ve already gleaned some interesting information about our data set. We know the company spends more, on average, on YouTube advertising than on Facebook or newspaper advertising.
We can get a deeper insight into the distributions of data with quantile.
> quantile(marketing$facebook)   0%   25%   50%   75%  100%  0.00 11.97 27.48 43.83 59.52
Great! So we have some preliminary information about our data set and now we can plot some graphs. We will use the ggplot() function available in the ggpubr() package. First, to demonstrate why we use the ggpubr package to plot our graphs, we will use plot(), R’s inbuilt plot function with basic graphics.
> plot(marketing$sales, marketing$facebook)
Now, we will use ggplot().
> ggplot(marketing, aes(x = facebook, y=sales)) + geom_point() + stat_smooth()
Facebook vs Sales
ggplot() allows for a more aesthetically-pleasing way to plot graphs. Here, the plot is the marketing data set with ‘aes’ or aesthetic mappings, which are derived from the facebook and sales variables, a set of points and a smoother.
The (publication-ready) graph above suggests a positive, linear relationship between the sales and facebook variables. This would suggest that an increase in the advertising budget would result in an increase in the company’s sales. Out of curiosity, we can also visualise the relationships between sales and YouTube advertising and sales and newspaper advertising as well.
YouTube vs Sales
Newspaper vs Sales
Interestingly, while the relationship between sales and youtube advertising is also linear and positive, the relationship between salesand newspaper advertising is not — it is a curved line which actually begins to negatively slope at very low and very high levels of newspaper advertising.
What about the correlations between the independent and dependent variables? Correlation analysis will determine the strength of a relationship between two continuous variables. We must compute the correlation coefficient to perform this analysis. Correlations take values between +1 and -1. A value close to 0 suggests a weak correlation between two variables.
In our example data set, the variables will be positively correlated if high values of sales go with high values of Facebook advertising and vice versa. As we’ve already seen in the graphs, there is a positive correlation between sales and Facebook advertising. However, there is a stronger positive correlation between sales and YouTube advertising.
The correlation between sales and newspaper advertising is the weakest. This suggests that much of the variation in sales (Y) is unexplained by newspaper advertising (X). To measure a significant variation in sales, we would need an explanatory variable with a strong correlation i.e. YouTube or Facebook.
> cor(marketing$sales, marketing$facebook)[1] 0.5762226> cor(marketing$sales, marketing$youtube)[1] 0.7822244> cor(marketing$sales, marketing$youtube)[1] 0.7822244
To perform a simple linear regression in R, we will use lm().
> model1 = lm(sales ~ facebook, data = marketing)> model1Call:lm(formula = sales ~ facebook, data = marketing)Coefficients:(Intercept)     facebook      11.1740       0.2025
Now, let’s generate a graph for our regression. The regression line will try to minimise the sum of squared residual values (RSS or residual sum of squares).
> ggplot(marketing, aes(facebook, sales)) + geom_point() + stat_smooth(method = lm)
Regression concepts:
Fitted values and residuals: In general, real world data will not fall exactly on the regression line which is why the regression equation contains an error term ε. The fitted/predicted values are are denoted with ^. If β is denoted with a ^, then the coefficient is estimated. Statistics differentiates between estimates and known values because estimates are uncertain and known values are fixed. The error term ε will contain this difference between estimated values and known values.
Least squares: In a simple regression, we would use the OLS or ordinary least squares method to estimate how well our regression model fits the data. The coefficients denoted with ^ are the values that minimise RSS. However, in a multiple regression we would use maximum likelihood to estimate model fit.
The estimated regression line is:
Sales=11.174+0.2025*Facebook+ε
The intercept (β0) is 11.174 and can be interpreted as the predicted dollar sales value for a Facebook advertising budget value of zero. So for a Facebook advertising budget equal to zero, we can expect sales of 11.174 *1000 = $11,174.
The regression coefficient (β1) shows that for a Facebook advertising budget equal to 1000 dollars, we can expect an increase of 202.5 units (0.2025*1000) in sales i.e.sales = 11.174 + 0.2025*1000 = 56.44 units. This represents a sales of $213,670.
Now that we have our regression model, we need to understand how to interpret it. It is not enough to create a regression model. We must check if (a) we have a statistically significant relationship for our regression to hold, and (b) our model fits the data well. A regression model that fits the data well is such that changes in X lead to changes in Y.
Before proceeding any further, we must understand standard errors and hypothesis testing.
Standard errors:
The standard error measures how much our coefficient estimates vary from the actual average value of our response variable. The standard error can be used to compute the confidence intervals. Confidence intervals quantify uncertainty around regression coefficients. By using the confint() function, we learn that there is a 95% chance that the interval [0.12,0.24] will contain the true value of β1.
> confint(model1)                2.5 %     97.5 %(Intercept) 9.8419062 12.5060253facebook    0.1622443  0.2427472
Hypothesis testing:
Standard errors can be used to perform hypothesis tests on the regression coefficients. The most common hypothesis test would be testing the null hypothesis versus the alternative hypothesis.
The null hypothesis (H0): The coefficients are equal to zero i.e., there is no relationship between the predictor and response variables.
The alternate hypothesis (H1): The coefficients are not equal to zero i.e., there is a relationship between the predictor and response variables.
To test the null hypothesis, we have to determine whether our estimate for β1 is sufficiently far from zero such that β1 is non-zero. If the standard error of our estimate of β1 is sufficiently small, then even small values of our estimate of β1 will provide evidence against the null hypothesis.
How do we measure how far our estimate of β1 is from zero? The t-statistic will measure the number of standard deviations our estimate of β1 is away from 0. We need to use the regression model to reject the null hypothesis and prove there is a relationship between the sales and facebook variables.
To interpret our linear regression, we will display a statistical summary of our model. To do this, we use summary().
> summary(model1)Call:lm(formula = sales ~ facebook, data = marketing)Residuals:     Min       1Q   Median       3Q      Max -18.8766  -2.5589   0.9248   3.3330   9.8173 Coefficients:            Estimate Std. Error t value Pr(>|t|)    (Intercept) 11.17397    0.67548  16.542   <2e-16 ***facebook     0.20250    0.02041   9.921   <2e-16 ***---Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1Residual standard error: 5.13 on 198 degrees of freedomMultiple R-squared:  0.332,	Adjusted R-squared:  0.3287 F-statistic: 98.42 on 1 and 198 DF,  p-value: < 2.2e-16
Let’s break down this summary output:
Residuals: As mentioned before, the residuals are the estimated coefficients denoted by ^. Residuals are the difference between the actual and estimated values. The distribution of our residuals should ideally be symmetrical.
Coefficients: Our coefficients β0 and β1 represent the intercept and slope respectively. We have already interpreted these coefficients in the section above.
The coefficient standard error, as mentioned above, measures how much our coefficient estimates vary from the actual average value of our response variable. In other words, it measures the accuracy of coefficient estimates. The closer our standard error is to zero, the better.
The coefficient t-value measures how far (in standard deviations) our coefficient estimate is from 0. A large t-value, relative to standard error, would provide evidence against the null hypothesis and indicate that a relationships exists between the predictor and response variables. Predictors with low t-statistics can be dropped. Ideally, the t-value should be greater than 1.96 for a p-value to be less than 0.05.
The coefficient — Pr(>t) represents the p-value or the probability of observing a value larger than t. The smaller the p-value, the more likely we are to reject the null hypothesis. Typically, a p-value of 5% or less is a good cut-off point. Note the ‘Signif. Codes’ associated to each estimate, in our example. Three asterisks represent a highly significant p-value. Since the relationship between sales and Facebook advertising is highly significant, we can reject our null hypothesis.
Residual standard error: This measures the quality of our regression fit. It is the average amount the sales variable will vary from the true regression line.
Multiple R-squared: Besides the t-statistic and p-value, this is our most important metric for measuring regression model fit. R2 measures the linear relationship between our predictor variable (sales) and our response / target variable (Facebook advertising). It always lies between 0 and 1. A number near 0 represents a regression that does not explain the variance in the response variable well and a number close to 1 does explain the observed variance in the response variable. In our example, the adjusted R2 (which adjusts for degrees of freedom) is 0.3287 — only 32.87% of an increase in sales can be explained by Facebook advertising. If we perform a multiple regression, we will find that the R2 will increase with an increase in the number of response variables. (Note: the relationship between YouTube advertising budgets and company sales is stronger, with an R2 of 0.612. Therefore, YouTube advertising is a better predictor of company sales data, when compared to Facebook advertising)
F-statistic: This is a good indicator of whether there is a relationship between Y and X. The further our F-statistic is away from 1, the better our regression model. In our example, the F-statistic is 98.42, which is relatively larger than 1 given the size of our data set (200 observations). The F-statistic is more relevant in a multiple regression model.
So the four key indicators of model fit we would need to primarily focus on are the t-statistic, p-value, R2 and F-statistic. The larger our t-statistic, the smaller the p-value. The smaller the p-value, the greater the odds of a relationship between X and Y. R2 measures how well a model fits the data — if R2 is close to 1, then this indicates that a large proportion of the variation in Y can be explained by X. The F-statistic shows the overall significance of the model. A large F-statistic will correspond to a statistically significant p-value (p < 0.05).
To summarise, we have performed a simple linear regression and have covered some basic introductory statistics as well. This is by no means a comprehensive analysis of the marketing data set but simply an example of how to perform and interpret a simple linear regression with R. It’s a good starting point, especially when attempting to understand the relevance of important statistical concepts like t-statistic, p-value and standard error. In the next article, we will work on a multiple regression of this same data set, in continuation of this article. The multiple regression exercise will be a step towards a comprehensive data analysis project.