Offline Attribution: How to evaluate the impact of Offline actions on Online environments — Part 2 of 2
Tags: Attribution, Data Science, Marketing

Last week in this post we talked a little about Offline Attribution, its parallel with Online, the use of Lift as a method of attribution and some ways of calculating it. Just to recap, let’s look again at the components of the Lift calculation.

The components of Lift: Observed, Baseline and Difference.

We stopped at the different ways of estimating the Baseline. We looked at:

The use of minutes before insertion as the baseline The use of average sessions on other days at the same time as the baseline

So now, without further delay…

3. Statistical Models for Estimating the Baseline

Of course, we would be remiss if we didn’t mention the crème de la crème of data science tools, statistical modeling!

In advance it is important to say that it is quite difficult to start working with Offline Attribution by diving straight into Bayesian time series, machine learning and predictive models, which require high technical and/or statistical knowledge. In addition, it is more difficult to treat the data — to feed the model to be used — than in the two previous methodologies.

Conceptually, however, the story here is the same: Using the minutes preceding the insertion (as well as the corresponding minutes on other days, depending on the model) to estimate a baseline that is then subtracted from the observed data.

In this example we use 30 minutes prior to insertion so that the model can predict a baseline for the 5 minutes after insertion.

The main advantage of using a statistical model for estimating the baseline is that we overcome the problem of over or underestimation because of anomalies and seasonality. A well-constructed model does not depend on long-term conclusions and is able to provide a reliable value for each insertion individually, even though a similar test may never have been done before (e.g. a new broadcaster, a new program, a new product or a new ad).

There are several package and library options for Python and R that help you work with time series. There is no perfect statistical model, so we recommend that you investigate the alternatives and evaluate the trade-off between the complexity of its operation and the reliability of the result.

Measurement of Results

Up to now we have talked a lot about how to calculate values that represent the impact of Offline on Online. But what can we do with those numbers?

Whatever the applied methodology, it is possible to assign to a value to each insertion on TV, for example, a comparative metric with which it is possible to identify the programs that performed better than others.

We can gain some insights from the following sample table:

Model B on Planet TV resulted in the largest volume of Lift (600). For a focus on performance or a more aggressive campaign, this investment may be a priority.

If the objective is economy i.e. the highest possible result at the lowest cost, the scenario reverses. All inserts have a Cost per Lift under $2.50, except for Model B on Planet TV, at a cost of $8.33 per Lift.

However, Tomorrow TV has a lower average cost per Lift of around $1.75. The Real Network comes next with an average cost of around $2.00 and then Planet TV with $5.40. If there were negotiations for the purchase of television space and it wasn’t possible to use only Model A or Model B with a given broadcaster, then the cheapest investment would be insertions on Tomorrow TV.

Points to Note

So far so good, but when working with Offline Attribution we need to keep a few things in mind. Here are some points in relation to what we discussed over the 2 posts.

1. Overlap of Insertions

When we work with attribution in the digital world, no user interaction occurs at the same time as another (except when viewing multiple banners from the same campaign on the same page, but let’s leave that aside for now). Each click occurs at a distinct moment, so it is not common to have overlap in this case.

When we use Lift in Offline Attribution, however, it is possible to display two ads in a very short time, such as one or two minutes, or even the same minute. This becomes even more problematic when we consider how many broadcasters there are on TV and radio.

You can use a variety of rules to mitigate this problem: consider audience ratings, attributing strictly to a channel, using machine learning to understand the attribution rule… Although this post has not addressed the subject, the important thing to keep in mind is that overlap may occur and must be dealt with.

2. Offline Attribution is not just about Lift

Lift is a comparative metric for insertions that provides information beyond audience ratings. Other information can be used, such as the Gross Rating Point itself, rate of new users, how the source/medium distribution stands after insertion, etc.

Particularly in relation to traffic sources, your analysis may even have a focus on Organic or Direct traffic, as the consumer is expected to search for your brand after an Offline placement.

In addition, the methodologies do not necessarily need to be applied only to sessions. You can use the same rationale for installing applications or a conversion in your digital environment.

3. Is TV taking people from my site?

Any of the methodologies can return a negative value for Lift for an insertion. Conceptually this does not make the slightest sense: Would a TV advertisement make fewer people open their application, for example? In cases where there is a negative Lift value, we recommend that you treat the result as zero.

4. Granularity of Data

The more sub-divisions you insert in your analyses (e.g. Lift by broadcaster, program, day, state, product, city etc.) the greater the granularity of the data. This means that for each combination of these dimensions the absolute value of Lift tends to be smaller (the volume of sessions itself, minute by minute, is smaller with a larger number of sub-divisions).

This makes simpler methodologies (such as using minutes before insertion as the baseline) less dependable. Bear in mind that with greater granularity, you need to have a more sophisticated methodology in accordance with the need for more specific insights.

Getting Your Hands Dirty!

Keeping all that we’ve said in mind, the most important step of Offline Attribution is to begin to see that Online and Offline are not two distinct entities that do not mix, but are complementary media in the journey of the consumer.

So prepare to get your hands dirty, choose a methodology, do some testing and start comparing the criteria traditionally used to purchase offline media with the new insights that Offline Attribution brings.