Ad Spend and Campaign RoI Analytics using Segment, Looker, dbt and Google BigQuery
By: Mark Rittman
Tags: Bigquery, Looker, Marketing Attribution, Segment, Dbt

For the first step our usual approach to sourcing data from Facebook Ads and Google Ads would be to use a service such as Stitch or Fivetran to create a fully-managed data extract pipeline into Google BigQuery, but as Segment Connections is already being used to bring in website and mobile app event data we can make use of another Segment feature, Object Cloud App Sources, to replicate the Facebook and Google Ads data into BigQuery instead.

Object Cloud App Sources (as opposed to Cloud Event Sources and regular Event Sources) don’t register ad activity as events in Segment’s event stream but they do the same job as Stitch and Fivetran’s API extract data pipelines, and they’re included in the entry-level Team Plan if you’re a paying Segment customer.

Before we can combine data from the two ad network Segment sources for step 2 in our plan, we first have to standardise table and column naming across the two sources. We can do this using SQL SELECT statements either in SQL views or using a tool such as dbt (“Data Build Tool”) that we use on most client analytics projects.

Google Ads’ API provides spend and click performance data at each of the ad, ad group and campaign levels whereas Facebook’s API provides it at just ad-level, so if you want to base your reports and dashboards at the campaign level you’ll need to roll-up the Facebook data to campaign-level before joining it to Google Ads’ campaign performance data.

To enable comparison of this ad click and cost data from Facebook and Google to the actual clicks and revenue we’re recording via Segment from our website page view data we’ll need a way of connecting these two sets of data together. Facebook Ads provides UTM parameter values for your ads in the API tables it provides, making it easy to look for those values in the session entrance page view data Segment records when people clicking on those adverts arrive at your site.

Google Ads prefers that you use their encrypted gclid (“Google Click Identifier”) tracking codes rather than utm parameters when tracking campaign performance as they enable more campaign metadata to be included in tracking details and then encrypt it, stopping what could potentially be sensitive campaign names or other personally-identifiable data being sent in clear text in the URL querystring.

However only Google Analytics can decrypt the contents of a gclid tracking code rendering them fairly useless when need that metadata to be readable by another process, for displaying the name of the campaign, ad or creative version in a non-GA dashboard report.

As we want our ad spend analytics to be cross-platform and usable outside of GA we instead match on ad_id and store it in the utm_content parameter when setting up tracking codes for new ads, rolling the numbers up to ad group and campaign level from there when needed for reporting.

Put together, the transformations for these first three steps in the process look like the subset shown below of the full transformation graph.

Each step in this graph is a SQL transformation needs to be executed in the right order, waiting for any other transformations the next step depends on to complete before handing-off control to the next stage, and so we use dbt to create a DAG (directed acyclic graph) of transformations to ensure everything gets executed in the correct order, and that we can test everything properly before rolling it out to end-users.

dbt directed acyclic graph to process ad spend data for analysis

You can read more about our use of this tool in Introducing the RA Warehouse dbt Framework : How Rittman Analytics Does Data Centralization using dbt, Google BigQuery, Stitch and Looker and the code we use for ad spend analysis and marketing attribution is contained within the RA Data Warehouse for dbt public git repo on our Github site.

Once we’ve run all of these ad spend and performance transformations and combined it with actual click data from our website through Segment, we end-up with a table that looks like the one below.

Using this data we can put together some useful charts and dashboards in Looker to show how spend on our advertising campaigns has translated into clicks and traffic sent to our website.