Level Up Your Conversion Rate in Just a Few Lines of Python
By: Benjamin Engen
Tags: Pyvis, Google Analytics, Conversion Optimization, Python

Level Up Your Conversion Rate in Just a Few Lines of Python

Introduction

Is your web experience costing you sales? If you aren’t making an active effort at Conversion Rate Optimization (CRO) the answer is yes. Yet another in the panoply of acronyms that abound in business, CRO is one of the cheapest and easiest ways to boost sales. A poor web experience on a lead’s journey to becoming a customer can sometimes be remedied by something as small as adding or removing a single button, reducing text, or cutting a page out of the process altogether.

But, to understand where you’re losing sales you first need to understand how users move through the collection of pages that make up your website. You need to examine the flow that they follow on the path to becoming a sale.

Google Analytics is free and almost universally adopted for web analytics. But, its path analysis options can be frustratingly linear — and linear is rarely the path customers follow to a sale.

Luckily, we can use the data from Google Analytics coupled with a few lines of Python to create an intuitive, interactive network graph that will allow us to examine how users travel through your site on their way to a conversion.

To do so we’ll:

If you’re not interested in the step-by-step you can access the full code for this undertaking in this GitHub repo.

Configure Google Analytics API

To begin you’ll need to allow yourself API access to Google Analytics. Google makes this process very easy but, if this is your first foray into the depths of Google’s Cloud Platform, it could be daunting.

Start by heading over to https://cloud.google.com/ and select either ‘Console’ or ‘Get started for free’ if you’re starting entirely from scratch.

Image by author

From here you’ll likely need to create a new project. Select the drop-down arrow to the left of the top search bar and then choose ‘New Project’ from the top-right of the pop-up.

Image by author

Now that you have your project, you’ll need to enable the Google Analytics API on your account. Since this is Google, it’s easy to search for that functionality. Head up to the top search bar and enter ‘Google Analytics API.’ Several options will come up, select ‘Google Analytics Reporting API.’ Then click ‘Enable.’ You shouldn’t be able to miss it.

Once you’ve done that you should find yourself on the ‘Overview’ portion of the Google Analytics Reporting API. From here you’ll want to select ‘Credentials’ on the left menu, then choose ‘+Create Credentials’ at the top of the new screen, and ‘Service Account’ from the options that pop up.

Image by author

This will take you to a three-step flow. Give your service account a memorable name, choose ‘Owner’ in Step 2 for project access, and then finish.

This should bring you to the ‘Service Accounts’ screen and you should see your new service account listed. Now you’ll need to download a key to your computer to use for access. Select the three dots under ‘Actions’ next to your service account and then choose ‘Manage keys.’

Image by author

On the next screen select the ‘Add Key’ dropdown menu and then choose JSON for your ‘Key type.’ This will at once create a JSON file and you’ll be prompted to download it to your computer. Download this file to the same folder that your code will reside in.

The last step before we dive into the code is to give this new service account access to the Google Analytics (GA/UA) account that you’ll be drawing the data from. As you can see in the above screenshot, the service account has its own email address. Copy this address.

Head over to your Google Analytics account. Choose the ‘Settings’ gear on the bottom-left of the screen and then ‘View Access Management’ for the view you’ll want to build your graph from. Add the service account email address that you copied from Google Cloud Platform as a viewer. While you’re here, note the ‘View ID’ of the view you’re adding your service account to. You can find this on the very top drop-down to the left of the search bar.

Image by author

Now we can get on with the actual coding!

Download Google Analytics Data with Python

In addition to the ubiquitous pandas, you’ll also need:

apiclient and oauth2client — to connect to the Goolge Analytics API.

pyvis — to create our network graph visualization.

All the necessary packages can easily be obtained via PIP.

In addition to importing your packages, you’ll need to define a few variables upfront. Most importantly, you’ll need to enter the name of the JSON file that you downloaded from Google Cloud Platform that contains your API access key. This file is named ‘KEY_FILE’ below. You’ll also need to create a variable called ‘VIEW_ID’ and give it the View ID of your Analytics property as noted earlier. Note, that while this is a number you should store it in Python as a string.

Now we’ll need to define several functions to help us access our Google Analytics data. We’ll create ‘initialize_analyticsreporting’ to build the API connection, ‘get_report’ to define the dimensions and metrics we want to download and retrieve them, and ‘response_to_df’ to grab the Analytics data, make sure it’s completely downloaded, and save it in a pandas dataframe so it’s more orderly.

Note that in the ‘get_report’ function, lines 23 and 24, we’re passing in the dimensions and metrics we want to retrieve as variables as these are likely to change often. The function needs the dimensions and metrics as dictionaries which we generate from lists that are passed to the function.

Google has a very easy-to-navigate site with the names for all of the dimensions and metrics that can be downloaded. Using this webpage we can make a list for the dimensions, and another for the metrics we wish to download. We’ll also need to define the date ranges we wish to examine. The Google Analytics API is handy in that it supports descriptive, relative, time frames like ‘14daysAgo’ and ‘yesterday.’ With that done we can call our Analytics functions.

…and inspect our resulting data.

Inspect, Clean & Format Data in Pandas Dataframe

Everything looks pretty good. …except… your page paths likely still have the query parameters attached to them. (Unless you planned ahead and have been filtering this out on your Analytics view). In order to get a true sense of the traffic between all your pages, we’ll need to remove those query parameters and aggregate the results by the true page paths. While we’re at it I’m also going to make sure to store the numbers as numbers and not strings.

Create a Network Graph to Visualize Your Traffic

With our data now cleaned and organized, we can assemble it as a network and visualize it.

The core of the visualization functionality is adding your nodes and edges with the ‘.add_node’ and ‘.add_edge’ lines and then displaying the HTML graph with ‘.show’

You can see I’ve made a few modifications to the data and display. For example, at the beginning, I’ve narrowed the paths to display to the top 10% most traveled in order to keep the graph from becoming too cluttered. I also include a ‘for’ loop at the end that adds additional information to the tooltip for each node. And, most complexly, I’ve used a color dictionary to make my nodes different colors based on the portion of the broader site that each page is a member of.

My network graph uses the traffic between two nodes to define the weight of the line. The nodes themselves are sized according to each page’s exit rate. The larger the node circle, the smaller the exit rate.

Note line 36; ‘.show_buttons(filter_=True)’. The first time you run your graph you can include this code to load an interactive menu of formatting options. This menu makes it easy to tweak the graph until you’re happy with its appearance. Then you can click ‘generate options’ to generate the options you’ve chosen as a dictionary which can be used to recreate your formatting in the future.

As you can see, I’ve set the options I chose and I invoke those on Line 7 of the code snippet above where I call the graph.

Call the visualization again with your pre-defined formatting and viola, an interactive visualization of your web traffic. You can use this to easily see the most common paths users travel through your site, which pages are driving exits, and where people are most likely to enter your page.

You can find the full code for this project saved as a notebook over on GitHub.

Enjoy!