Using Google Trends at Scale. How to use Google Trends when your... | by Carrie Fowle | Towards Data Science

Google Trends is a powerful measure of public interest in a topic, in other words, its hype; however, the way it is set up can make it difficult to use outside the simplest of applications. In this article, we dive into exactly what Google Trends is measuring and explore how to use it when your question exceeds the five-topic limit of the public facing tool.
Google Trends gives you a normalized measure of search volume for a given search term over a selected period of time. From their FAQs:
Each data point is divided by the total searches of the geography and time range it represents to compare relative popularity. Otherwise, places with the most search volume would always be ranked highest.
The resulting numbers are then scaled on a range of 0 to 100 based on a topic’s proportion to all searches on all topics. — Google Trends FAQs
That is to say, for each word in your search Google finds how much search volume in each region and time period your term had relative to all the searches in that region and time period. It combines all of these measures into a single measure of popularity, and then it scales the values across your topics, so the largest measure is set to 100.
In short: Google Trends doesn’t exactly tell you how many searches occurred for your topic, but it does give you a nice proxy.
As handy as Google Trends is for quickly taking the pulse of the internet, the structure of the service itself makes larger scale application difficult for two reasons:
Currently, the public-facing Google Trends site will not allow a query with more than five terms. This is obviously limiting if you want to explore any real world question. For example, early in the democratic primary you wouldn’t have been able to easily compare the popularity of all the major candidates.
The obvious reaction to this limit is, “well, I’ll just use multiple queries”. But as we mentioned earlier, all searches are scaled to the highest volume topic in your query, so two different queries aren’t comparable if they don’t have the same largest topic.
The last statement is the key:
if they don’t have the same largest topic
So long as two queries share the same most popular topic, they will be scaled in the same way, so the trends will be comparable. It follows then that if you want to compare more than five topics in Google Trends, you just need to include a control topic in each search.
For example, in a recent project, my team and I wanted to compare the popularity of Netflix’s and Amazon’s content. We went through the list of both platforms’ best content and pulled each title’s Google Trends measure relative to that of the word “France”.
Exactly how did we decide to use the word “France”?
It’s a bit like a recipe: you want something stable and safely higher than all of the terms you are interested in.
Using domain knowledge, I knew which titles would see particularly high search volume, so I looked for a word that was safely more popular than our most popular titles, but not so popular that I’d lose any real information on the less popular titles. For example, “Bitcoin” has such high search volume that many titles were normalized to zero (see what I mean here).
Yes.
For the streaming content project, I used the Python library pytrends, an unofficial API for Google Trends. Below you can find a quick snippet to get you started:
import pytrendskw_list = ##list of topics I wanted to searchtrends = dict()for i in kw_list:   ##build out query   pytrends.build_payload([i,'France'], cat=0,timeframe='today 5-y')      ##save trend to dictionary   trends[i] = pytrends.interest_over_time()[i]
You should note that if you are trying to do this for more than a handful of queries, you will hit Google’s DoS (Denial of Service) limits relatively quickly, so you will want to pause your loop for a few seconds between each query.
Google Trends is a good proxy for public interestTo make multiple queries comparable, include a control term that will be the largest in every searchUse pytrends to automate the process
Google Trends is a good proxy for public interest
To make multiple queries comparable, include a control term that will be the largest in every search
Use pytrends to automate the process
With this method, you can now answer big questions like: how popular are my products compared to those of my competitors, did a recent ad campaign impact interest in different product categories differently...
In short: the five word limit is no longer yours.